Unmanned aerial vehicle gimbal control method and platform for avoiding severe camera shaking

Abstract

This application provides a method and platform for controlling a drone gimbal to avoid severe camera shake. The method is applied to a drone gimbal system and includes a data acquisition unit and a control platform. The acquisition unit obtains attitude change information of the drone gimbal and environmental information of the drone gimbal. The control platform determines first feature information corresponding to the attitude change information and second feature information corresponding to the environmental information based on the attitude change information and environmental information. Based on the first and second feature information, the control platform determines a third feature vector of the drone gimbal, which is a feature vector obtained by fusing the first and second feature information. Then, based on the third feature vector, the control platform performs actions to stabilize the camera. This technical solution effectively avoids severe camera shake during drone flight.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV gimbal control method and platform for avoiding severe camera shake. Background Technology

[0002] During flight, drones are affected by various external factors, such as wind, airflow disturbances, and mechanical vibrations. These factors directly lead to instability in the drone's fuselage, which is then transmitted to the onboard camera system, causing camera shake. This camera shake problem is particularly pronounced in complex or harsh flight environments, such as mountainous areas, between tall buildings in cities, or in strong winds.

[0003] In existing technologies, a drone gimbal is a mechanical structure installed on a drone to support and stabilize camera equipment. It uses a series of hardware methods to reduce or eliminate vibrations and shaking during flight, thereby ensuring that the lens captures stable and clear images.

[0004] However, the above methods still cannot eliminate the technical problem of severe camera shaking. Summary of the Invention

[0005] This application provides a method and platform for controlling a drone gimbal to avoid severe camera shake, thereby solving the technical problem that existing technologies struggle to eliminate severe camera shake.

[0006] In a first aspect, embodiments of this application provide a method for controlling a drone gimbal to avoid severe camera shake, applied to a drone gimbal, the drone gimbal comprising: a data acquisition unit and a control platform, the method comprising:

[0007] The acquisition unit obtains the attitude change information of the UAV gimbal and the environmental information of the UAV gimbal.

[0008] The control platform determines, based on the attitude change information and the environmental information, the first feature information corresponding to the attitude change information and the second feature information corresponding to the environmental information related to the jitter of the UAV gimbal;

[0009] The control platform determines a third feature vector of the UAV gimbal based on the first feature information and the second feature information. The third feature vector is a feature vector obtained by fusing the first feature information and the second feature information.

[0010] The control platform controls the drone gimbal to perform actions based on the third feature vector in order to stabilize the camera.

[0011] In conjunction with the first aspect, in some embodiments, before determining the first feature information corresponding to the attitude change information and the second feature information corresponding to the environment information based on the attitude change information and the environment information, the method further includes:

[0012] The control platform performs noise reduction and calibration processing on the attitude change information and the environmental information respectively to obtain the processed attitude change information and the environmental information.

[0013] The control platform synchronizes the attitude change information and the environmental information based on timestamp information to obtain synchronized attitude change information and environmental information.

[0014] In conjunction with the first aspect, in some embodiments, the control platform determines, based on the attitude change information and the environmental information, first feature information corresponding to the attitude change information related to the UAV gimbal jitter and second feature information corresponding to the environmental information, including:

[0015] The control platform performs feature extraction on the attitude change information and the environmental information to obtain third feature information corresponding to the attitude change information and fourth feature information corresponding to the environmental information;

[0016] The control platform determines the first feature information and the second feature information from the third feature information and the fourth feature information respectively, based on the preset factor data of the drone gimbal vibration.

[0017] In conjunction with the first aspect, in some embodiments, the acquisition unit includes the following sensors: gyroscope, accelerometer, vision sensor, barometric pressure sensor, magnetic sensor, and infrared sensor;

[0018] Accordingly, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the vision sensor collects video stream data, the barometer collects barometer data, the magnetometer collects magnetic field data, and the infrared sensor collects infrared data.

[0019] Accordingly, the first feature information includes: a first sub-feature vector of the angular velocity data and a second sub-feature vector of the acceleration data; the second feature information includes: a third sub-feature vector of the video stream data, a fourth sub-feature vector of the air pressure data, a fifth sub-feature vector of the magnetic field data, and a sixth sub-feature vector of the infrared data.

[0020] In conjunction with the first aspect, in some embodiments, the control platform determines a third feature vector of the UAV gimbal based on the first feature information and the second feature information, including:

[0021] The control platform determines the contribution value of the sensor in the third feature vector for each sensor's corresponding sub-feature vector, based on the sub-feature vector, the weight coefficient corresponding to the sensor, the preset positive real number parameter of the control attenuation rate, the preset reference vector, and the preprocessing function corresponding to the sensor. The weight coefficient is determined based on the flight state and environmental conditions.

[0022] The control platform determines the third feature vector based on the contribution values ​​of each sensor.

[0023] In conjunction with the first aspect, in some embodiments, the control platform controls the UAV gimbal to perform actions based on the third feature vector to stabilize the camera, including:

[0024] The control platform determines the target flight information corresponding to the third feature vector according to a preset mapping relationship. The mapping relationship records at least one piece of flight information and the feature vector corresponding to the at least one piece of flight information.

[0025] The control platform controls the drone gimbal to perform flight maneuvers based on the target flight information in order to stabilize the camera.

[0026] In conjunction with the first aspect, in some embodiments, after the control platform controls the UAV gimbal to perform actions based on the third feature vector to stabilize the lens, the method further includes:

[0027] The acquisition unit obtains the flight status information of the UAV gimbal;

[0028] The control platform controls the UAV gimbal to perform actions based on the flight status information and the target flight information.

[0029] In conjunction with the first aspect, in some embodiments, after the control platform controls the UAV gimbal to perform actions based on the third feature vector to stabilize the lens, the method further includes:

[0030] The acquisition unit obtains the flight path setting command issued by the user;

[0031] The control platform corrects the flight trajectory corresponding to the flight path setting command based on the target flight information to obtain the corrected flight trajectory, and controls the UAV gimbal to fly based on the corrected flight trajectory.

[0032] Secondly, this application provides a control platform, including: a processor, and a memory and a communication interface communicatively connected to the processor;

[0033] The memory stores computer-executed instructions;

[0034] The processor executes computer execution instructions stored in the memory to implement the drone gimbal control method for avoiding severe camera shaking as described in any one of the first aspects.

[0035] Thirdly, this application provides a drone gimbal system, which includes: a data acquisition unit and a control platform;

[0036] The control platform and the acquisition unit are communicatively connected, and are used to execute the UAV gimbal control method for avoiding violent camera shaking as described in any one of the first aspects.

[0037] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which are executed by a processor using the unmanned aerial vehicle gimbal control method for avoiding severe camera shaking as described in the first aspect.

[0038] Fifthly, this application provides a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor of the unmanned aerial vehicle gimbal control method for avoiding severe camera shaking as described in the first aspect.

[0039] This application provides a method and platform for controlling a drone gimbal to avoid severe camera shake. The method, applied to a drone gimbal system, includes a data acquisition unit and a control platform. The acquisition unit obtains attitude change information of the drone gimbal and environmental information. The control platform, based on the attitude change information and environmental information, determines first feature information corresponding to the attitude change information and second feature information corresponding to the environmental information. Then, based on the first and second feature information, it determines a third feature vector of the drone gimbal, which is a feature vector obtained by fusing the first and second feature information. Finally, based on the third feature vector, the drone gimbal is controlled to perform actions to stabilize the camera. This technical solution, by comprehensively considering environmental and attitude change information, improves the accuracy and robustness of attitude estimation, thus avoiding severe camera shake during drone flight. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 This is a schematic diagram of the architecture provided for an embodiment of this application;

[0042] Figure 2 Flowchart of the method provided in the embodiments of this application Figure 1 ;

[0043] Figure 3 Flowchart of the method provided in the embodiments of this application Figure 2 ;

[0044] Figure 4 Flowchart of the method provided in the embodiments of this application Figure 3 ;

[0045] Figure 5 Flowchart of the method provided in the embodiments of this application Figure 4 ;

[0046] Figure 6 Flowchart of the method provided in the embodiments of this application Figure 5 ;

[0047] Figure 7 This is a schematic diagram of the structure of the device embodiment provided in this application;

[0048] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0049] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0050] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0051] With the rapid development of drone technology, its applications in aerial photography, monitoring, search and rescue, and other fields are becoming increasingly widespread, placing higher demands on the stability of the camera systems carried by drones. For example, during flight, drones are affected by various external factors, such as wind, airflow disturbances, and mechanical vibrations. These factors directly lead to instability in the drone's fuselage, which is then transmitted to the onboard camera system, causing lens shake. This lens shake problem is particularly prominent in complex or harsh flight environments, such as mountainous areas, between tall buildings in cities, or in strong winds. Some existing technologies rely on tightly integrating the camera system with the drone gimbal using mechanical structures to avoid factors affecting drone operation and causing camera shake. However, while these methods solve the shake problem between the drone gimbal and the camera system, serious shake issues still exist due to external forces causing the camera system to fail to accurately capture images.

[0052] To address the aforementioned problems, this application provides a method and platform for controlling a drone gimbal to avoid severe camera shake, thereby reducing such shake. Specifically, existing technologies only involve mechanical structural improvements and are still insufficient to address the overall drone vibration caused by external forces, which can lead to severe camera shake. Considering these issues, the inventors investigated the possibility of using multiple sensors to collect data on external forces affecting drone gimbal shake or jitter. By fusing and optimizing this data, a more accurate and comprehensive description of the external forces can be obtained. This avoids redundancy and conflicts in multi-source data while improving data reliability and accuracy. The processed results can then be used to control the drone, thus preventing severe camera shake. Based on this, the solution proposed in this application is presented.

[0053] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0054] Figure 1 This is a flowchart illustrating an embodiment of the UAV gimbal control method for avoiding severe camera shake provided in this application. Figure 1 As shown, this method is applied to an unmanned aerial vehicle (UAV) gimbal system, which includes a data acquisition unit and a control platform, and the data acquisition unit and the control platform are communicatively connected.

[0055] Optionally, the communication connection between the acquisition unit and the control platform can be a wired communication connection, a wireless connection, a low-power wide area network connection, a serial communication protocol connection, a message queue telemetry transmission, Bluetooth, etc., and this application does not make specific limitations.

[0056] In this embodiment, the drone gimbal control method for avoiding severe camera shake may include:

[0057] S101: The acquisition unit acquires the attitude change information of the UAV gimbal and the environmental information of the UAV gimbal.

[0058] In this step, in order to avoid severe camera shake caused by external factors during drone operation, it is first necessary to collect relevant information on the attitude change of the drone gimbal itself through the acquisition unit, namely attitude change information, and relevant information on the environment in which the drone gimbal is located, namely environmental information.

[0059] Optionally, the data acquisition unit may include the following sensors: gyroscope, accelerometer, vision sensor, barometric pressure sensor, magnetic sensor, and infrared sensor.

[0060] Among them, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the vision sensor collects video stream data, the barometer collects barometer data, the magnetometer collects magnetic field data, and the infrared sensor collects infrared data.

[0061] Specifically:

[0062] 1. The data collected by the barometric pressure sensor and the infrared sensor can be used to assess the drone's flight altitude and the temperature difference in the surrounding environment. This helps to understand wind force and other environmental factors that may affect flight stability.

[0063] 2. Visual sensors can analyze lens shake in real time, which helps to identify wind and other environmental factors that may affect flight stability, such as obstacles, terrain altitude, etc.

[0064] 3. Magnetic sensors are used for navigation and attitude estimation. In some cases, changes in magnetic field data may indirectly reflect changes in the attitude of the UAV, thus being related to jitter.

[0065] 4. Gyroscopes and accelerometers can directly reflect the attitude changes of the drone gimbal, such as acceleration and angular velocity.

[0066] S102: The control platform determines the first feature information corresponding to the attitude change information and the second feature information corresponding to the environmental information related to the UAV gimbal jitter based on the attitude change information and environmental information.

[0067] In this step, feature information corresponding to attitude change information related to UAV gimbal jitter and feature information corresponding to environmental information can be extracted, which are denoted as first feature information and second feature information, respectively.

[0068] For example, feature information directly related to gimbal jitter can be extracted from attitude jitter information, such as the amplitude, frequency, and duration of attitude changes. This feature information will be used as the first feature information for subsequent analysis and processing.

[0069] For example, feature information related to gimbal vibration is extracted from environmental information to identify environmental factors that may affect gimbal stability, such as wind speed, wind direction changes, terrain undulation, and obstacle distance. This feature information will be used as secondary feature information for subsequent analysis and processing.

[0070] Optionally, the first feature information includes: a first sub-feature vector of angular velocity data and a second sub-feature vector of acceleration data; the second feature information includes: a third sub-feature vector of video stream data, a fourth sub-feature vector of air pressure data, a fifth sub-feature vector of magnetic field data, and a sixth sub-feature vector of infrared data.

[0071] Optionally, step S102 can be implemented as follows:

[0072] Step 1: The control platform extracts features from the attitude change information and environmental information to obtain the third feature information corresponding to the attitude change information and the fourth feature information corresponding to the environmental information;

[0073] For example, the control platform extracts feature information from the attitude change information, which is denoted as the third feature information; the control platform extracts feature information from the environmental information, which is denoted as the fourth feature information.

[0074] Step 2: The control platform determines the first and second feature information from the third and fourth feature information respectively, based on the preset data of factors causing drone gimbal vibration.

[0075] For example, in the control platform, it is necessary to predefine the possible causes of drone gimbal jitter and their corresponding characteristic thresholds or patterns, i.e., the factor data of drone gimbal jitter. These preset factor data can be set based on experimental data, expert experience, or machine learning models.

[0076] For example, the control platform determines the first feature information from the third feature information based on the preset data of factors causing drone gimbal vibration.

[0077] Data related to attitude characteristics directly related to drone gimbal jitter, such as the amplitude of attitude changes (maximum / minimum difference), frequency (obtained through Fourier transform and other methods), acceleration (absolute value or rate of change of angular acceleration), and number of attitude abrupt changes.

[0078] For example, the control platform determines the second feature information from the fourth feature information based on the preset data of factors causing drone gimbal vibration.

[0079] Data on environmental factors affecting the stability of a drone gimbal, such as wind speed and its rate of change, air pressure fluctuations, temperature gradients, sudden changes in light intensity, and the approach speed of obstacles.

[0080] S103: The control platform determines the third feature vector of the UAV gimbal based on the first feature information and the second feature information. The third feature vector is the feature vector after fusing the first feature information and the second feature information.

[0081] In this step, after obtaining the feature information of the external and internal factors affecting the lens shake in the UAV gimbal, the first feature information and the second feature information are fused to obtain the feature vector affecting the lens shake.

[0082] In other words, the third feature vector aims to comprehensively reflect the current state of the drone and its interaction with the environment. It is a high-dimensional vector that integrates the drone's attitude change characteristics and the characteristics of its environment. Each element in this vector represents feature information related to a specific aspect of gimbal stability. By fusing these features, the current state of the drone and its potential impact on gimbal stability can be assessed more comprehensively. The fused feature vector helps in subsequent decision-making and optimization of control strategies to improve the shooting quality and stability of the drone gimbal.

[0083] S104: The control platform controls the drone gimbal to perform actions based on the third feature vector in order to stabilize the camera.

[0084] In this step, the control platform controls the drone gimbal to perform actions based on vectors that integrate the drone's attitude change characteristics and the characteristics of its environment, so as to keep the lens stable, which can improve shooting quality, expand application scenarios, and enhance user experience.

[0085] For example, features related to the drone's attitude stability, such as the rate of change of angular velocity and acceleration deviation, are extracted from the fused feature vector. Then, based on these features, the drone gimbal is controlled to perform actions to stabilize the camera.

[0086] In other words, the fused feature vectors are analyzed to assess the drone's jitter status. This may include calculating certain statistics of the feature vectors (such as norm, entropy, etc.), or inputting the feature vectors into another machine learning model for classification or regression. Based on the jitter assessment results, the drone's control strategy is adjusted to reduce or eliminate jitter.

[0087] This application provides a method for controlling a drone gimbal to avoid severe camera shake. The method is applied to a drone gimbal system and includes a data acquisition unit and a control platform. The acquisition unit obtains attitude change information of the drone gimbal and environmental information. The control platform determines first feature information corresponding to the attitude change information and second feature information corresponding to the environmental information based on the attitude change information and environmental information. Then, based on the first and second feature information, it determines a third feature vector of the drone gimbal, which is a feature vector obtained by fusing the first and second feature information. Finally, based on the third feature vector, the drone gimbal is controlled to perform actions to stabilize the camera. This technical solution improves the accuracy and robustness of attitude estimation by comprehensively considering environmental and attitude change information, thus avoiding severe camera shake during drone flight.

[0088] Based on the above embodiments, Figure 2 This is a flowchart illustrating Embodiment 2 of the UAV gimbal control method for avoiding severe camera shake provided in this application. Figure 2 As shown, prior to S102 in the above embodiment, the method may also include the following:

[0089] S201: The control platform performs noise reduction and calibration processing on the attitude change information and environmental information respectively to obtain the processed attitude change information and environmental information;

[0090] In this step, noise reduction and calibration are key steps to improve data quality, which can increase the accuracy and efficiency of subsequent data processing.

[0091] Optionally, in the noise reduction process:

[0092] 1. For angular velocity data, acceleration data, and magnetic field data, the following formula can be used (taking angular velocity as an example):

[0093] y2 = y1*a + y1(1-a)

[0094] Where y1 represents angular velocity data, a represents the filtering coefficient, and y2 represents the filtered acceleration data. The filtering coefficient is determined by considering the following factors: the frequency characteristics, waveform shape, and real-time response of the angular velocity signal; the filtering methods for acceleration data and magnetic field data are similar.

[0095] 2. For video stream data, a sorting algorithm can be used to replace the value of each pixel with the median of the values ​​of its neighboring pixels, followed by a weighted average. The weights are given by a Gaussian function, and the formula can be as follows:

[0096]

[0097] Where g(x,y) is the output pixel value of the image at point (x,y), f(x,y) is the input pixel value of the image at point (x,y), s is the average pooling window, and M is the number of pixels in s.

[0098] 3. For air pressure data, calculate the average value of the data over a period of time to smooth the signal. The following formula can be used:

[0099]

[0100] Where y(n) is the filtered air pressure data, x(n) is the air pressure data before filtering, and N is the size of the sliding window.

[0101] 4. For infrared data, for each pixel in the infrared image, sort the gray values ​​of all pixels in the neighborhood window centered on that pixel, and select the gray value in the middle of the sorted range as the new value after filtering for that pixel.

[0102] Optionally, during the calibration process:

[0103] 1. For angular velocity data, zero-point offset calibration can be removed;

[0104] The calibrated angular velocity data is obtained by subtracting the zero-point offset from the angular velocity data. The zero-point offset can be based on the value measured under static conditions.

[0105] For example, it can be obtained by comparing the angular velocity of the drone platform during its flight with the theoretical angular velocity.

[0106] 2. For acceleration data, the influence of gravitational acceleration can be removed; taking a horizontal plane as an example, the same applies to other planar solutions.

[0107]

[0108] Where l is the corrected acceleration, q and w are the two-dimensional accelerations in the horizontal plane dimension before correction, and g is the gravitational acceleration.

[0109] 3. For video stream data, correction can be performed based on image brightness to avoid errors in subsequent feature extraction. The formula that can be used is:

[0110] r2(x, y) = t * r1(x, y)

[0111] Where r2(x,y) is the brightness of point (x,y) after correction, t is the brightness parameter, and r1(x,y) is the brightness of point (x,y) before correction; the value of t is related to the brightness of the actual scene and the brightness of the acquired video stream.

[0112] 4. Regarding the air pressure data, adjustments can be made based on local atmospheric pressure and temperature.

[0113] Temperature can be determined based on infrared data or a temperature sensor.

[0114] 5. For magnetic field data, the formula that can be used is:

[0115] S2=b+R*S1

[0116] Where S2 is the corrected magnetic field data, S1 is the original magnetic field data, R is the non-orthogonality matrix used to correct the magnetic field data, and b is the offset vector.

[0117] 6. For infrared data, the formula that can be used is:

[0118] F2=z+j*F1

[0119] Where F2 is the corrected infrared data, F1 is the uncorrected infrared data, j is the gain coefficient, and z is the offset compensation.

[0120] S202: The control platform synchronizes attitude change information and environmental information based on timestamp information to obtain synchronized attitude change information and environmental information.

[0121] In this step, the attitude sensors on the UAV automatically add a timestamp each time they collect attitude change information (such as pitch angle, roll angle, yaw angle, etc.). This timestamp records the specific time when the data was collected. Similarly, the environmental sensors also add a timestamp when collecting environmental information. This timestamp records the specific time when the environmental data was collected.

[0122] Since different sensors may have different sampling frequencies, the timestamps of environmental information may not be completely consistent with the timestamps of attitude change information. Therefore, it is necessary to synchronize attitude change information and environmental information based on timestamps.

[0123] For example, the implementation can be as follows:

[0124] 1. During the system startup or initialization phase, the timestamps of all sensors need to be calibrated to ensure that the time deviation between them is as small as possible. The calibration process may involve using high-precision time synchronization methods, such as using Global Positioning System (GPS) time or other high-precision time sources or built-in time calibration algorithms for drones.

[0125] 2. For sensors with different sampling frequencies, the control platform can synchronize data through timestamp interpolation or extrapolation;

[0126] For example, interpolation is a method for estimating unknown data points between two known data points. If the timestamp of some environmental data does not match the timestamp of the attitude data, an interpolation algorithm can be used to find the closest attitude data near that timestamp. Extrapolation is based on the trend of known data points to predict future or past data points. In some cases, if the timestamp of the environmental data is significantly ahead or behind the timestamp of the attitude data, an extrapolation algorithm may be needed to estimate the corresponding attitude data.

[0127] 3. The control platform can use advanced timestamp synchronization algorithms (such as Kalman filtering, particle filtering, etc.) to further improve the accuracy and real-time performance of data synchronization. These algorithms can comprehensively consider data and timestamp information from multiple sensors, and obtain more accurate and reliable synchronization data through data fusion and state estimation.

[0128] The UAV gimbal control method for avoiding severe camera shake provided in this application involves a control platform that performs denoising and calibration processing on attitude change information and environmental information respectively, obtaining processed attitude change information and environmental information. The control platform then synchronizes the attitude change information and environmental information based on timestamp information, obtaining synchronized attitude change information and environmental information. This technical solution, by performing denoising and calibration processing on attitude change information and environmental information separately, as well as synchronization, can increase the accuracy of the collected data, thereby improving the effect of preventing severe camera shake.

[0129] Based on the above embodiments, Figure 3 This is a flowchart illustrating Embodiment 3 of the UAV gimbal control method for avoiding severe camera shake provided in this application. Figure 3 As shown, S103 of the above embodiment can have the following:

[0130] S301: For each sensor's corresponding sub-feature vector, the control platform determines the sensor's contribution value in the third feature vector based on the sub-feature vector, the sensor's corresponding weight coefficient, the preset positive real number parameter of the control attenuation rate, the preset reference vector, and the sensor's corresponding preprocessing function.

[0131] The weighting coefficients are determined based on both flight status and environmental conditions.

[0132] For example, regarding weighting coefficients, when strong winds or mechanical failures are detected, the weighting coefficients of the acceleration sensor and angular velocity sensor can be appropriately increased; when the flight environment is relatively stable, the weighting coefficients of the barometric pressure sensor, magnetic field sensor, infrared sensor, and visual sensor can be increased.

[0133] Optionally, for any given sensor, the formula for determining the sensor's contribution value in the third feature vector, based on the sub-feature vector, the sensor's corresponding weight coefficient, the preset positive real parameter for controlling attenuation rate, the preset reference vector, and the sensor's corresponding preprocessing function, can be:

[0134]

[0135] Where h is the number of sensors, c i For the sub-feature vector (such as angular velocity, angular acceleration, etc.) involved by the i-th sensor, v i Let p be the weighting coefficient of the i-th sensor, p be a positive real parameter controlling the attenuation rate, m be a preset reference vector, and e(c i ) is the preprocessing function corresponding to the i-th sensor, c j Let z(c) be the sub-feature vector involved in the j-th sensor. i v i ) represents the contribution value of the sensor in the third feature vector.

[0136] It should be understood that the sum of the weighting coefficients corresponding to each sensor is 1.

[0137] S302: The control platform determines the third feature vector based on the contribution values ​​of each sensor.

[0138] In this step, in the UAV gimbal system, in order to comprehensively evaluate the jitter impact caused by information from different sensors, the contribution values ​​obtained from different sensors can be summed or subjected to more complex fusion processing to generate a feature vector that reflects the jitter state of the UAV.

[0139] Method 1: Determine the third feature vector based on the contribution values ​​of each sensor. This can be achieved by summing the contribution values ​​of each sensor to obtain the third feature vector.

[0140] Method 2 considers the different importance of each sensor in jitter assessment and achieves weighted summation by assigning different weights to the contribution values ​​of each sensor. These weights can be determined based on experimental data, expert knowledge, or machine learning algorithms.

[0141] Method 3 uses more complex fusion methods, such as Kalman filters, particle filters, or machine learning-based fusion algorithms (such as neural networks, support vector machines, etc.), to more accurately integrate the contributions from different sensors.

[0142] The UAV gimbal control method for avoiding severe camera shake provided in this application involves a control platform determining the sensor's contribution value in a third feature vector based on the sub-feature vector corresponding to each sensor, the sensor's corresponding weight coefficient, a preset positive real parameter for control attenuation speed, a preset reference vector, and the sensor's corresponding preprocessing function. The weight coefficient is determined jointly based on flight state and environmental conditions. The control platform then determines the third feature vector based on the contribution values ​​of each sensor. In this technical solution, weight coefficients are assigned to each sensor with flight state and environmental conditions as references to more accurately correct the contributions of each sensor and increase the accuracy of the fused feature vector.

[0143] Based on the above embodiments, Figure 4 This is a flowchart illustrating Embodiment 4 of the UAV gimbal control method for avoiding severe camera shake provided in this application. Figure 4 As shown, S104 of the above embodiment can have the following:

[0144] S401: The control platform determines the target flight information corresponding to the third feature vector according to the preset mapping relationship. The mapping relationship records at least one piece of flight information and at least one feature vector corresponding to the flight information.

[0145] In this step, a mapping relationship from the fused feature vector to flight information can be pre-established or learned in the control platform. This allows for the determination of how the fused feature vector can be directly mapped to control the UAV's flight information, simplifying the subsequent execution process and improving response speed.

[0146] For example, historical flight data and corresponding feature vectors can be used to train a machine learning model. The model's learning objective is to find the optimal mapping between the fused feature vectors and the target flight information. The fused feature vectors are then input into the trained machine learning model, which outputs predicted target flight information based on the learned mapping.

[0147] S402: The control platform controls the drone gimbal to perform flight maneuvers based on the target flight information in order to stabilize the camera.

[0148] In this step, a control strategy for the UAV is formulated based on the predicted target flight information, which may include adjusting flight speed, changing flight direction, adjusting attitude, etc.

[0149] In addition, new flight information can be acquired in real time through the drone's sensor system, and the above process can be repeated to achieve continuous monitoring and adjustment of the drone's flight status.

[0150] Furthermore, based on flight performance feedback, feature extraction methods, fusion algorithms, and machine learning models are continuously optimized to improve the accuracy of target flight information and the stability of the control system.

[0151] The UAV gimbal control method provided in this application for avoiding severe camera shake involves a control platform determining the target flight information corresponding to a third feature vector based on a preset mapping relationship. The mapping relationship records at least one piece of flight information and at least one corresponding feature vector. The control platform then controls the UAV gimbal to perform flight maneuvers based on the target flight information to stabilize the camera. This technical solution, after obtaining the target flight information, can directly determine the flight information to be executed by the UAV gimbal based on the preset mapping relationship. Its execution process is simple, improving the real-time adjustment efficiency for avoiding severe camera shake.

[0152] Based on the above embodiments, Figure 5 This is a flowchart illustrating Embodiment 5 of the UAV gimbal control method for avoiding severe camera shake provided in this application. Figure 5 As shown, this method can also include the following:

[0153] S501: The acquisition unit obtains flight status information of the UAV gimbal;

[0154] In this step, the drone gimbal is usually equipped with a variety of sensors, such as accelerometers, gyroscopes, magnetometers, and GPS, to collect the drone's flight status information in real time.

[0155] Furthermore, the collected raw data needs to be preprocessed and analyzed to extract useful flight status information. This includes steps such as data denoising, calibration, and time synchronization to ensure the accuracy and reliability of the data.

[0156] S502: The control platform controls the UAV gimbal to perform actions based on flight status information and target flight information.

[0157] In this step, based on the target flight information and flight status information of the drone shaking, the control platform needs to determine the control target of the drone gimbal, such as the adjustment of the lens, the drone wings, rotation speed, angle, etc.

[0158] The control method can be as follows: the control platform generates corresponding control commands and sends the control commands to the controller of the drone gimbal (or it can be itself) to execute corresponding actions, such as adjusting attitude and stabilizing the image.

[0159] The UAV gimbal control method provided in this application to avoid severe camera shake involves a control platform acquisition unit obtaining flight status information of the UAV gimbal, and the control platform controlling the UAV gimbal to perform actions based on the flight status information and target flight information. This technical solution can provide corrections for subsequent UAV flight to counteract severe camera shake caused by external forces.

[0160] Based on the above embodiments, Figure 6 This is a flowchart illustrating Embodiment Six of the UAV gimbal control method for avoiding severe camera shake provided in this application. Figure 6 As shown, this method can also include the following:

[0161] S601: The acquisition unit acquires the flight path setting command issued by the user;

[0162] For example, users set flight paths using input devices such as drone remote controllers, mobile applications, tablets, or dedicated flight control software. These devices typically offer intuitive user interfaces, such as touchscreens, buttons, or joysticks.

[0163] Afterward, users can set the flight path by drawing a path on a map, entering coordinates, selecting a preset path template, or directly using gesture control. The user's input commands are first received by the flight control software or application. The software has an internal parsing mechanism to recognize the various user commands and convert them into a format that the drone can understand. After parsing the commands, the software may use path planning algorithms to optimize the path to ensure that the drone flies along the most efficient and safest route.

[0164] Furthermore, the optimized flight path instructions are transmitted to the UAV via wireless communication technology. This step typically requires the UAV's data acquisition unit to establish a stable communication connection with the input device, meaning the data acquisition unit receives the flight path setting instructions from the input device.

[0165] S602: The control platform corrects the flight trajectory corresponding to the flight path setting command based on the target flight information, obtains the corrected flight trajectory, and controls the UAV gimbal to fly based on the corrected flight trajectory.

[0166] In this step, the control platform first parses the flight path setting command input by the user to obtain parameters such as path point coordinates, flight altitude, and speed.

[0167] Since target flight information reflects environmental and safety assessments, such as the presence of no-fly zones, obstacles, or severe weather along the path, the flight trajectory corresponding to the flight path setting command can be corrected based on real-time acquired target flight information to ensure the safety and stability of UAV flight.

[0168] Optionally, the flight trajectory can be corrected based on the target flight information, and a trajectory correction algorithm can be applied to optimize the original flight trajectory. The correction algorithm may include path smoothing, obstacle avoidance strategies, energy consumption minimization, etc., and may take into account the dynamic characteristics of the UAV, flight constraints (such as maximum speed, maximum rate of climb, etc.), and external environmental factors (such as wind field, temperature, etc.).

[0169] After processing by the correction algorithm, a corrected flight trajectory is generated. This trajectory should satisfy all flight constraints and optimize flight performance and safety as much as possible.

[0170] After the above steps, the corrected flight trajectory can be the desired flight information, but in reality, there may be a small error. This application provides the following control strategy to adjust the flight situation in real time:

[0171]

[0172] Where u(u) is the control output at time u, such as the thrust and rudder angle of the UAV; e(u) is the error signal at time u, i.e., the difference between the expected flight information and the actual flight information; K p K i K d The gains of each of the three factors can be adjusted based on the actual situation of the UAV gimbal system.

[0173] The UAV gimbal control method provided in this application to avoid severe camera shake involves a data acquisition unit acquiring flight path setting commands issued by the user, and a control platform correcting the flight trajectory corresponding to the flight path setting commands based on target flight information to obtain a corrected flight trajectory. Based on this corrected flight trajectory, the UAV gimbal is then controlled to fly. This technical solution achieves pre-emptive correction of flight actions along the path after the user inputs the path, thus avoiding severe camera shake during the subsequent path.

[0174] Figure 7 This is a schematic diagram of the structure of the control platform provided in the embodiments of this application, as shown below. Figure 7 As shown, the control platform 700 includes: a processor 702, a memory 701 communicatively connected to the processor 702, and a communication interface 703;

[0175] Memory 701 stores instructions executed by the computer;

[0176] The processor 702 executes computer execution instructions stored in the memory 701 to implement the UAV gimbal control method for avoiding severe camera shaking in any method embodiment.

[0177] Figure 8 This is a schematic diagram of the structure of the UAV gimbal system provided in the embodiments of this application, as shown below. Figure 8 As shown, the UAV gimbal system 800 includes a control platform 801 and a data acquisition unit 802. The control platform 801 and the data acquisition unit 802 are communicatively connected and are used to execute the UAV gimbal control method for avoiding severe camera shaking in any of the foregoing embodiments.

[0178] The acquisition unit 802 is used to acquire attitude change information of the UAV gimbal and environmental information of the UAV gimbal.

[0179] Specifically, the acquisition unit 802 may include the following sensors: gyroscope, accelerometer, vision sensor, barometric pressure sensor, magnetic sensor, and infrared sensor.

[0180] Among them, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the vision sensor collects video stream data, the barometer collects barometer data, the magnetometer collects magnetic field data, and the infrared sensor collects infrared data.

[0181] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to perform the drone gimbal control method for avoiding severe camera shaking provided in the various embodiments described above.

[0182] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0183] Optionally, a readable storage medium can be coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Alternatively, the readable storage medium can be an integral part of the processor. Both the processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components within the device.

[0184] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solutions provided in any of the above method embodiments.

[0185] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates an "or" relationship between the preceding and following related objects; in formulas, the character " / " indicates a "division" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0186] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. In the embodiments of this application, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0187] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0188] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for controlling a drone gimbal to avoid severe camera shake, characterized in that, Applied to a drone gimbal system, the drone gimbal system comprising: a data acquisition unit and a control platform, the method comprising: The acquisition unit obtains the attitude change information of the UAV gimbal and the environmental information of the UAV gimbal. The control platform determines, based on the attitude change information and the environmental information, the first feature information corresponding to the attitude change information and the second feature information corresponding to the environmental information related to the jitter of the UAV gimbal; The control platform determines a third feature vector of the UAV gimbal based on the first feature information and the second feature information. The third feature vector is a feature vector obtained by fusing the first feature information and the second feature information. The control platform controls the drone gimbal to perform actions based on the third feature vector in order to stabilize the camera. The acquisition unit includes the following sensors: gyroscope, accelerometer, vision sensor, barometric pressure sensor, magnetic sensor, and infrared sensor; Accordingly, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the vision sensor collects video stream data, the barometer collects barometer data, the magnetometer collects magnetic field data, and the infrared sensor collects infrared data. Accordingly, the first feature information includes: a first sub-feature vector of the angular velocity data and a second sub-feature vector of the acceleration data; the second feature information includes: a third sub-feature vector of the video stream data, a fourth sub-feature vector of the air pressure data, a fifth sub-feature vector of the magnetic field data, and a sixth sub-feature vector of the infrared data; The control platform determines the third feature vector of the UAV gimbal based on the first feature information and the second feature information, including: The control platform determines the contribution value of the sensor in the third feature vector for each sensor's corresponding sub-feature vector, based on the sub-feature vector, the weight coefficient corresponding to the sensor, the preset positive real number parameter of the control attenuation rate, the preset reference vector, and the preprocessing function corresponding to the sensor. The weight coefficient is determined based on the flight state and environmental conditions. The control platform determines the third feature vector based on the contribution values ​​of each sensor; The formula for calculating the contribution value of the sensor in the third feature vector is as follows: ， in, For the number of sensors, For the first Sub-feature vectors involved in each sensor, For the first The weighting coefficients of each sensor A positive real-valued parameter to control the decay rate, where m is a preset reference vector. For the first Preprocessing functions corresponding to each sensor, For the first Sub-feature vectors involved in each sensor, The contribution value of the sensor mentioned in the third feature vector; Before determining the first feature information corresponding to the posture change information and the second feature information corresponding to the environment information based on the posture change information and the environment information, the method further includes: the control platform performs noise reduction processing and calibration processing on the posture change information and the environment information respectively to obtain the processed posture change information and the environment information, and synchronizes the posture change information and the environment information based on timestamp information to obtain the synchronized posture change information and the environment information.

2. The method according to claim 1, characterized in that, The control platform determines, based on the attitude change information and the environmental information, first feature information corresponding to the attitude change information related to the UAV gimbal jitter and second feature information corresponding to the environmental information, including: The control platform performs feature extraction on the attitude change information and the environmental information to obtain third feature information corresponding to the attitude change information and fourth feature information corresponding to the environmental information; The control platform determines the first feature information and the second feature information from the third feature information and the fourth feature information respectively, based on the preset factor data of the drone gimbal vibration.

3. The method according to claim 1 or 2, characterized in that, The control platform controls the drone gimbal to perform actions based on the third feature vector to stabilize the camera, including: The control platform determines the target flight information corresponding to the third feature vector according to a preset mapping relationship. The mapping relationship records at least one piece of flight information and the feature vector corresponding to the at least one piece of flight information. The control platform controls the drone gimbal to perform flight maneuvers based on the target flight information in order to stabilize the camera.

4. The method according to claim 3, characterized in that, After the control platform controls the UAV gimbal to perform actions based on the third feature vector to stabilize the lens, the method further includes: The acquisition unit obtains the flight status information of the UAV gimbal; The control platform controls the UAV gimbal to perform actions based on the flight status information and the target flight information.

5. The method according to claim 3, characterized in that, After the control platform controls the UAV gimbal to perform actions based on the third feature vector to stabilize the lens, the method further includes: The acquisition unit obtains the flight path setting command issued by the user; The control platform corrects the flight trajectory corresponding to the flight path setting command based on the target flight information to obtain the corrected flight trajectory, and controls the UAV gimbal to fly based on the corrected flight trajectory.

6. A control platform, characterized in that, include: A processor, and a memory and a communication interface communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the drone gimbal control method for avoiding severe camera shaking as described in any one of claims 1 to 5.

7. A UAV gimbal system, characterized in that, The UAV gimbal system includes: a data acquisition unit and a control platform; The control platform and the acquisition unit are communicatively connected and are used to execute the UAV gimbal control method for avoiding severe lens shaking as described in any one of claims 1 to 5.

Images