Unmanned aerial vehicle holder control method and platform for avoiding violent shaking of lens

Abstract

The invention provides an unmanned aerial vehicle holder control method and platform for avoiding violent shaking of a lens, the method is applied to an unmanned aerial vehicle holder system, the unmanned aerial vehicle holder system comprises an acquisition unit and a control platform, and the method comprises the following steps: acquiring attitude change information of an unmanned aerial vehicle holder and information of an environment where the unmanned aerial vehicle holder is located through the acquisition unit; the control platform determines first feature information corresponding to the attitude change information related to the shake of the unmanned aerial vehicle holder and second feature information corresponding to the environment information according to the attitude change information and the environment information, and determines a third feature vector of the unmanned aerial vehicle holder according to the first feature information and the second feature information, the third feature vector is a feature vector obtained after the first feature information and the second feature information are fused, and then the unmanned aerial vehicle holder is controlled to act according to the third feature vector, so that the lens is stable. According to the technical scheme, the problem that the lens shakes violently when the unmanned aerial vehicle flies is solved.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to a method and platform for controlling a pan / tilt platform of an unmanned aerial vehicle for preventing severe shaking of a lens. Background Art

[0002] During the flight, drones are affected by various external factors, such as wind, air flow disturbance, mechanical vibration, etc. These factors will directly lead to the instability of the drone body, which will then be transmitted to the camera system on board, causing the lens to shake. The problem of lens shaking is particularly prominent in complex or harsh flight environments, such as mountainous areas, between high-rise buildings in cities, or in strong winds.

[0003] In the prior art, a drone gimbal is a mechanical structure installed on a drone to support and stabilize the camera equipment. It uses a series of hardware means to reduce or eliminate the vibration and shaking generated during the flight, thereby ensuring that the lens captures stable and clear images.

[0004] However, the above-mentioned method still has difficulty in eliminating the technical problem of severe lens shaking. Summary of the invention

[0005] The present application provides a drone gimbal control method and platform for preventing severe lens shaking, so as to solve the technical problem that it is difficult to eliminate severe lens shaking in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for controlling a drone gimbal to prevent severe shaking of a lens, which is applied to a drone gimbal. The drone gimbal includes: a collection unit and a control platform. The method includes:

[0007] The acquisition unit acquires the posture change information of the drone gimbal and the environment information of the drone gimbal;

[0008] The control platform determines, according to the attitude change information and the environmental information, first feature information corresponding to the attitude change information related to the jitter of the UAV gimbal and second feature information corresponding to the environmental information;

[0009] The control platform determines a third feature vector of the UAV gimbal according to the first feature information and the second feature information, wherein the third feature vector is a feature vector obtained by fusing the first feature information and the second feature information;

[0010] The control platform controls the drone gimbal to move according to the third eigenvector to stabilize the lens.

[0011] In combination with the first aspect, in some embodiments, before determining, according to the posture change information and the environmental information, first feature information corresponding to the posture change information and second feature information corresponding to the environmental information, the method further includes:

[0012] The control platform performs denoising and calibration processing on the posture change information and the environmental information respectively to obtain the processed posture change information and the environmental information;

[0013] The control platform synchronizes the posture change information and the environmental information based on the timestamp information to obtain the synchronized posture change information and the environmental information.

[0014] In combination with the first aspect, in some embodiments, the control platform determines, according to the attitude change information and the environmental information, first feature information corresponding to the attitude change information related to the jitter of the drone gimbal and second feature information corresponding to the environmental information, including:

[0015] The control platform performs feature extraction on the posture change information and the environmental information to obtain third feature information corresponding to the posture change information and fourth feature information corresponding to the environmental information;

[0016] The control platform determines the first feature information and the second feature information from the third feature information and the fourth feature information respectively according to the preset factor data of the jitter of the drone gimbal.

[0017] In combination with the first aspect, in some embodiments, the acquisition unit includes the following sensors: a gyroscope, an accelerometer, a visual sensor, an air pressure sensor, a magnetic sensor, and an infrared sensor;

[0018] Correspondingly, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the visual sensor collects video stream data, the air pressure sensor collects air pressure data, the magnetic sensor collects magnetic field data, and the infrared sensor collects infrared data;

[0019] Correspondingly, the first feature information includes: the first sub-feature vector of the angular velocity data, the second sub-feature vector of the acceleration data; the second feature information includes: the third sub-feature vector of the video stream data, the fourth sub-feature vector of the air pressure data, the fifth sub-feature vector of the magnetic field data, and the sixth sub-feature vector of the infrared data.

[0020] In combination with the first aspect, in some embodiments, the control platform determines a third feature vector of the drone gimbal according to the first feature information and the second feature information, including:

[0021] The control platform determines, for each sensor corresponding to a sub-eigenvector, a weight coefficient corresponding to the sensor, a preset positive real number parameter for controlling the attenuation speed, a preset reference vector, and a preprocessing function corresponding to the sensor, wherein the weight coefficient is determined based on the flight state and the environmental conditions;

[0022] The control platform determines the third eigenvector according to the contribution values ​​of each sensor.

[0023] In combination with the first aspect, in some embodiments, the control platform controls the drone gimbal to move according to the third eigenvector to stabilize the lens, including:

[0024] The control platform determines the target flight information corresponding to the third feature vector according to a preset mapping relationship, wherein the mapping relationship records at least one flight information and a feature vector corresponding to the at least one flight information;

[0025] The control platform controls the UAV gimbal to perform flight actions according to the target flight information to stabilize the lens.

[0026] In combination with the first aspect, in some embodiments, after the control platform controls the drone gimbal to perform an action according to the third eigenvector to stabilize the lens, the method further includes:

[0027] The acquisition unit obtains the flight status information of the UAV gimbal;

[0028] The control platform controls the UAV gimbal to perform actions according to the flight status information and the target flight information.

[0029] In combination with the first aspect, in some embodiments, after the control platform controls the drone gimbal to perform an action according to the third eigenvector to stabilize the lens, the method further includes:

[0030] The acquisition unit acquires a flight path setting instruction issued by a user;

[0031] The control platform corrects the flight trajectory corresponding to the flight path setting instruction according to the target flight information to obtain a corrected flight trajectory, and controls the flight of the UAV gimbal based on the corrected flight trajectory.

[0032] In a second aspect, the present application provides a control platform, comprising: a processor, and a memory and a communication interface communicatively connected to the processor;

[0033] The memory stores computer-executable instructions;

[0034] The processor executes the computer-executable instructions stored in the memory to implement any of the drone gimbal control methods for avoiding severe lens shaking as described in the first aspect.

[0035] In a third aspect, the present application provides a drone gimbal system, the drone gimbal system comprising: a collection unit and a control platform;

[0036] The control platform is communicatively connected with the acquisition unit, and is used to execute any one of the UAV gimbal control methods for avoiding severe lens shaking as described in the first aspect.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the drone gimbal control method for avoiding severe lens shaking as described in any one of the first aspects.

[0038] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the drone gimbal control method for avoiding severe shaking of the lens as described in any one of the first aspects.

[0039] The application provides a drone gimbal control method and platform for avoiding severe lens shaking. The method is applied to a drone gimbal system and includes: an acquisition unit and a control platform. The method obtains the attitude change information of the drone gimbal and the environmental information of the drone gimbal through the acquisition unit; the control platform determines the first feature information corresponding to the attitude change information related to the jitter of the drone gimbal and the second feature information corresponding to the environmental information according to the attitude change information and the environmental information, and determines the third feature vector of the drone gimbal according to the first feature information and the second feature information. The third feature vector is the feature vector after the first feature information and the second feature information are fused. Then, according to the third feature vector, the drone gimbal is controlled to move to stabilize the lens. In this technical solution, the accuracy and robustness of attitude estimation are improved by comprehensively considering the environmental information and attitude change information, so as to avoid the problem of severe lens shaking when the drone is flying. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A schematic diagram of the architecture provided for an embodiment of the present application;

[0042] Figure 2 Schematic diagram of the process of the method provided in the embodiment of the present application Figure 1 ;

[0043] Figure 3 Schematic diagram of the process of the method provided in the embodiment of the present application Figure 2 ;

[0044] Figure 4 Schematic diagram of the process of the method provided in the embodiment of the present application Figure 3 ;

[0045] Figure 5 Schematic diagram of the process of the method provided in the embodiment of the present application Figure 4 ;

[0046] Figure 6 Schematic diagram of the process of the method provided in the embodiment of the present application Figure 5 ;

[0047] Figure 7 A schematic diagram of the structure of an embodiment of the device provided in the embodiment of the present application;

[0048] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0049] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0050] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0051] With the rapid development of drone technology, its application in aerial photography, monitoring, search and rescue and other fields is becoming more and more extensive, and the stability requirements for the camera system carried by drones are also getting higher and higher. For example, during the flight of the drone, it will be affected by various external factors, such as wind, airflow disturbance, mechanical vibration, etc. These factors will directly lead to the instability of the drone fuselage, which will then be transmitted to the camera system carried, causing the lens to shake. Especially in complex or harsh flight environments, such as mountainous areas, between high-rise buildings in cities, or in strong winds, the problem of lens shaking is particularly prominent. Some methods in the prior art are based on the use of some mechanical structures to closely combine the camera system and the drone gimbal to avoid factors that affect the operation of the drone and cause the camera system to shake. However, although the above method solves the shaking problem between the drone gimbal and the camera system, the camera system cannot accurately capture the picture due to external forces and other factors, and there is still a serious shaking problem.

[0052] In response to the above problems, the present application provides a drone gimbal control method and platform that avoids severe shaking of the lens, thereby reducing severe shaking of the lens. Specifically, the existing technology only has improvements in mechanical structure, and still cannot adapt to the shaking of the entire drone caused by external factors, which leads to severe shaking of the lens. In view of these problems, the inventors have studied if it is possible to use a variety of different sensors to collect relevant data on external factors that affect the shaking or shaking of the drone gimbal, and fuse and optimize these data, so that a more accurate and comprehensive description of the external factors can be obtained. While avoiding the redundancy and conflict of multi-source data, the reliability and accuracy of the data can also be improved, and the drone can be controlled based on the processing results, thereby avoiding severe shaking of the lens. Based on this, the present application scheme is proposed.

[0053] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0054] Figure 1 The flowchart of the first embodiment of the method for controlling the gimbal of a drone to prevent severe shaking of the lens provided in the embodiment of the present application is as follows: Figure 1 As shown, the method is applied to a UAV gimbal system, wherein the UAV gimbal system includes a collection unit and a control platform, and the collection unit and the control platform are communicatively connected.

[0055] Optionally, the communication connection method between the acquisition unit and the control platform can be wired communication connection, wireless connection, low-power wide area network connection, serial communication protocol connection, message queue telemetry transmission, Bluetooth, etc., which is not specifically limited in this application.

[0056] In this embodiment, the drone gimbal control method for preventing severe lens shaking may include:

[0057] S101: The collection unit obtains the attitude change information of the drone gimbal and the environment information of the drone gimbal;

[0058] In this step, in order to avoid severe shaking of the lens due to external factors during the operation of the drone, it is first necessary to collect relevant information about the posture changes of the drone gimbal itself, that is, the posture change information, and collect relevant information about the environment in which the drone gimbal is located, that is, the environmental information, through the acquisition unit.

[0059] Optionally, the acquisition unit includes the following sensors: a gyroscope, an accelerometer, a visual sensor, an air pressure sensor, a magnetic sensor, and an infrared sensor.

[0060] Among them, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the visual sensor collects video stream data, the air pressure sensor collects air pressure data, the magnetic sensor collects magnetic field data, and the infrared sensor collects infrared data.

[0061] Specific:

[0062] 1. The data collected by the air pressure sensor and infrared sensor respectively, that is, the air pressure data and infrared data can be used to evaluate the flight altitude of the drone and the temperature difference of the surrounding environment, which helps to understand the wind force and other environmental factors that may affect flight stability.

[0063] 2. Visual sensors can analyze the jitter of the lens in real time, which helps to understand wind speed and other environmental factors that may affect flight stability, such as obstacles, terrain height, etc.

[0064] 3. Magnetic sensors are used for navigation and attitude estimation. In some cases, changes in magnetic field data may indirectly reflect changes in the attitude of the drone, which is related to jitter.

[0065] 4. The gyroscope and accelerometer can directly reflect the acceleration, angular velocity and other attitude changes of the drone gimbal.

[0066] S102: The control platform determines, based on the attitude change information and the environmental information, first characteristic information corresponding to the attitude change information related to the jitter of the gimbal of the drone and second characteristic information corresponding to the environmental information;

[0067] In this step, feature information corresponding to the attitude change information related to the jitter of the drone gimbal and feature information corresponding to the environmental information can be extracted and recorded as the first feature information and the second feature information respectively.

[0068] Exemplarily, feature information directly related to the gimbal jitter is extracted from the posture jitter information, such as the amplitude, frequency, and duration of the posture change, etc. These feature information will be used as the first feature information for subsequent analysis and processing.

[0069] Exemplarily, feature information related to gimbal jitter is extracted from environmental information to identify environmental factors that may affect gimbal stability, such as wind speed, wind direction, terrain undulation, obstacle distance, etc. These feature information will be used as second feature information for subsequent analysis and processing.

[0070] Optionally, the first feature information includes: a first sub-feature vector of angular velocity data, a second sub-feature vector of acceleration data; the second feature information includes: a third sub-feature vector of video stream data, a fourth sub-feature vector of air pressure data, a fifth sub-feature vector of magnetic field data, and a sixth sub-feature vector of infrared data.

[0071] Optionally, step S102 may be implemented as follows:

[0072] Step 1: The control platform extracts features of the posture change information and the environmental information to obtain third feature information corresponding to the posture change information and fourth feature information corresponding to the environmental information;

[0073] Exemplarily, the control platform extracts feature information from the posture change information, which is recorded as the third feature information; the control platform extracts feature information from the environment information, which is recorded as the fourth feature information.

[0074] Step 2: The control platform determines the first feature information and the second feature information from the third feature information and the fourth feature information respectively according to the preset UAV gimbal jitter factor data.

[0075] For example, in the control platform, it is necessary to predefine the possible causes of drone gimbal jitter and their corresponding characteristic thresholds or patterns, that is, the factor data of drone gimbal jitter. These preset factor data can be set based on experimental data, expert experience or machine learning models.

[0076] For example, the control platform determines the first feature information from the third feature information according to the preset UAV gimbal jitter factor data.

[0077] Factor data related to the attitude characteristics directly related to the jitter of the drone gimbal, such as the amplitude of attitude change (maximum / minimum difference), frequency (obtained through Fourier transform and other methods), acceleration (absolute value or rate of change of angular acceleration), number of attitude mutations, etc.

[0078] For example, the control platform determines the second feature information from the fourth feature information based on preset drone gimbal jitter factor data.

[0079] Data on environmental factors that affect the stability of the drone gimbal, such as wind speed and direction change rate, air pressure fluctuations, temperature gradients, sudden changes in light intensity, obstacle approach speed, etc.

[0080] S103: The control platform determines a third feature vector of the drone gimbal according to the first feature information and the second feature information, where the third feature vector is a feature vector obtained by fusing the first feature information and the second feature information;

[0081] In this step, after obtaining the feature information of the external and internal factors affecting the lens shake in the drone gimbal, the first feature information and the second feature information are fused to obtain a feature vector affecting the lens shake.

[0082] That is, the third eigenvector aims to fully reflect the current state of the drone and its interaction with the environment. It is a high-dimensional vector that integrates the characteristics of the drone's attitude change and the characteristics of the environment in which it is located. Each element in this vector represents a specific aspect of characteristic information related to the stability of the gimbal. By fusing these features, the current state of the drone and the possible impact of this state on the stability of the gimbal can be more comprehensively evaluated. The fused eigenvector helps in subsequent decision-making and optimization of control strategies to improve the shooting quality and stability of the drone gimbal.

[0083] S104: The control platform controls the gimbal of the drone to move according to the third eigenvector to stabilize the lens.

[0084] In this step, the control platform controls the drone gimbal to move according to a vector that combines the drone's attitude change characteristics and the characteristics of the environment in which it is located, so as to keep the lens stable, improve shooting quality, expand application scenarios, and enhance user experience.

[0085] Exemplarily, features related to the attitude stability of the drone are extracted from the fused feature vector, such as the rate of change of angular velocity, the deviation of acceleration, etc. Then, based on the features related to the attitude stability of the drone, the drone gimbal is controlled to move so as to stabilize the lens.

[0086] That is, the fused feature vector is analyzed to evaluate the jitter state of the drone. This may include calculating certain statistics of the feature vector (such as norm, entropy, etc.), or inputting the feature vector into another machine learning model for classification or regression, and adjusting the control strategy of the drone based on the results of the jitter evaluation to reduce or eliminate the jitter.

[0087] The application provides a drone gimbal control method for avoiding severe lens shaking. The method is applied to a drone gimbal system and includes: an acquisition unit and a control platform. The method obtains the attitude change information of the drone gimbal and the environmental information of the drone gimbal through the acquisition unit; the control platform determines the first feature information corresponding to the attitude change information related to the jitter of the drone gimbal and the second feature information corresponding to the environmental information based on the attitude change information and the environmental information, and determines the third feature vector of the drone gimbal based on the first feature information and the second feature information. The third feature vector is the feature vector after the first feature information and the second feature information are fused. Then, based on the third feature vector, the drone gimbal is controlled to move to stabilize the lens. In this technical solution, the accuracy and robustness of attitude estimation are improved by comprehensively considering the environmental information and attitude change information, so as to avoid the problem of severe lens shaking when the drone is flying.

[0088] Based on the above embodiments, Figure 2 The flowchart of the second embodiment of the drone gimbal control method for preventing severe lens shaking provided in the embodiment of the present application is as follows: Figure 2 As shown, before S102 in the above embodiment, the method may also be as follows:

[0089] S201: The control platform performs denoising and calibration processing on the posture change information and the environmental information respectively to obtain processed posture change information and environmental information;

[0090] In this step, denoising and calibration are key steps to improve data quality, which can increase the accuracy of subsequent data processing and improve the efficiency of subsequent data processing.

[0091] Optionally, during denoising:

[0092] 1. For angular velocity data, acceleration data, and magnetic field data, the following formula can be used (taking angular velocity as an example):

[0093] y2=y1*a+y1(1-a)

[0094] Among them, y1 is the angular velocity data, a is the filter coefficient, and y2 is the filtered acceleration data. The filter coefficient is determined by the following factors: the frequency characteristics, waveform shape, and real-time response of the angular velocity signal; the filtering methods of acceleration data and magnetic field data are similar.

[0095] 2. For video stream data, the value of each pixel can be replaced by the median of the pixel values ​​in its neighborhood based on a sorting algorithm, and then implemented through weighted averaging. The weight is given by a Gaussian function, which can be expressed as follows:

[0096]

[0097] Among them, g(x, y) outputs the pixel value of the image at point (x, y), f(x, y) inputs the pixel value of the image at point (x, y), s is the average pooling window, and M is the number of pixels in s.

[0098] 3. For the air pressure data, calculate the average value of the data over a period of time to smooth the signal. The following formula can be used:

[0099]

[0100] Among them, y(n) is the air pressure data after filtering, x(n) is the air pressure data before filtering, and N is the size of the sliding window.

[0101] 4. For infrared data, for each pixel in the infrared image, sort the grayscale values ​​of all pixels in the neighborhood window centered on the pixel, and select the grayscale value in the middle after sorting as the new value of the pixel after filtering.

[0102] Optionally, during the calibration process:

[0103] 1. For angular velocity data, the zero offset calibration can be removed;

[0104] The angular velocity data is subtracted from the zero offset to obtain the calibrated angular velocity data, wherein the zero offset can be based on a value measured in a stationary state.

[0105] For example, it can be obtained by comparing the angular velocity of the UAV platform in flight state with the theoretical angular velocity.

[0106] 2. For acceleration data, the influence of gravity acceleration can be removed; taking the horizontal plane as an example, other plane solutions are similar;

[0107]

[0108] Among them, l is the acceleration after correction, q and w are the two-dimensional acceleration in the horizontal plane dimension before correction, and g is the gravitational acceleration.

[0109] 3. For video stream data, correction can be performed based on the brightness of the image to avoid errors in subsequent feature extraction. The formula that can be used is:

[0110] r2(x,y)=t*r1(x,y)

[0111] Among them, r2(x, y) is the brightness of the (x, y) point after correction, t is the brightness parameter, and r1(x, y) is the brightness of the (x, y) point before correction; the value of t is related to the brightness in the actual scene and the brightness of the captured video stream.

[0112] 4. For air pressure data, it can be adjusted according to local atmospheric pressure and temperature.

[0113] The temperature may be determined based on infrared data or a temperature sensor.

[0114] 5. For magnetic field data, the formula that can be used is:

[0115] S2=b+R*S1

[0116] Among them, S2 is the magnetic field data after correction, S1 is the magnetic field data before correction, R is the non-orthogonal matrix used to correct the magnetic field data, and b is the offset vector.

[0117] 6. For infrared data, the formula that can be used is:

[0118] F2=z+j*F1

[0119] Among them, F2 is the infrared data after correction, F1 is the infrared data before correction, j is the gain coefficient, and z is the offset compensation.

[0120] S202: The control platform synchronizes the posture change information and the environmental information based on the timestamp information to obtain the synchronized posture change information and environmental information.

[0121] In this step, each time the attitude-related sensors on the drone collect attitude change information (such as pitch angle, roll angle, yaw angle, etc.), a timestamp will be automatically added. This timestamp records the specific time when the data was collected; similarly, when the environment-related sensors collect environmental information, a timestamp will also be added. This timestamp records the specific time when the environmental data was collected.

[0122] Since the sampling frequencies of different sensors may be different, the timestamp of the environmental information may not be completely consistent with the timestamp of the posture change information, so the posture change information and the environmental information must be synchronized based on the timestamp.

[0123] Exemplarily, the implementation may be as follows:

[0124] 1. During the system startup or initialization phase, the timestamps of all sensors need to be calibrated to ensure that the time deviation between them is as small as possible; the calibration process may involve the use of high-precision time synchronization methods, such as the drone can use the Global Positioning System (GPS) time or other high-precision time sources or built-in time calibration algorithms.

[0125] 2. For sensors with different sampling frequencies, the control platform can synchronize data through timestamp interpolation or extrapolation;

[0126] For example, interpolation is a method of estimating unknown data points between two known data points. If the timestamp of a certain environmental data does not match the timestamp of the posture data, the interpolation algorithm can be used to find the closest posture data near the timestamp; extrapolation is to predict future or past data points based on the trend of known data points. In some cases, if the timestamp of the environmental data is ahead of or behind the timestamp of the posture data by a large amount, an extrapolation algorithm may be needed to estimate the corresponding posture data.

[0127] 3. The control platform can adopt advanced timestamp synchronization algorithms (such as Kalman filtering, particle filtering, etc.) to further improve the accuracy and real-time performance of data synchronization; these algorithms can comprehensively consider the data and timestamp information of multiple sensors, and obtain more accurate and reliable synchronized data through data fusion and state estimation.

[0128] The application provides a method for controlling a drone gimbal to prevent severe lens shaking. The control platform performs denoising and calibration processing on attitude change information and environmental information respectively to obtain processed attitude change information and environmental information. The control platform synchronizes the attitude change information and environmental information based on timestamp information to obtain synchronized attitude change information and environmental information. In this technical solution, by performing denoising and calibration processing and synchronization on attitude change information and environmental information respectively, the accuracy of the collected data can be increased to improve the effect of preventing severe lens shaking.

[0129] Based on the above embodiments, Figure 3 The flowchart of the third embodiment of the method for controlling the gimbal of a drone to prevent severe shaking of the lens provided in the embodiment of the present application is as follows: Figure 3 As shown, S103 of the above embodiment may be as follows:

[0130] S301: The control platform determines, for each sensor corresponding to a sub-eigenvector, a weight coefficient corresponding to the sensor, a preset positive real number parameter for controlling the attenuation speed, a preset reference vector, and a preprocessing function corresponding to the sensor, a contribution value of the sensor in a third eigenvector;

[0131] Among them, the weight coefficient is determined based on the flight status and environmental conditions;

[0132] Exemplarily, with respect to the weight coefficients, for example, when strong winds or mechanical failures are detected, the weight coefficients of the acceleration sensor and angular velocity sensor can be appropriately increased; when the flight environment is relatively stable, the weight coefficients of the air pressure sensor, magnetic field sensor, infrared sensor, and visual sensor can be increased.

[0133] Optionally, for any sensor, the calculation formula for determining the contribution value of the sensor in the third eigenvector according to the sub-eigenvector, the weight coefficient corresponding to the sensor, the preset positive real number parameter for controlling the attenuation speed, the preset reference vector, and the preprocessing function corresponding to the sensor may be:

[0134]

[0135] Where h is the number of sensors, c i is the sub-feature vector involved in the i-th sensor (such as angular velocity, angular acceleration, etc.), v i is the weight coefficient of the i-th sensor, p is a positive real number parameter that controls the decay speed, m is a preset reference vector, e(c i ) is the preprocessing function corresponding to the i-th sensor, c j is the sub-feature vector involved in the jth sensor, z(c i , v i ) is the contribution value of the sensor in the third eigenvector.

[0136] It should be understood that the sum of the weight coefficients corresponding to each sensor is 1.

[0137] S302: The control platform determines a third eigenvector according to contribution values ​​of each sensor.

[0138] In this step, in the UAV gimbal system, in order to comprehensively evaluate the jitter impact caused by information from different sensors, the contribution values ​​from different sensors obtained as above can be added or more complexly fused to generate a feature vector reflecting the jitter state of the UAV.

[0139] Method 1, determining the third eigenvector according to the contribution values ​​of each sensor, may be to add up the contribution values ​​of each sensor to obtain the third eigenvector.

[0140] Method 2: Considering the different importance of each sensor to jitter evaluation, weighted summation is achieved by assigning different weights to the contribution value of each sensor. The weight can be determined based on experimental data, expert knowledge, or machine learning algorithms.

[0141] Method 3 uses more complex fusion methods, such as Kalman filters, particle filters, or machine learning-based fusion algorithms (such as neural networks, support vector machines, etc.) to more accurately integrate the contribution values ​​from different sensors.

[0142] The application provides a method for controlling a drone gimbal to avoid severe lens shaking. The control platform determines the contribution value of the sensor in the third eigenvector according to the sub-eigenvector corresponding to each sensor, the weight coefficient corresponding to the sensor, the preset positive real number parameter for controlling the attenuation speed, the preset reference vector, and the preprocessing function corresponding to the sensor. The weight coefficient is determined based on the flight state and environmental conditions. The control platform determines the third eigenvector according to the contribution value of each sensor. In this technical solution, weight coefficients are assigned to each sensor with reference to the flight state and environmental conditions, so as to correct the contribution of each sensor in a more realistic manner and increase the accuracy of the fused eigenvector.

[0143] Based on the above embodiments, Figure 4 The flowchart of the fourth embodiment of the method for controlling the gimbal of a drone to prevent severe shaking of the lens provided in the embodiment of the present application is as follows: Figure 4 As shown, S104 in the above embodiment may be as follows:

[0144] S401: The control platform determines the target flight information corresponding to the third characteristic vector according to a preset mapping relationship, where the mapping relationship records at least one flight information and at least one characteristic vector corresponding to the flight information;

[0145] In this step, in the control platform, a mapping relationship from the fused feature vector to the flight information can be pre-established or learned, so that it can be subsequently determined that the fused feature vector can be directly mapped to obtain how to control the flight information of the drone, so as to simplify the subsequent execution process and improve the response speed.

[0146] For example, historical flight data and corresponding feature vectors can be used to train a machine learning model. The learning goal of the model is to find the best mapping relationship between the fused feature vector and the target flight information. The fused feature vector is input into the trained machine learning model, and the model outputs the predicted target flight information based on the learned mapping relationship.

[0147] S402: The control platform controls the gimbal of the drone to perform flight actions according to the target flight information to stabilize the lens.

[0148] In this step, a control strategy for the UAV is formulated based on the predicted target flight information, which may include adjusting the flight speed, changing the flight direction, adjusting the attitude, etc.

[0149] In addition, new flight information can be obtained in real time through the drone's sensor system, and the above process can be repeated to achieve continuous monitoring and adjustment of the drone's flight status;

[0150] Furthermore, based on the feedback from flight performance, feature extraction methods, fusion algorithms, machine learning models, etc. are continuously optimized to improve the accuracy of target flight information and the stability of the control system.

[0151] The application provides a method for controlling a drone gimbal to avoid severe lens shaking. The control platform determines the target flight information corresponding to the third feature vector according to a preset mapping relationship. The mapping relationship records at least one flight information and at least one feature vector corresponding to the flight information. The control platform controls the drone gimbal to perform flight actions according to the target flight information to stabilize the lens. In this technical solution, after obtaining the target flight information, the flight information executed by the drone gimbal can be directly determined based on the preset mapping relationship. The execution process is simple, which improves the efficiency of real-time adjustment to avoid severe lens shaking.

[0152] Based on the above embodiments, Figure 5 The flowchart of the fifth embodiment of the method for controlling the gimbal of a drone to prevent severe shaking of the lens provided in the embodiment of the present application is as follows: Figure 5 As shown, the method can also be as follows:

[0153] S501: The acquisition unit obtains the flight status information of the UAV gimbal;

[0154] In this step, the drone gimbal is usually equipped with a variety of sensors, such as accelerometers, gyroscopes, magnetometers, GPS, etc., which are used to collect the flight status information of the drone in real time.

[0155] Furthermore, the collected raw data needs to be preprocessed and analyzed to extract useful flight status information, which includes data denoising, calibration, time synchronization and other steps to ensure the accuracy and reliability of the data.

[0156] S502: The control platform controls the UAV gimbal to perform actions according to the flight status information and the target flight information.

[0157] In this step, according to the target flight information and flight status information of the UAV jitter, the control platform needs to determine the control target of the UAV gimbal, such as the adjustment of the lens, the UAV wings, the rotation speed, the angle, etc.

[0158] The control method can be: the control platform generates corresponding control instructions, sends control instructions to the controller of the drone gimbal (or itself), and then performs corresponding actions, such as adjusting the posture, stabilizing the picture, etc.

[0159] The application provides a method for controlling a drone gimbal to prevent severe lens shaking. The control platform acquisition unit obtains the flight status information of the drone gimbal, and the control platform controls the drone gimbal to perform actions based on the flight status information and target flight information. In this technical solution, corrections can be provided for subsequent drone flights to counteract severe lens shaking caused by external factors.

[0160] Based on the above embodiments, Figure 6 The flowchart of the sixth embodiment of the drone gimbal control method for preventing severe lens shaking provided in the embodiment of the present application is as follows: Figure 6 As shown, the method can also be as follows:

[0161] S601: The acquisition unit obtains a flight path setting instruction issued by a user;

[0162] For example, the user sets the flight path through input devices such as drone remote controllers, mobile phone applications, tablet computers, or dedicated flight control software. These devices usually provide intuitive operating interfaces such as touch screens, buttons, or joysticks;

[0163] After that, the user can set the flight path by drawing a path on the map, entering coordinate points, selecting a preset path template, or directly controlling it through gestures. The instructions entered by the user are first received by the flight control software or application. There will be a parsing mechanism inside the software to identify the various instructions entered by the user and convert them into a format that the drone can understand. After parsing the instructions, the software may use a path planning algorithm to optimize the path to ensure that the drone can fly along the most efficient and safest route.

[0164] Furthermore, the optimized flight path instruction is transmitted to the UAV via wireless communication technology. This step usually requires a stable communication connection to be established between the acquisition unit of the UAV and the input device, that is, the acquisition unit receives the flight path setting instruction from the input device.

[0165] S602: The control platform corrects the flight trajectory corresponding to the flight path setting instruction according to the target flight information, obtains a corrected flight trajectory, and controls the UAV gimbal flight based on the corrected flight trajectory.

[0166] In this step, the control platform first parses the flight path setting instructions input by the user to obtain parameters such as path point coordinates, flight altitude, speed, etc.

[0167] Since the target flight information reflects the environment and safety assessment, which may be whether there are no-fly zones, obstacles, bad weather and other unfavorable factors on the path, the flight trajectory corresponding to the flight path setting instruction can be corrected based on the target flight information obtained in real time to ensure the safety and stability of the UAV flight.

[0168] Optionally, the flight trajectory is corrected according to the target flight information, and the original flight trajectory is optimized by applying a trajectory correction algorithm. The correction algorithm may include path smoothing, obstacle avoidance strategy, energy consumption minimization, etc., and may take into account the UAV's dynamic characteristics, flight constraints (such as maximum speed, maximum climb rate, etc.) and external environmental factors (such as wind field, temperature, etc.).

[0169] After being processed by the correction algorithm, a corrected flight trajectory is generated. This trajectory should meet all flight constraints and optimize flight performance and safety as much as possible.

[0170] After the above steps, the corrected flight trajectory can be the expected flight information, but in fact there may be a small error. This application provides the following control strategy to adjust the flight situation in real time:

[0171]

[0172] Among them, u(u) is the control output at the uth moment, such as the thrust and rudder angle of the drone; e(u) is the error signal at the uth moment, that is, the difference between the expected flight information and the actual flight information; K p , K i , K d are the gains of the three factors respectively, which can be adjusted based on the actual situation of the UAV gimbal system.

[0173] The drone gimbal control method for avoiding severe lens shaking provided by the present application is that the acquisition unit obtains the flight path setting instruction issued by the user, and the control platform corrects the flight trajectory corresponding to the flight path setting instruction according to the target flight information, obtains the corrected flight trajectory, and controls the drone gimbal flight based on the corrected flight trajectory. In this technical solution, after the user inputs the path, the flight action on the path is directly corrected in advance to avoid the problem of severe lens shaking on the subsequent path.

[0174] Figure 7 A schematic diagram of the structure of the control platform provided in the embodiment of the present application, such as Figure 7 As shown, the control platform 700 includes: a processor 702, a memory 701 and a communication interface 703 which are communicatively connected to the processor 702;

[0175] The memory 701 stores computer executable instructions;

[0176] The processor 702 executes the computer-executable instructions stored in the memory 701 to implement the drone gimbal control method for avoiding severe shaking of the lens in any method embodiment.

[0177] Figure 8 The schematic diagram of the structure of the drone gimbal system provided in the embodiment of the present application is as follows: Figure 8 As shown, the drone gimbal system 800 includes a control platform 801 and a collection unit 802, and the control platform 801 and the collection unit 802 are communicatively connected to execute the drone gimbal control method for avoiding severe shaking of the lens in any of the aforementioned embodiments.

[0178] The collection unit 802 is used to collect the posture change information of the drone gimbal and the environment information of the drone gimbal.

[0179] Specifically, the acquisition unit 802 may include the following sensors: a gyroscope, an accelerometer, a visual sensor, an air pressure sensor, a magnetic sensor, and an infrared sensor.

[0180] Among them, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the visual sensor collects video stream data, the air pressure sensor collects air pressure data, the magnetic sensor collects magnetic field data, and the infrared sensor collects infrared data.

[0181] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to execute the drone gimbal control method for avoiding severe shaking of the lens provided in the above-mentioned various embodiments.

[0182] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0183] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0184] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution provided by any of the above method embodiments can be implemented.

[0185] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0186] It is to be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0187] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0188] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for controlling a drone gimbal to avoid severe lens shaking, characterized in that: Applied to a UAV pan-tilt system, the UAV pan-tilt system includes: a collection unit and a control platform, and the method includes: The acquisition unit acquires the posture change information of the drone gimbal and the environment information of the drone gimbal; The control platform determines, according to the attitude change information and the environmental information, first feature information corresponding to the attitude change information related to the jitter of the UAV gimbal and second feature information corresponding to the environmental information; The control platform determines a third feature vector of the UAV gimbal according to the first feature information and the second feature information, wherein the third feature vector is a feature vector obtained by fusing the first feature information and the second feature information; The control platform controls the drone gimbal to move according to the third eigenvector to stabilize the lens.

2. The method according to claim 1, characterized in that Before determining, according to the posture change information and the environmental information, first feature information corresponding to the posture change information and second feature information corresponding to the environmental information, the method further includes: The control platform performs denoising and calibration processing on the posture change information and the environmental information respectively to obtain the processed posture change information and the environmental information; The control platform synchronizes the posture change information and the environmental information based on the timestamp information to obtain the synchronized posture change information and the environmental information.

3. The method according to claim 1, characterized in that The control platform determines, according to the attitude change information and the environmental information, first feature information corresponding to the attitude change information related to the jitter of the UAV gimbal and second feature information corresponding to the environmental information, including: The control platform performs feature extraction on the posture change information and the environmental information to obtain third feature information corresponding to the posture change information and fourth feature information corresponding to the environmental information; The control platform determines the first feature information and the second feature information from the third feature information and the fourth feature information respectively according to the preset factor data of the jitter of the drone gimbal.

4. The method according to any one of claims 1 to 3, characterized in that: The acquisition unit includes the following sensors: gyroscope, accelerometer, visual sensor, air pressure sensor, magnetic sensor, infrared sensor; Correspondingly, the gyroscope collects angular velocity data, the accelerometer collects acceleration data, the visual sensor collects video stream data, the air pressure sensor collects air pressure data, the magnetic sensor collects magnetic field data, and the infrared sensor collects infrared data; Correspondingly, the first feature information includes: the first sub-feature vector of the angular velocity data, the second sub-feature vector of the acceleration data; the second feature information includes: the third sub-feature vector of the video stream data, the fourth sub-feature vector of the air pressure data, the fifth sub-feature vector of the magnetic field data, and the sixth sub-feature vector of the infrared data.

5. The method according to claim 4, characterized in that The control platform determines a third feature vector of the UAV gimbal according to the first feature information and the second feature information, including: The control platform determines, for each sensor corresponding to a sub-eigenvector, a weight coefficient corresponding to the sensor, a preset positive real number parameter for controlling the attenuation speed, a preset reference vector, and a preprocessing function corresponding to the sensor, wherein the weight coefficient is determined based on the flight state and the environmental conditions; The control platform determines the third eigenvector according to the contribution values ​​of each sensor.

6. The method according to any one of claims 1 to 3, characterized in that: The control platform controls the drone gimbal to move according to the third eigenvector to stabilize the lens, including: The control platform determines the target flight information corresponding to the third feature vector according to a preset mapping relationship, wherein the mapping relationship records at least one flight information and a feature vector corresponding to the at least one flight information; The control platform controls the UAV gimbal to perform flight actions according to the target flight information to stabilize the lens.

7. The method according to claim 6, characterized in that After the control platform controls the drone gimbal to move according to the third eigenvector to stabilize the lens, the method further includes: The acquisition unit obtains the flight status information of the UAV gimbal; The control platform controls the UAV gimbal to perform actions according to the flight status information and the target flight information.

8. The method according to claim 6, characterized in that After the control platform controls the drone gimbal to move according to the third eigenvector to stabilize the lens, the method further includes: The acquisition unit acquires a flight path setting instruction issued by a user; The control platform corrects the flight trajectory corresponding to the flight path setting instruction according to the target flight information to obtain a corrected flight trajectory, and controls the flight of the UAV gimbal based on the corrected flight trajectory.

9. A control platform, characterized in that: include: A processor, a memory and a communication interface connected to the processor in communication; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the drone gimbal control method for avoiding severe lens shaking as described in any one of claims 1 to 8.

10. A drone gimbal system, characterized in that: The UAV gimbal system includes: a collection unit and a control platform; The control platform and the acquisition unit are communicatively connected to execute the drone gimbal control method for avoiding severe lens shaking as described in any one of claims 1 to 8.

Images