Unmanned aerial vehicle de-skewing method based on visual recognition

Abstract

A visual recognition-based unmanned aerial vehicle landing correction method, comprising: acquiring a shaft opening area image in an unmanned aerial vehicle landing mode through a visual sensor, identifying a preset landing mark in the shaft opening area image, and outputting a center pixel position corresponding to the landing mark; projecting a geometric center position of the unmanned aerial vehicle to an image coordinate system unified with the landing mark based on attitude information when the unmanned aerial vehicle is descending and calibration parameters of the visual sensor, to obtain a body center pixel coordinate; calculating a pixel deviation of the body center pixel coordinate and the center pixel position in the image coordinate system, and converting the pixel deviation into a spatial horizontal deviation and a spatial vertical deviation; determining a near-ground aerodynamic compensation coefficient corresponding to a descending height of the unmanned aerial vehicle according to a ratio of a shaft opening diameter to a rotor disc diameter of the unmanned aerial vehicle and the descending height; and generating a flight control execution instruction set based on the spatial horizontal deviation, the spatial vertical deviation and the near-ground aerodynamic compensation coefficient, to control the unmanned aerial vehicle to correct the descending action.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicle control, and more particularly, relates to a method for correcting the landing deviation of an unmanned aerial vehicle based on visual recognition. BACKGROUND

[0002] When an unmanned aerial vehicle is operated in a narrow area or a high-risk area (for example, the top of a high-voltage transmission tower, a building facade maintenance platform, a chimney or the top of a wellhead), the unmanned aerial vehicle needs to be continuously corrected for landing deviation, and the main reason is that the available space for landing in these areas is extremely small, and there may be high-voltage electricity, hot air or vortex air flow around, so if the unmanned aerial vehicle lands off by a small distance, it may touch the obstacles and cause a crash or damage to the equipment.

[0003] Currently, in the chimney, shaft or other annular enclosed top area, the downwash of the propeller is limited by the radial constraint of the annular wall surface during the terminal descent of the unmanned aerial vehicle, and a significant annular backflow and longitudinal vortex system structure is formed. The backflow repeatedly reflects and guides between the shaft inner wall and the lower edge of the propeller disc, so that the induced flow velocity distribution in different propeller disc areas appears obvious asymmetry. When the vertical descent is about 1~1.5 times the propeller disc diameter from the wall height, this asymmetric flow field will trigger an alternating process of typical local ground effect enhancement and lateral ground effect weakening, which will significantly increase the single-sided propeller lift fluctuation amplitude, directly causing short-period roll and pitch coupling oscillation of the machine body. At the same time, the turbulent component carried by the radial backflow and the vertical plume superimpose, which will change the overall lift center position in a short time, resulting in the repeated alternating phenomenon of height overshoot (instantaneous lifting) or height collapse (instantaneous sinking) of the machine body.

[0004] However, in the prior art, when the flight control system senses the terminal attitude and height error, if only the thrust distribution and attitude correction are performed according to the open field aerodynamic assumption, the phase mismatch between the correction action and the actual aerodynamic environment will be caused, which will significantly amplify the lateral drift and height fluctuation in the last about 1 meter of the descent, and it is more likely to cause the risk of touching the shaft wall or falling when the unmanned aerial vehicle lands in a narrow shaft. SUMMARY

[0005] To solve the problems in the prior art, the present application aims to solve the above-mentioned defects, and further proposes a method for correcting the landing deviation of an unmanned aerial vehicle based on visual recognition.

[0006] The present application adopts the following technical solutions.

[0007] The present application discloses a method for correcting the landing deviation of an unmanned aerial vehicle based on visual recognition, which comprises:

[0008] acquire an image of a shaft mouth area in a landing mode of the unmanned aerial vehicle through a visual sensor, and identify a preset landing mark in the image of the shaft mouth area to output a center pixel position corresponding to the landing mark;

[0009] project a geometric center position of the unmanned aerial vehicle body in an actual space to an image coordinate system unified with the landing mark based on attitude information when the unmanned aerial vehicle is descending and calibration parameters of the visual sensor to obtain a body center pixel coordinate;

[0010] calculate a pixel deviation of the body center pixel coordinate and the center pixel position in the image coordinate system, and convert the pixel deviation into a spatial horizontal deviation and a spatial vertical deviation;

[0011] determine a near-ground aerodynamic compensation coefficient corresponding to the descending height of the unmanned aerial vehicle according to a ratio of a shaft mouth diameter to a propeller diameter of the unmanned aerial vehicle body and the descending height of the unmanned aerial vehicle;

[0012] generate a flight control execution instruction set based on the spatial horizontal deviation, the spatial vertical deviation, and the near-ground aerodynamic compensation coefficient, and control the unmanned aerial vehicle to correct a descending action in cooperation with aerodynamic disturbance in response to the flight control execution instruction set.

[0013] Further, the acquiring the image of the shaft mouth area in the landing mode of the unmanned aerial vehicle through the visual sensor and identifying the preset landing mark in the image of the shaft mouth area to output the center pixel position corresponding to the landing mark comprises:

[0014] when the unmanned aerial vehicle flight control system detects a landing trigger signal, controlling the unmanned aerial vehicle to enter a landing mode in response to the landing trigger signal, and simultaneously calling an on-board visual sensor to collect the image of the shaft mouth area at a fixed frame rate;

[0015] constructing an original image matrix based on a plurality of images of the shaft mouth area collected at the fixed frame rate, and performing binary processing on each image element in the original image matrix according to a preset black and white contrast and shape feature.

[0016] Further, the acquiring the image of the shaft mouth area in the landing mode of the unmanned aerial vehicle through the visual sensor and identifying the preset landing mark in the image of the shaft mouth area to output the center pixel position corresponding to the landing mark further comprises:

[0017] performing an open operation processing on the original image matrix to remove noise of the image elements and identify all connected pixel areas, calculating areas and aspect ratios of the connected pixel areas to screen out candidate pixel areas conforming to a preset mark specification;

[0018] The pixel coordinates of the bounding rectangle of each candidate pixel region are calculated, and each candidate pixel region is normalized matched with a preset landing mark template to output the pixel region coordinates with the highest matching degree as the center pixel position of the landing mark.

[0019] Further, the geometric center position of the UAV body in the actual space is projected to the image coordinate system unified with the landing mark based on the attitude information of the UAV during landing and the calibration parameters of the visual sensor to obtain the body center pixel coordinates, including:

[0020] The attitude information of the UAV during landing is obtained from the UAV flight control system, and the attitude information includes the roll angle, pitch angle and yaw angle of the UAV, and the attitude information and the calibration coefficient are time-synchronized to output combined data of the attitude information and the calibration coefficient with unified timestamps;

[0021] The geometric center position of the UAV body in the actual space is determined according to the structural parameters of the UAV body, and the geometric center position is mapped to the image coordinate system monitored by the visual sensor in combination with the combined data to determine the body center pixel coordinates.

[0022] Further, the pixel deviation between the body center pixel coordinates and the center pixel position in the image coordinate system is calculated, and the pixel deviation is converted into spatial horizontal deviation and spatial vertical deviation, including:

[0023] The lateral pixel deviation and longitudinal pixel deviation between the body center pixel coordinates and the center pixel position coordinates are calculated in the unified image coordinate system to output the pixel deviation composed of the lateral pixel deviation and the longitudinal pixel deviation;

[0024] The pixel deviation is converted into actual distance according to the imaging scale relationship of the visual sensor at the height of the UAV to output the spatial horizontal deviation and the spatial vertical deviation, and the spatial horizontal deviation and the spatial vertical deviation are combined into a binary vector, and the binary vector is a relative deviation vector.

[0025] Further, the near-ground aerodynamic compensation coefficient corresponding to the UAV descent height is determined according to the ratio of the shaft opening diameter to the UAV body rotor diameter and the UAV descent height, including:

[0026] Based on the relative deviation vector, the rotor diameter, the shaft opening diameter and the UAV descent height, a dimensionless index and a deviation amplitude are calculated, the dimensionless index includes the ratio of the UAV descent height to the rotor diameter and the ratio of the shaft opening diameter to the rotor diameter, and the deviation amplitude is the Euclidean norm of each component in the relative deviation vector;

[0027] According to the dimensionless index, a two-dimensional table is constructed offline, and an axial ground effect amplification coefficient is calculated by bilinear interpolation, while an asymmetric lift compensation coefficient and a lateral disturbance suppression coefficient corresponding to linear scaling of the deviation amplitude are calculated online;

[0028] The near-ground aerodynamic compensation coefficient includes the axial ground effect amplification coefficient, the asymmetric lift compensation coefficient, and the lateral disturbance suppression coefficient.

[0029] Further, the flight control execution instruction set is generated based on the spatial horizontal deviation, the spatial vertical deviation, and the near-ground aerodynamic compensation coefficient, and the unmanned aerial vehicle is corrected in response to the flight control execution instruction set and the aerodynamic disturbance control to perform a descending action, comprising:

[0030] The spatial vertical deviation is weighted and corrected according to the axial ground effect amplification coefficient to output a vertical correction amount after axial compensation.

[0031] The horizontal deviation amplitude corresponding to the spatial horizontal deviation is calculated, and a correction direction angle is determined according to the position angle of the unmanned aerial vehicle rotor, and the rotor thrust is adjusted in combination with the asymmetric lift compensation coefficient to output a thrust adjustment amount corresponding to each rotor.

[0032] Further, the flight control execution instruction set is generated based on the spatial horizontal deviation, the spatial vertical deviation, and the near-ground aerodynamic compensation coefficient, and the unmanned aerial vehicle is corrected in response to the flight control execution instruction set and the aerodynamic disturbance control to perform a descending action, further comprising:

[0033] Based on the spatial horizontal deviation and the lateral disturbance suppression coefficient, a target roll angle and a target pitch angle of the unmanned aerial vehicle attitude correction are calculated, and an amplitude limit corresponding to the target roll angle and the target pitch angle is set.

[0034] When the axial ground effect amplification coefficient exceeds a first threshold value or the ratio of the unmanned aerial vehicle descending height to the rotor disc diameter is lower than a second threshold value, a preset limiting descending rate is executed, otherwise an end rate is executed.

[0035] The flight control execution instruction set is a flight control execution instruction corresponding to each rotor thrust adjustment amount, target roll angle and target pitch angle, limiting descending rate and end rate.

[0036] The second aspect of the present application discloses a visual recognition-based unmanned aerial vehicle descent deviation correction system for realizing the visual recognition-based unmanned aerial vehicle descent deviation correction method of the first aspect, the system comprising:

[0037] A landing mark recognition module is configured to acquire an image of a shaft opening area in a landing mode of the unmanned aerial vehicle through a visual sensor, and recognize a preset landing mark in the image of the shaft opening area to output a center pixel position corresponding to the landing mark.

[0038] a body center mapping module, configured to project a geometric center position of the UAV body in a real space to an image coordinate system unified with the landing mark based on attitude information when the UAV is descending and calibration parameters of the visual sensor, to obtain a body center pixel coordinate;

[0039] a pixel deviation calculation module, configured to calculate a pixel deviation of the body center pixel coordinate and the center pixel position in the image coordinate system, and convert the pixel deviation into a spatial horizontal deviation and a spatial vertical deviation;

[0040] a compensation coefficient determination module, configured to determine a near-ground aerodynamic compensation coefficient corresponding to the UAV descending height according to a ratio of a shaft mouth diameter to a UAV body rotor diameter and the UAV descending height;

[0041] a UAV descent deviation correction module, configured to generate a flight control execution instruction set based on the spatial horizontal deviation, the spatial vertical deviation and the near-ground aerodynamic compensation coefficient, and correct the descending action of the UAV in cooperation with aerodynamic disturbance in response to the flight control execution instruction set.

[0042] The third aspect of the present application discloses a terminal, comprising a processor and a storage medium;

[0043] The storage medium is used for storing instructions;

[0044] The processor is used for operating according to the instructions to perform the steps of the method of the first aspect.

[0045] The fourth aspect of the present application discloses a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the method of the first aspect.

[0046] The present application has the following advantages compared with the prior art:

[0047] (1) After the UAV enters the landing mode, the present application acquires the images of the shaft mouth area by the on-board visual sensor at a fixed frame rate, identifies the preset landing mark (such as a high-contrast geometric shape) in the images, and obtains the center pixel position of the landing mark in the image coordinate system through image processing, thereby providing a target reference point for subsequent relative deviation calculation. Meanwhile, the position of the geometric center of the UAV body in the real space is projected into the image coordinate system in combination with the attitude information of the flight control system and the camera calibration parameters, a comparable coordinate system between the landing point reference and the body center is established, the horizontal and vertical pixel differences of the landing mark center and the body center in the image coordinate system are calculated, and the pixel differences are converted into the spatial horizontal deviation and the vertical deviation by using the camera calibration coefficient and the current height, so that the quantized errors available for attitude and position correction are obtained.

[0048] (2) The present application finds or calculates the near-ground aerodynamic compensation coefficient according to the ratio of the shaft opening diameter to the rotor disc diameter and the current descent height, including the asymmetric lift compensation coefficient and the lateral disturbance suppression coefficient, quantifies the asymmetric aerodynamic characteristics in the terminal stage, and provides a correction factor for the correction instruction generation. Finally, the relative deviation and the aerodynamic compensation coefficient are combined to generate a flight control execution instruction set containing thrust distribution, attitude correction, and descent rate limitation, so that the correction action is synchronized with the change trend of the aerodynamic disturbance, and the asymmetric lift and turbulent disturbance caused by the wall and ground effect coupling are offset, and the phase mismatch amplification drift is prevented. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is the flowchart of the unmanned aerial vehicle descent deviation correction method based on visual recognition provided by the present application;

[0050] Figure 2 is the structural schematic diagram of the unmanned aerial vehicle descent deviation correction system based on visual recognition provided by the present application. DETAILED DESCRIPTION

[0051] The present application will be further described below in conjunction with the drawings. The following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0052] As Figure 1 shown, in one embodiment, an unmanned aerial vehicle descent deviation correction method based on visual recognition includes the following steps:

[0053] Step S110, acquiring the shaft opening area image in the unmanned aerial vehicle landing mode through the visual sensor, and identifying the preset landing mark in the shaft opening area image to output the center pixel position corresponding to the landing mark.

[0054] In some embodiments, the unmanned aerial vehicle descent deviation correction method based on visual recognition provided by the present application specifically includes the following steps in step S110:

[0055] Step S111, when the unmanned aerial vehicle flight control system detects the landing trigger signal, responding to the landing trigger signal to control the unmanned aerial vehicle to enter the landing mode, and simultaneously calling the airborne visual sensor to collect the shaft opening area image at a fixed frame rate.

[0056] Step S112, constructing an original image matrix based on a plurality of fixed frame rate collected shaft opening area images, and performing binaryzation processing on each image element in the original image matrix according to the preset black and white contrast and shape feature.

[0057] In some embodiments, the unmanned aerial vehicle descent deviation correction method based on visual recognition provided by the present application specifically further includes the following steps in step S110:

[0058] Step S113, open operation processing is performed on the original image matrix to remove the noise of the image elements, and all connected pixel regions are identified, the area and the aspect ratio of the connected pixel regions are calculated to screen out the candidate pixel regions meeting the preset flag specification.

[0059] Step S114, the pixel coordinates of the bounding rectangular boundary frame corresponding to each candidate pixel region are calculated, and each candidate pixel region is normalized matched with the preset landing flag template to output the pixel region coordinates with the highest matching degree as the center pixel position of the landing flag.

[0060] In specific embodiments, the present application provides a visual recognition-based unmanned aerial vehicle landing deviation correction method, which includes steps 1-5:

[0061] Step 1, landing area visual acquisition and flag identification.

[0062] After the unmanned aerial vehicle enters the landing mode, the onboard downward-looking visual sensor acquires the vertical shaft opening area image at a fixed frame rate, and identifies the preset landing flag (such as a high-contrast geometric shape) in the image, and obtains the center pixel position of the landing flag in the image coordinate system through image processing, to provide a target reference point for subsequent relative deviation calculation. It includes the following sub-steps:

[0063] Sub-step 1.1, landing mode triggering and sensor initialization.

[0064] Specifically, when the unmanned aerial vehicle flight control system detects a landing trigger signal, it sends a start instruction to the onboard downward-looking camera (i.e., the visual sensor), and reads or sets the calibration parameters of the camera, including the pixel focal length of the camera in the horizontal or vertical direction, the pixel position of the principal point (optical axis center) in the image coordinate system, and the acquisition frame rate (frames / second, range 15-60). The above parameters are obtained by camera calibration process and stored in the onboard storage unit, and are sent to the visual processing unit through a digital interface (MIPI / USB).

[0065] Sub-step 1.2, landing area original image acquisition.

[0066] Specifically, the camera acquires downward-looking images of the vertical shaft opening and the surrounding area at a set acquisition frame rate, with a resolution of WxH (pixels), W being the image width and H being the height. Finally, an original image matrix is constructed based on the acquired downward-looking images, which is composed of red channel gray values, green channel gray values and blue channel gray values (0-255) of the pixel points, and each gray value has horizontal and vertical pixel coordinates as an index.

[0067] Sub-step 1.3, landing flag candidate region extraction.

[0068] Specifically, according to the preset color range (black and white contrast) or shape feature (circular or cross-shaped) to perform binarization processing, and at the same time, performing an open operation to remove noise and small area targets, identifying all connected pixel regions, calculating the area and aspect ratio thereof, screening out candidate regions meeting the preset specifications of the mark, and then calculating the four boundary pixel coordinates of the surrounding rectangle for each candidate region, finally outputting a candidate region set, and each candidate region in the set is represented by a rectangular boundary box.

[0069] Sub-step 1.4, further verification is performed on each candidate region, the candidate region image is matched with a pre-stored landing point mark template (such as a high-contrast cross-shaped) by normalized cross-correlation, and finally the region with the highest matching degree score is taken as the final target region, and the coordinate of the target region is output as the center pixel position coordinate of the landing mark.

[0070] Step S120, based on the attitude information of the unmanned aerial vehicle during landing and the calibration parameters of the visual sensor, the geometric center position of the unmanned aerial vehicle body in the actual space is projected to the image coordinate system unified with the landing mark, to obtain the body center pixel coordinate.

[0071] In some embodiments, the present application provides a visual recognition-based unmanned aerial vehicle landing offset correction method, and step S120 specifically includes the following steps:

[0072] Step S121, the attitude information of the unmanned aerial vehicle during landing is obtained from the unmanned aerial vehicle flight control system, the attitude information includes the roll angle, pitch angle and yaw angle of the unmanned aerial vehicle, and the attitude information and the calibration coefficient are time-synchronized to output combined data of the attitude information and the calibration coefficient with a unified timestamp.

[0073] Step S122, the geometric center position of the unmanned aerial vehicle body in the actual space is determined according to the structural parameters of the unmanned aerial vehicle body, and the geometric center position is mapped to the image coordinate system monitored by the visual sensor in combination with the combined data, to determine the body center pixel coordinate.

[0074] In a specific embodiment, the present application provides a visual recognition-based unmanned aerial vehicle landing offset correction method, and step 2, the body center projection position calculation. The position of the geometric center of the body in the actual space is projected to the image coordinate system in combination with the attitude information of the flight control system and the camera calibration parameters, to obtain the body center pixel coordinate, and to establish a comparable coordinate system between the landing point reference and the body center. Including the following sub-steps:

[0075] Sub-step 2.1, synchronous acquisition of attitude and camera parameters.

[0076] Specifically, in the process of UAV landing control, the flight control system is built-in time synchronization unit, to trigger a fixed period (10-20 ms) of synchronization data acquisition task, the task is read from the inertial measurement unit (IMU) at the same time body attitude information, and from the camera control unit to obtain the calibration parameters, body attitude information and camera calibration parameters in the same task cycle is packaged as a data frame, and attached to the uniform timestamp, finally get the uniform timestamp of the attitude-camera parameter combination data.

[0077] Wherein, the attitude information includes roll angle (describing the body around the longitudinal axis of rotation state), pitch angle (describing the body around the lateral axis of rotation state) and yaw angle (describing the body around the vertical axis of rotation state). Camera calibration parameters include pixel focal length (divided into horizontal and vertical direction, obtained by camera imaging system in the calibration stage, used to describe the proportion of space point and pixel coordinates), optical center position (point corresponding to the intersection of the camera optical axis in the image coordinate system) and external parameter (for the installation position and installation angle of the camera, describing the relative position relationship between the camera coordinate system and the body coordinate system).

[0078] Substep 2.2, determine the position of the body geometric center in the body coordinate system.

[0079] Specifically, the body geometric center is defined as the intersection position of the arm, which is fixed in the body coordinate system origin in the design and manufacture, which is the geometric center of the UAV, and the design reference point of the center of gravity, which does not change with the flight attitude or environmental conditions of the UAV. In order to ensure the accuracy of subsequent calculation, the position of the body center is confirmed by three coordinate measuring machine in the manufacturing stage, and stored as a constant in the flight control system.

[0080] Substep 2.3, convert the body center position to the monitoring direction of the camera.

[0081] Specifically, the space transformation parameters between the body coordinate system and the camera coordinate system are obtained by camera calibration, including rotation matrix (describing the direction relationship) and translation vector (describing the position offset). When calculating, the position of the body center in the body coordinate system is mapped to the camera coordinate system through the space transformation relationship, so as to obtain the three-dimensional position of the body center in the monitoring direction of the camera. This process is used to convert the coordinates in the body view into the coordinates in the camera view, so as to ensure that the position of the body center in the camera monitoring image coordinate system is consistent with the actual position of the body center.

[0082] Substep 2.4, project the body center position to the image plane.

[0083] Specifically, the three-dimensional position of the body center in the camera coordinate system is converted into a two-dimensional pixel position on the image plane by using the camera perspective imaging principle. In this process, the three-dimensional space coordinates are first converted into pixel plane coordinates in proportion to the focal length and optical center position parameters, and then the conversion results are kept in the same coordinate system (i.e., the unified image coordinate system) as the landing mark pixel coordinates for direct comparison. In mass-produced models of unmanned aerial vehicles, if the camera installation position and angle are consistent, a mapping table of the body center position and the pixel coordinates can be obtained after the first calibration, and the body center pixel position can be directly obtained by looking up the table during operation, thereby reducing the real-time calculation burden.

[0084] In step S130, the pixel deviation between the body center pixel coordinates and the center pixel position in the image coordinate system is calculated, and the pixel deviation is converted into a spatial horizontal deviation and a spatial vertical deviation.

[0085] In some embodiments, the visual recognition-based unmanned aerial vehicle landing offset correction method provided by the present application specifically includes the following steps in step S130:

[0086] In step S131, the lateral pixel deviation and the longitudinal pixel deviation between the body center pixel coordinates and the center pixel position coordinates in the unified image coordinate system are calculated to output the pixel deviation composed of the lateral pixel deviation and the longitudinal pixel deviation.

[0087] In step S132, the pixel deviation is converted into an actual distance according to the imaging scale relationship of the visual sensor to output the spatial horizontal deviation and the spatial vertical deviation, and the spatial horizontal deviation and the spatial vertical deviation are combined into a binary vector, which is a relative deviation vector.

[0088] In a specific embodiment, the visual recognition-based unmanned aerial vehicle landing offset correction method provided by the present application includes the following sub-steps in step 3, relative deviation calculation. The lateral and longitudinal pixel differences between the landing mark center and the body center in the image coordinate system are calculated, and the pixel differences are converted into spatial horizontal deviation and vertical deviation using the camera calibration coefficient and the current height to obtain quantized errors for attitude and position correction.

[0089] Sub-step 3.1, obtaining the pixel positions of the landing mark and the body center.

[0090] Specifically, the center pixel position coordinates corresponding to the landing mark and the body center pixel position coordinates are placed in a unified image coordinate system (including a horizontal axis and a vertical axis, with units of pixels), and the two center pixel position coordinates are based on the same resolution, the same optical center position, and the same acquisition frame rate.

[0091] Sub-step 3.2, calculating the lateral and longitudinal pixel differences.

[0092] Specifically, the lateral pixel difference is the difference between the lateral pixel position of the landing mark center and the lateral pixel position of the body center, and the longitudinal pixel difference is the difference between the longitudinal pixel position of the landing mark center and the longitudinal pixel position of the body center.

[0093] Sub-step 3.3, converting the pixel difference into an actual space difference.

[0094] Specifically, the lateral and longitudinal pixel differences are converted into actual space distances according to the imaging scale relationship of the camera, and are divided into an actual deviation of the unmanned aerial vehicle in the horizontal plane in the lateral direction and an actual deviation of the unmanned aerial vehicle in the horizontal plane in the longitudinal direction. The actual deviation of the unmanned aerial vehicle in the horizontal plane in the lateral direction is equal to the ratio of the lateral pixel difference to the horizontal pixel focal length in the camera calibration parameters multiplied by the vertical height of the unmanned aerial vehicle from the landing plane. The actual deviation of the unmanned aerial vehicle in the horizontal plane in the longitudinal direction is equal to the ratio of the longitudinal pixel difference to the vertical pixel focal length in the camera calibration parameters multiplied by the vertical height of the unmanned aerial vehicle from the landing plane.

[0095] Sub-step 3.4, outputting a quantized error that can be used for attitude and position correction.

[0096] Specifically, the actual deviation of the unmanned aerial vehicle in the horizontal plane in the lateral direction and the actual deviation of the unmanned aerial vehicle in the horizontal plane in the longitudinal direction are combined into a two-dimensional vector as the relative position error of the current unmanned aerial vehicle and the landing mark, and are packaged into a data frame and sent to the unmanned aerial vehicle flight control system to provide input for subsequent aerodynamic compensation and correction instruction generation.

[0097] Step S140, determining a near-ground aerodynamic compensation coefficient corresponding to the unmanned aerial vehicle descent height according to the ratio of the shaft opening diameter to the unmanned aerial vehicle body rotor diameter and the unmanned aerial vehicle descent height.

[0098] In some embodiments, the present application provides a visual recognition-based unmanned aerial vehicle descent correction method, and step S140 specifically includes the following steps:

[0099] Step S141, calculating a dimensionless index and a deviation amplitude based on the relative deviation vector, the rotor diameter, the shaft opening diameter, and the unmanned aerial vehicle descent height. The dimensionless index includes the ratio of the unmanned aerial vehicle descent height to the rotor diameter and the ratio of the shaft opening diameter to the rotor diameter, and the deviation amplitude is the Euclidean norm of each component in the relative deviation vector.

[0100] Step S142, constructing a two-dimensional table offline according to the dimensionless index, and calculating the axial near-ground effect amplification coefficient by bilinear interpolation, and simultaneously calculating the asymmetric lift compensation coefficient and the lateral disturbance suppression coefficient corresponding to the linear scaling of the deviation amplitude online.

[0101] The near-ground aerodynamic compensation coefficient includes the axial near-ground effect amplification coefficient, the asymmetric lift compensation coefficient, and the lateral disturbance suppression coefficient.

[0102] In specific embodiments, the present application provides a visual recognition-based UAV descent correction method, step 4, aerodynamic compensation coefficient determination. According to the ratio of the shaft opening diameter to the body rotor diameter and the current descent height, the near-ground aerodynamic compensation coefficient is looked up or calculated, including the asymmetric lift compensation coefficient and the lateral disturbance suppression coefficient, which quantifies the asymmetric aerodynamic characteristics in the terminal stage and provides a correction factor for the correction instruction generation. It includes the following sub-steps:

[0103] Sub-step 4.1, aerodynamic compensation coefficient determination based on two-dimensional table interpolation method.

[0104] Specifically, based on the relative position error obtained in step 3 and the UAV body geometric parameters (rotor diameter, shaft opening diameter) and the UAV descent height, the dimensionless index (height ratio and diameter ratio) and the deviation amplitude are calculated. The height ratio is the ratio of the UAV descent height to the rotor diameter, and the diameter ratio is the ratio of the shaft opening diameter to the rotor diameter; the deviation amplitude is equal to the square root of the sum of the actual deviation of the UAV in the horizontal plane and the actual deviation in the horizontal plane, i.e. the Euclidean norm of the two.

[0105] Sub-step 4.2, axial near-ground effect amplification coefficient acquisition.

[0106] Specifically, a two-dimensional table is constructed in the offline calibration stage to record the required total thrust amplification at different height ratios and diameter ratios. During operation, bilinear interpolation is used:

[0107] Let the height ratio fall into the grid , the diameter ratio fall into the grid , and the four table point values are , then:

[0108] ;

[0109] Then

[0110] ;

[0111] In the formula, , the boundary is extrapolated and limited to the empirical safety range [1.0, 1.8] as a whole; is the interpolation weight in the height direction, indicating the position ratio of the current height ratio between the two adjacent table height nodes and ; when , it means falling exactly at the low height node and does not need to be interpolated upwards; when , it means falling exactly at the high height node and does not need to be interpolated downwards; when , it means in the middle of the two nodes, and the weight is half. Axial near-ground amplification factor.

[0112] It should be noted that in the process of offline building a two-dimensional table, in the typical working condition of the unmanned aerial vehicle entering the shaft mouth and landing, through the ground test platform or physical flight test, two key factors are gradually changed: one is the height ratio (the ratio of the current descending height to the rotor disc diameter), and the other is the diameter ratio (the ratio of the shaft mouth diameter to the rotor disc diameter). For each combination of height ratio and diameter ratio, the body is kept in a horizontal fixed-point hovering state, and a certain normalized horizontal deviation (for example, 0.2 times the rotor disc diameter) is artificially applied, and the change amount of the thrust distribution required by the flight control system to maintain the attitude stability under the interference is recorded. The thrust change amount is converted according to the unit normalized deviation to obtain the corresponding asymmetric lift base response value. Repeat the above measurement at different height ratio and diameter ratio combination points until the preset working condition range is covered. Finally, all the measured base response values are filled in the two-dimensional table with the height ratio as the vertical axis and the diameter ratio as the horizontal axis, as the look-up table basis for online compensation, and the boundary value is extrapolated and limited to the safe range outside the table.

[0113] It should be noted that, are the axial thrust amplification factors measured offline in advance, corresponding to the four corner points of the current index interval as the basis data points for bilinear interpolation, equivalent to the four corners of the map, and the values in the middle position need to be calculated according to the four corners in proportion. Among them, is the predicted value of low height ratio and low diameter ratio; is the predicted value of low height ratio and high diameter ratio; is the predicted value of high height ratio and low diameter ratio; is the predicted value of high height ratio and high diameter ratio.

[0114] Substep 4.3, asymmetric lift compensation coefficient acquisition based on deviation amplitude linear scaling.

[0115] Specifically, the two-dimensional table is established offline The value of the two-dimensional table represents the asymmetric lift base response (dimensionless, typical range 0.3-1.0) caused by unit normalized deviation, which is linearly scaled and limited according to the current deviation amplitude in online calculation, and the expression is:

[0116] ;

[0117] In the formula, represents the strength of the deviation relative to the rotor disc scale, is the horizontal deviation amplitude, is the rotor disc diameter; The minimum and maximum of the engineering limit are 0.85 and 1.25, respectively. If the minimum of the engineering limit is too small, the correction force is insufficient, which causes the attitude drift to fail to converge in time. If the maximum of the engineering limit is too large, the thrust of the paddle on one side may be too high, causing the aircraft to shake or even lose stability. The asymmetric lift compensation coefficient is a.

[0118] It should be noted that, The clipping function is used to limit a value within a specified upper and lower limit, and is expressed as:

[0119] ;

[0120] In the formula, ; .

[0121] Sub-step 4.4, obtaining the lateral disturbance suppression coefficient based on the attitude and lateral channel desensitization.

[0122] Specifically, the two-dimensional table is established offline The value of the two-dimensional table describes the attitude and lateral channel desensitization requirement (dimensionless, range 0.0-0.8) corresponding to the vertical shaft lateral backflow intensity. Online, the amplitude of the deviation is slightly enhanced and limited, and the expression is:

[0123] ;

[0124] In the formula, The linear enhancement coefficient is in the range of [0.2, 0.6]; The upper limit of the limit is 1.5; The lateral disturbance suppression coefficient is a.

[0125] It should be noted that the linear enhancement coefficient is used to amplify or weaken the compensation of the flight control to the lateral disturbance. The lateral correction amount (roll or lateral displacement command) is multiplied by the linear enhancement coefficient and then sent to the actuator. When the linear enhancement coefficient is equal to 0.2, it means that only 20% of the original correction amount is used to control the lateral channel to prevent oscillation caused by excessive correction, which is suitable for the end stage of the vertical shaft wall disturbance. When the linear enhancement coefficient is equal to 0.6, it means that the flight control only uses 60% of the original correction amount, which is suitable for the relatively stable descent interval in the aerodynamic environment.

[0126] Step S150, generating a flight control execution instruction set based on the spatial horizontal deviation, the spatial vertical deviation, and the near-ground aerodynamic compensation coefficient, and responding to the flight control execution instruction set to coordinate the aerodynamic disturbance to control the unmanned aerial vehicle to correct the descent action.

[0127] In some embodiments, the visual recognition-based unmanned aerial vehicle deviation correction method provided by the present application specifically includes the following steps:

[0128] Step S151, the spatial vertical deviation is weighted and corrected according to the axial ground effect amplification coefficient, to output an axial compensated vertical correction amount.

[0129] Step S152, the spatial horizontal deviation corresponding horizontal deviation amplitude is calculated, and the correction direction angle is determined according to the unmanned aerial vehicle rotor position angle, the rotor thrust is adjusted combined with the asymmetric lift compensation coefficient, to output the thrust adjustment amount corresponding to each rotor.

[0130] In some embodiments, the present application provides a visual recognition-based unmanned aerial vehicle deviation correction method, and step S150 specifically further includes the following steps:

[0131] Step S153, based on the spatial horizontal deviation and the lateral spoiler suppression coefficient, the target roll angle and the target pitch angle of the unmanned aerial vehicle attitude correction are calculated, and the target roll angle and the target pitch angle corresponding amplitude limiting are set.

[0132] Step S154, when the axial ground effect amplification coefficient exceeds the first threshold or the ratio of the unmanned aerial vehicle descent height to the rotor disc diameter is lower than the second threshold, the preset limit descent rate is executed, otherwise the terminal rate is executed.

[0133] Wherein, the flight control execution instruction set is obtained by packing the flight control execution instructions corresponding to the rotor thrust adjustment amount, the target roll angle and the target pitch angle, the limit descent rate and the terminal rate.

[0134] In a specific embodiment, the present application provides a visual recognition-based unmanned aerial vehicle deviation correction method, and step 5, the correction instruction generation. The relative deviation obtained in step 3 is combined with the aerodynamic compensation coefficient in step 4 to generate a flight control execution instruction set containing thrust distribution, attitude correction, and descent rate limitation, so that the correction action is synchronized with the change trend of the aerodynamic disturbance, and the asymmetric lift and turbulent disturbance caused by the coupling of the wall surface and the ground effect are offset, and the phase mismatch amplification drift is prevented. Including the following sub-steps:

[0135] Sub-step 5.1, aerodynamic compensation weighted deviation calculation.

[0136] Specifically, according to the axial ground effect amplification coefficient The spatial vertical deviation is weighted and corrected to obtain the axial compensated vertical correction amount , the expression is:

[0137] ;

[0138] When ≥0, it indicates the amplification degree of the ground effect on the vertical aerodynamic response, and when When the height is large, it represents the ground effect of the current height of the UAV and the wellhead geometry, which needs to be compensated in advance by increasing the vertical correction amplitude to offset the height fluctuation.

[0139] Sub-step 5.2, asymmetric thrust distribution calculation.

[0140] Specifically, based on the actual deviation of the UAV in the horizontal plane transversely and the actual deviation in the horizontal plane longitudinally , combined with the asymmetric lift compensation coefficient obtained in step 4 and the rotor position angle of the UAV, the horizontal deviation amplitude, the correction direction angle and the adjustment amount of the thrust of each rotor are calculated.

[0141] The expression for calculating the horizontal deviation amplitude is:

[0142] ;

[0143] The expression for calculating the correction direction angle is:

[0144] ;

[0145] The expression for calculating the thrust adjustment amount of each rotor is:

[0146] ;

[0147] In the formula, is the thrust proportional gain, ranging from 0.1 to 0.5 N / m. When the thrust adjustment amount is added to the basic hovering thrust, the new thrust control command is obtained.

[0148] Sub-step 5.3, attitude correction instruction generation.

[0149] Specifically, based on the actual deviation of the UAV in the horizontal plane transversely and the actual deviation in the horizontal plane longitudinally , combined with the lateral disturbance suppression coefficient obtained in step 4, the target roll angle and the target pitch angle are calculated. The target roll angle is the product of the corresponding attitude proportional coefficient, the lateral disturbance suppression coefficient and the actual deviation in the horizontal plane longitudinally ; the target pitch angle is the product of the opposite of the corresponding attitude proportional coefficient and the lateral disturbance suppression coefficient and the actual deviation in the horizontal plane transversely The product of the three; the target roll angle and the target pitch angle each corresponding to the attitude scale factor in the range of 2-5° per meter, after calculating the target roll angle and the target pitch angle, set the engineering limit (±8°) to prevent attitude oscillation caused by excessive correction.

[0150] Sub-step 5.4, descent rate limit and instruction packaging.

[0151] Specifically, according to the axial near-earth amplification coefficient Determine the target descent rate with the current height of the unmanned aerial vehicle:

[0152] When > 0.5 or the height ratio , limit the descent rate to below 0.3 m / s; otherwise, execute according to the normal terminal rate (0.4-0.6 m / s).

[0153] Finally, the aforementioned rotor thrust adjustment amount, target roll angle and target pitch angle, limited descent rate and terminal rate corresponding to the flight control execution instruction are packaged into an instruction set, and are issued once through the flight control bus to execute the instruction set and control the unmanned aerial vehicle for descent bias correction.

[0154] The following describes the unmanned aerial vehicle descent bias correction system based on visual recognition provided by the present application, which can be mutually corresponding and referenced with the unmanned aerial vehicle descent bias correction method based on visual recognition described above.

[0155] As Figure 2 shown, in one embodiment, an unmanned aerial vehicle descent bias correction system based on visual recognition includes a landing mark recognition module, a body center mapping module, a pixel deviation calculation module, a compensation coefficient determination module, and an unmanned aerial vehicle descent bias correction module.

[0156] The landing mark recognition module is used to acquire an image of the shaft opening area in the unmanned aerial vehicle landing mode through a visual sensor, and identify a preset landing mark in the image of the shaft opening area, to output a center pixel position corresponding to the landing mark.

[0157] The body center mapping module is used to project the geometric center position of the unmanned aerial vehicle body in the actual space to an image coordinate system unified with the landing mark based on the attitude information when the unmanned aerial vehicle descends and the calibration parameters of the visual sensor, to obtain a body center pixel coordinate.

[0158] The pixel deviation calculation module is used to calculate the pixel deviation of the body center pixel coordinate and the center pixel position in the image coordinate system, and convert the pixel deviation into a spatial horizontal deviation and a spatial vertical deviation.

[0159] The compensation coefficient determination module is configured to determine a near-ground aerodynamic compensation coefficient corresponding to the unmanned aerial vehicle descending height according to a ratio of a shaft mouth diameter to a propeller diameter of the unmanned aerial vehicle body and the unmanned aerial vehicle descending height.

[0160] The unmanned aerial vehicle descent bias correction module is configured to generate a flight control execution instruction set based on the spatial horizontal deviation, the spatial vertical deviation and the near-ground aerodynamic compensation coefficient, and correct the descending action of the unmanned aerial vehicle in cooperation with the aerodynamic disturbance in response to the flight control execution instruction set.

[0161] The applicant of the present application has made a detailed description and explanation of the embodiments of the present application in combination with the drawings of the specification, but those skilled in the art should understand that the above embodiments are only preferred embodiments of the present application, and the detailed description is only to help the reader better understand the spirit of the present application, and is not a limitation on the protection scope of the present application, on the contrary, any improvement or modification based on the spirit of the present application should fall within the protection scope of the present application.

Claims

1. A visual recognition-based UAV deskew correction method, characterized in that, The method comprises: acquiring, by a visual sensor, a shaft mouth area image in a landing mode of a UAV, and identifying a preset landing mark in the shaft mouth area image to output a center pixel position corresponding to the landing mark; projecting a geometric center position of a UAV body in an actual space to an image coordinate system unified with the landing mark based on attitude information when the UAV is descending and calibration parameters of the visual sensor to obtain a body center pixel coordinate; calculating a pixel deviation of the body center pixel coordinate and the center pixel position in the image coordinate system, and converting the pixel deviation into a spatial horizontal deviation and a spatial vertical deviation; determining a near-ground aerodynamic compensation coefficient corresponding to a UAV descent height according to a ratio of a shaft mouth diameter to a UAV body rotor diameter and the UAV descent height; generating a flight control execution instruction set based on the spatial horizontal deviation, the spatial vertical deviation and the near-ground aerodynamic compensation coefficient, and controlling the UAV to correct a descending action in cooperation with aerodynamic disturbance in response to the flight control execution instruction set; the calculating of the pixel deviation of the body center pixel coordinate and the center pixel position in the image coordinate system, and the converting of the pixel deviation into the spatial horizontal deviation and the spatial vertical deviation, comprises: calculating a lateral pixel deviation and a longitudinal pixel deviation between the body center pixel coordinate and the center pixel position coordinate in the unified image coordinate system to output the pixel deviation composed of the lateral pixel deviation and the longitudinal pixel deviation; converting the pixel deviation into an actual distance according to a UAV height through an imaging proportion relationship of the visual sensor to output the spatial horizontal deviation and the spatial vertical deviation, and combining the spatial horizontal deviation and the spatial vertical deviation into a binary vector, the binary vector being a relative deviation vector; the determining of the near-ground aerodynamic compensation coefficient corresponding to the UAV descent height according to the ratio of the shaft mouth diameter to the UAV body rotor diameter and the UAV descent height, comprises: calculating a dimensionless index and a deviation amplitude based on the relative deviation vector, the rotor diameter, the shaft mouth diameter and the UAV descent height, the dimensionless index comprising a ratio of the UAV descent height to the rotor diameter and a ratio of the shaft mouth diameter to the rotor diameter, and the deviation amplitude being an Euclidean norm of each component in the relative deviation vector; constructing a two-dimensional table offline according to the dimensionless index, and calculating an axial near-ground effect amplification coefficient through bilinear interpolation, and simultaneously calculating an asymmetric lift compensation coefficient and a lateral disturbance suppression coefficient corresponding to a linear scaling of the deviation amplitude online; wherein the near-ground aerodynamic compensation coefficient comprises the axial near-ground effect amplification coefficient, the asymmetric lift compensation coefficient and the lateral disturbance suppression coefficient. 2.The vision recognition based UAV deskew correction method of claim 1, wherein, the acquiring of the shaft mouth area image in the landing mode of the UAV by the visual sensor, and the identifying of the preset landing mark in the shaft mouth area image to output the center pixel position corresponding to the landing mark, comprises: after the UAV flight control system detects a landing trigger signal, controlling the UAV to enter the landing mode in response to the landing trigger signal, and simultaneously calling an on-board visual sensor to collect the shaft mouth area image at a fixed frame rate; Construct an original image matrix based on multiple fixed frame rate collected shaft mouth area images, and perform binaryzation processing on each image element in the original image matrix according to a preset black and white contrast and shape feature. 3.The vision recognition based UAV deskew correction method of claim 2, wherein, The shaft mouth area image in the unmanned aerial vehicle landing mode is acquired through the visual sensor, and a preset landing mark is identified in the shaft mouth area image to output a center pixel position corresponding to the landing mark, and the method further comprises: Performing an open operation processing on the original image matrix to remove noise of the image element and identify all connected pixel regions, calculating an area and an aspect ratio of the connected pixel regions to screen out a candidate pixel region meeting a preset mark specification; Calculating pixel coordinates of a bounding rectangle boundary box corresponding to each candidate pixel region, and performing a normalized matching between each candidate pixel region and a preset landing mark template to output a pixel region coordinate with a highest matching degree as the center pixel position of the landing mark.

4. The vision recognition based UAV deskew method of claim 1, wherein, The geometric center position of the unmanned aerial vehicle body in the actual space is projected to an image coordinate system unified with the landing mark based on the attitude information when the unmanned aerial vehicle is descending and the calibration parameters of the visual sensor to obtain a body center pixel coordinate, and the method comprises: Obtaining the attitude information when the unmanned aerial vehicle is descending from a flight control system of the unmanned aerial vehicle, the attitude information including a roll angle, a pitch angle and a yaw angle of the unmanned aerial vehicle, and performing time synchronization on the attitude information and the calibration coefficient to output combined data of the attitude information and the calibration coefficient with a unified time stamp; Determining the geometric center position of the unmanned aerial vehicle body in the actual space according to the structural parameters of the unmanned aerial vehicle body, mapping the geometric center position to an image coordinate system monitored by the visual sensor in combination with the combined data to determine the body center pixel coordinate.

5. The vision recognition based UAV deskew method of claim 1, wherein, The flight control execution instruction set is generated based on the spatial horizontal deviation, the spatial vertical deviation and the near-ground aerodynamic compensation coefficient, and the unmanned aerial vehicle is controlled to correct the descending action in response to the flight control execution instruction set, and the method further comprises: The spatial vertical deviation is weighted and corrected according to the axial near-ground effect amplification coefficient to output a vertical correction amount after axial compensation; The horizontal deviation amplitude corresponding to the spatial horizontal deviation is calculated, a correction direction angle is determined according to the position angle of the unmanned aerial vehicle rotor, the rotor thrust is adjusted in combination with the asymmetric lift compensation coefficient to output a thrust adjustment amount corresponding to each rotor.

6. The vision recognition based UAV deskew correction method of claim 5, wherein, The flight control execution instruction set is generated based on the spatial horizontal deviation, the spatial vertical deviation and the near-ground aerodynamic compensation coefficient, and the unmanned aerial vehicle is controlled to correct the descending action in response to the flight control execution instruction set, and the method further comprises: Based on the spatial horizontal deviation and the lateral spoiler suppression coefficient, a target roll angle and a target pitch angle of the unmanned aerial vehicle attitude correction are calculated, and an amplitude limit corresponding to the target roll angle and the target pitch angle is set; When the axial near-ground effect amplification coefficient exceeds a first threshold value or the ratio of the unmanned aerial vehicle descending height to the rotor disc diameter is lower than a second threshold value, a preset limit descending rate is executed, otherwise an end rate is executed. The flight control execution instruction set is a flight control execution instruction corresponding to each rotor thrust adjustment amount, a target roll angle, a target pitch angle, a limited descent rate, and a terminal speed, which is obtained by packing. 7.A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is configured to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-6.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method according to any one of claims 1-6.

Images