A UAV flight control method and system based on target image

Abstract

The present invention discloses a method and system for controlling a UAV flight based on a target image, and relates to the field of UAV flight control. The method comprises: obtaining the relative position of the UAV in the flight area based on the target image and the position calculation algorithm, and judging the yaw state of the UAV by the position comparison method; determining the flight yaw angle of the UAV by analyzing the target image using the recognition processing technology based on the yaw state judgment, and generating a yaw control amount according to the flight yaw angle to control the flight state of the UAV; collecting the real-time target image of the UAV after the flight state is adjusted, and analyzing the flight altitude of the UAV according to the real-time target image, and planning the landing trajectory of the UAV based on the flight altitude; generating an autonomous landing control instruction according to the landing trajectory of the UAV and the preset flight requirements to complete the autonomous landing flight control of the UAV. The present invention can accurately plan the landing trajectory of the UAV by collecting the target image in real time and analyzing the flight altitude.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle flight control, and in particular to a method and system for unmanned aerial vehicle flight control based on a target image. Background Art

[0002] UAV, also known as unmanned aircraft, is an unmanned aircraft controlled by radio remote control equipment and self-contained program control devices. According to different uses, UAVs can be mainly divided into two categories: military UAVs and civilian UAVs. From a technical point of view, UAVs can be divided into unmanned fixed-wing aircraft, unmanned vertical take-off and landing aircraft, unmanned airships, unmanned helicopters, unmanned multi-rotor aircraft, unmanned paragliders, etc. As an advanced aircraft, UAVs have been widely used and developed in many fields in recent years.

[0003] The basic principle of UAV flight control is to detect the flight status and environmental information of the UAV through sensors and feed it back to the controller. The controller calculates the control instructions of the UAV based on the feedback information and mission requirements and sends them to the actuator. The actuator adjusts the flight attitude and speed of the UAV according to the control instructions of the controller to achieve stable control of the UAV. Effective control of the UAV is the core of UAV technology.

[0004] However, the drone control technology in the prior art cannot generate the landing trajectory of the drone according to the flight altitude of the drone, so that the drone controller cannot pre-define the flight control instructions of the drone according to the landing trajectory, and thus cannot realize the autonomous landing flight of the drone, which reduces the autonomous operation capability and application scope of the drone.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for controlling a UAV flight based on a target image, the method comprising the following steps:

[0008] S1. Obtain the relative position of the UAV in the flight area based on the target image and position calculation algorithm, and determine the yaw state of the UAV by position comparison method;

[0009] S2, based on the yaw state judgment, the target image is analyzed using recognition processing technology to determine the flight yaw angle of the UAV, and the yaw control amount is generated according to the flight yaw angle to control the flight state of the UAV;

[0010] S3, collecting the real-time target image of the UAV after the flight state is adjusted, analyzing the flight altitude of the UAV according to the real-time target image, and planning the landing trajectory of the UAV based on the flight altitude;

[0011] S4. Generate an autonomous landing control command according to the landing trajectory of the UAV and the preset flight requirements, and send the autonomous landing control command to the controller to complete the autonomous landing flight control of the UAV;

[0012] Obtaining the relative position of the UAV in the flight area based on the target image and the position calculation algorithm, and determining the yaw state of the UAV by the position comparison method includes the following steps:

[0013] S11, using a camera to capture a flying target image of the UAV within a preset time period, and converting the target image into a color model, and the color model segments the target image according to a component threshold;

[0014] S12, after the segmentation is completed, a binary image of the target image is obtained, and the binary image is subjected to corrosion and expansion processing respectively by using morphological processing technology to determine the position of the flight point in the binary image;

[0015] S13, based on the position calculation algorithm, the flight point position is converted into a three-dimensional space coordinate, and the horizontal distance between the flight point position and the ground point is determined to obtain the relative position of the UAV and the ground;

[0016] S14. Restore the flight trajectory of the UAV according to the relative position, determine the overlap between the relative position flight trajectory and the reference position flight trajectory, and analyze the yaw state of the UAV based on the overlap result.

[0017] Preferably, restoring the flight trajectory of the drone according to the relative position, determining the overlap between the flight trajectory of the relative position and the flight trajectory of the set position, and analyzing the yaw state of the drone based on the overlap result comprises the following steps:

[0018] S141, inputting the relative position values ​​of the UAV and the ground into the visualization analysis software, connecting the relative position values ​​in the order of timestamps, and restoring the flight trajectory of the UAV in the flight area;

[0019] S142, using a matching positioning algorithm to determine the overlap distance between the relative position flight trajectory and the set position flight trajectory at the same flight position;

[0020] S143, setting the trajectory point overlap distance as the estimated value of the yaw angle of the UAV, and adding all the estimated values ​​of the yaw angle to obtain the yaw deviation state of the UAV.

[0021] Preferably, the calculation formula of the trajectory point overlap distance is: ;

[0022] In the formula, dis represents the overlap distance of trajectory points, p i Indicated in i Set the flight trajectory point under each flight position. q i Indicated in i The relative position flight trajectory points under the flight position, i Indicates the number of flight position points, tw max Indicates the preset maximum threshold of the trajectory point overlap distance.

[0023] Preferably, collecting a real-time target image of the UAV after the flight state is adjusted, analyzing the flight altitude of the UAV according to the real-time target image, and planning the landing trajectory of the UAV based on the flight altitude includes the following steps:

[0024] S31, using a camera to capture a real-time target image of the drone after the flight state is adjusted, and selecting a sample area calculated by the flight altitude from the real-time target image;

[0025] S32, obtaining the pixel value of each pixel point in the flight height calculation sample area, and converting the pixel value into a linear color system value based on a linear system calculation formula;

[0026] S33, judging the type of the pixel point based on the linear color system value, and establishing a UAV flight altitude algorithm according to the judgment result to obtain the UAV flight altitude;

[0027] S34. Characterize the UAV landing trajectory space based on the flight altitude and the dynamic window, and construct a target probability distribution model according to the landing trajectory space to generate the optimal landing trajectory of the UAV.

[0028] Preferably, judging the type of the pixel point based on the linear color system value, and establishing a UAV flight altitude algorithm to obtain the UAV flight altitude according to the judgment result includes the following steps:

[0029] S331, determining the type of the pixel point based on the linear color system value and the preset flight height threshold range, the type including building area, ground area and vegetation area;

[0030] S332, obtaining the types of all pixel points, and determining the proportion of each type, dividing the sample area in the east-west, south-north directions using the longitude and latitude grid division technology and the size of the sample area, and performing a positioning operation on the pixel points according to the division results to obtain positioning points;

[0031] S333, setting a ruler position point for the pixel point at the center position of the sample area, and deriving a horizontal distance of the pixel point on the center straight line of the sample area according to the ruler position point and the positioning point;

[0032] S334. Determine the imaging height corresponding to the pixel points based on the horizontal distance, and generate an initial UAV flight height algorithm according to the imaging height and the horizontal distance.

[0033] Preferably, characterizing the UAV landing trajectory space based on the flight altitude and the dynamic window, and constructing a target probability distribution model according to the landing trajectory space to generate the optimal landing trajectory of the UAV includes the following steps:

[0034] S341. Based on the descriptive coordinate system and the flight altitude of the UAV, the displacement of the UAV at each moment is decomposed into s Axis and t Axis in both directions, s The axis represents the direction of the drone parallel to the ground. t The axis represents the direction of the drone perpendicular to the ground;

[0035] S342, based on s Axis direction, t The three sets of parameters, namely, the axis direction and the flight speed range, generate a dynamic window, and characterize the feasible landing trajectory space of the UAV according to the dynamic window;

[0036] S343, constructing a target probability distribution based on the landing feasible trajectory space, and generating a set of candidate trajectories that meet the requirements for the landing of the UAV according to the target probability distribution and the stratified sampling strategy;

[0037] S344. A safety model based on the uncertainty of the UAV's flight speed selects an optimal trajectory from a set of candidate trajectories as a basis for generating an optimal landing trajectory for the UAV.

[0038] Preferably, constructing a target probability distribution based on the landing feasible trajectory space, and generating a set of candidate trajectories that meet the landing requirements of the UAV according to the target probability distribution and the stratified sampling strategy includes the following steps:

[0039] S3431, based on the feasible landing trajectory space s A Gaussian distribution centered on the highest point of the UAV’s flight altitude is established in the axial direction, and the cumulative probability within the dynamic window range is determined;

[0040] S3432, weighting the cumulative probability based on the maximum flight speed allowed by the dynamic window range, and normalizing the weighted cumulative probability to obtain the window selection probability;

[0041] S3433, constructing a probability distribution of the expected flight speed in the dynamic window according to the window selection probability, analyzing the probability distribution of the acceleration of the UAV based on the probability distribution result, and randomly sampling a set of height values ​​from the UAV flight altitude as the horizontal coordinate of the target point;

[0042] S3434. Based on the stratified sampling strategy of speed first and path later, the sampling order is set to the expected speed, acceleration and the horizontal coordinate of the target point to generate a set of candidate trajectories that meet the landing requirements of the UAV.

[0043] Preferably, based on the stratified sampling strategy of speed first and path second, setting the sampling order to sequentially generate a set of candidate trajectories that meet the requirements of drone landing as expected speed, acceleration and target point abscissa includes the following steps:

[0044] S34341, taking the acceleration probability distribution, the expected flight speed probability distribution, the target point horizontal coordinate, the UAV flight altitude and the window selection probability as input, and the candidate trajectory set as output;

[0045] S34342, sampling the window selection probability, the probability distribution of the expected flight speed, the acceleration probability and the target point horizontal coordinate to generate a target window, a target flight speed, a target acceleration and a horizontal coordinate respectively;

[0046] S34343. Generate a speed curve according to the target flight speed and the target acceleration, and project the speed curve into time space to solve the quadratic programming problem to generate a smooth speed curve;

[0047] S34344. Calculate the target point's ordinate based on the target window and the abscissa, and generate a set of candidate trajectories that meet the requirements for UAV landing by combining the smooth speed curve and the Bezier curve.

[0048] Preferably, generating an autonomous landing control instruction according to the landing trajectory of the UAV and the preset flight requirements, and sending the autonomous landing control instruction to the controller to complete the autonomous landing flight control of the UAV includes the following steps:

[0049] S41, obtaining the environmental conditions when the drone is to land based on the preset flight requirements, and analyzing the thrust and yaw angle required by the drone during the landing process according to the environmental conditions and the landing trajectory of the drone;

[0050] S42, generating an autonomous landing control instruction using a model predictive control algorithm based on the thrust and yaw angle analysis results, and sending the autonomous landing control instruction to a controller;

[0051] S43, the controller receives and executes the autonomous landing control command to adjust the flight attitude of the UAV, implements the autonomous landing flight control of the UAV, and monitors the actual flight status of the UAV during the landing process;

[0052] S44, determining the deviation between the actual flight status result and the landing trajectory of the UAV, and adjusting the control command based on the deviation to complete the autonomous landing flight control of the UAV.

[0053] In a second aspect, the present invention further provides a UAV flight control system based on a target image, the UAV flight control system comprising:

[0054] The yaw state judgment module is used to obtain the relative position of the UAV in the flight area based on the target image and the position solution algorithm, and to judge the yaw state of the UAV by the position comparison method;

[0055] The flight state control module is used to determine the flight yaw angle of the UAV by analyzing the target image using recognition and processing technology based on the yaw state judgment, and to generate a yaw control amount according to the flight yaw angle to control the flight state of the UAV;

[0056] The landing trajectory planning module is used to collect the real-time target image of the UAV after the flight status is adjusted, analyze the flight altitude of the UAV according to the real-time target image, and plan the landing trajectory of the UAV based on the flight altitude;

[0057] The flight landing control module is used to generate autonomous landing control instructions according to the UAV landing trajectory and preset flight requirements, and send the autonomous landing control instructions to the controller to complete the autonomous landing flight control of the UAV.

[0058] The beneficial effects of the present invention are:

[0059] 1. The present invention is based on a position calculation algorithm and a position comparison method, and can determine the yaw state of the UAV in real time, analyze the flight yaw angle according to the yaw state, and generate a yaw control amount accordingly, which helps the UAV to quickly adjust the flight attitude, optimize the flight path, reduce unnecessary flight time and energy consumption, and improve flight efficiency. At the same time, by real-time acquisition of target images and analysis of flight altitude, the landing trajectory of the UAV can be accurately planned, and autonomous landing control instructions can be generated according to the landing trajectory and preset flight requirements, thereby realizing autonomous landing flight control of the UAV without human intervention, thereby improving the autonomous operation capability and application scope of the UAV.

[0060] 2. The present invention converts pixel values ​​into linear color system values ​​and utilizes scientific linear system calculation formulas to make data processing more standardized and systematic, providing a reliable basis for subsequent judgment and calculation. At the same time, the type of pixel points is judged based on the linear color system value, and the accuracy of flight altitude calculation is improved based on sophisticated classification processing. The flight altitude and dynamic window are used to characterize the landing trajectory space of the UAV, and a trajectory probability distribution model is constructed in combination with the Bayesian network to generate the optimal landing trajectory of the UAV. Through real-time image acquisition, precise data processing and intelligent trajectory planning, the landing safety of the UAV is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0062] Figure 1 is a flow chart of a method for controlling a drone flight based on a target image according to an embodiment of the present invention;

[0063] Figure 2 The present invention is a block diagram of a UAV flight control system based on a target image according to an embodiment of the present invention.

[0064] In the figure:

[0065] 1. Yaw state judgment module; 2. Flight state control module; 3. Landing trajectory planning module; 4. Flight landing control module. DETAILED DESCRIPTION

[0066] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0067] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0068] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0069] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0070] See also Figure 1 The present invention provides a UAV flight control method based on a target image, and the UAV flight control method comprises the following steps:

[0071] S1. Obtain the relative position of the UAV in the flight area based on the target image and position calculation algorithm, and determine the yaw state of the UAV through the position comparison method.

[0072] In this embodiment, obtaining the relative position of the UAV in the flight area based on the target image and the position calculation algorithm, and determining the yaw state of the UAV by the position comparison method includes the following steps:

[0073] S11, using a camera to capture the flying target image of the UAV within a preset time period, and converting the target image into a color model, and the color model segments the target image according to a component threshold.

[0074] Specifically, when a camera is used to capture a flying target image of a UAV within a preset time period, and the target image is converted into a color model, and the color model segments the target image according to a component threshold, a camera can be configured at the bottom of the UAV, and the camera parameters, including focal length, exposure, frame rate, etc., can be set. Within the set time period, the camera continuously captures the flying image of the UAV; the captured RGB image is converted into a selected HSV color model to separate color features by hue (H) and saturation (S), and the image is smoothed and denoised at the same time, and a color component threshold is set in the HSV color model to distinguish pixels in the image that meet a specific color range to form a target image.

[0075] S12, after the segmentation is completed, a binary image of the target image is obtained, and morphological processing technology is applied to the binary image to perform corrosion and expansion processing respectively, so as to determine the position of the flying point in the binary image.

[0076] Specifically, after the segmentation is completed, a binary image of the target image is obtained, and the morphological processing technology is used to perform corrosion and expansion processing on the binary image respectively. When determining the flight point position in the binary image, the binary image can be corroded by a structural element to eliminate small noise points on the edge of the target and make the boundary of the target clearer; after corrosion, the image is expanded using the same structural element to restore the original size of the target or fill small gaps generated during the corrosion process; after corrosion and expansion, a connected component labeling algorithm is applied to identify and label each independent target in the image, and the geometric center or centroid is calculated for each connected area, and the center is defined as the flight point position of the UAV.

[0077] S13. Based on the position calculation algorithm, the flight point position is converted into three-dimensional space coordinates, and the horizontal distance between the flight point position and the ground point is determined to obtain the relative position of the UAV and the ground.

[0078] Specifically, when the flight point position is converted into three-dimensional space coordinates based on the position solution algorithm, and the horizontal distance between the flight point position and the ground point is determined to obtain the relative position of the UAV and the ground, the three-dimensional coordinates of the ground reference point can be obtained. The specific three-dimensional coordinates of the ground reference point are preset landmarks or obtained through other methods such as GPS positioning; the horizontal distance between the UAV flight point and the ground reference point is calculated, usually by ignoring the height difference and only calculating the Euclidean distance of the X and Y coordinate differences; the relative position of the UAV and the ground is analyzed based on the horizontal distance and the flight altitude.

[0079] S14. Restore the flight trajectory of the UAV according to the relative position, determine the overlap between the relative position flight trajectory and the reference position flight trajectory, and analyze the yaw state of the UAV based on the overlap result.

[0080] Specifically, restoring the flight trajectory of the drone according to the relative position, determining the overlap between the flight trajectory of the relative position and the flight trajectory of the set position, and analyzing the yaw state of the drone based on the overlap result include the following steps:

[0081] S141, inputting the relative position values ​​of the UAV and the ground into the visualization analysis software, connecting the relative position values ​​in the order of timestamps, and restoring the flight trajectory of the UAV in the flight area;

[0082] S142, using a matching positioning algorithm to determine the overlap distance between the relative position flight trajectory and the set position flight trajectory at the same flight position;

[0083] S143, setting the trajectory point overlap distance as the estimated value of the yaw angle of the UAV, and adding all the estimated values ​​of the yaw angle to obtain the yaw deviation state of the UAV.

[0084] Among them, the calculation formula of trajectory point coincidence distance is: ;

[0085] In the formula, dis represents the overlap distance of trajectory points, p i Indicated in i Set the flight trajectory point under each flight position. q i Indicated in i The relative position flight trajectory points under the flight position, i Indicates the number of flight position points, tw maxIndicates the preset maximum threshold of the trajectory point overlap distance.

[0086] Specifically, when the trajectory point overlap distance is set as the estimated value of the yaw angle of the UAV, and all the yaw angle estimates are added to obtain the yaw deviation state of the UAV, a threshold value can be pre-set to evaluate the degree of deviation of the UAV from the set trajectory during flight. This distance characterizes the yaw angle between the actual flight position of the UAV and the predetermined trajectory point; the yaw angle is calculated according to the overlap distance between the actual position of the UAV and the predetermined trajectory point and the vector angle difference; the yaw angle estimates of all trajectory points are accumulated to obtain the total yaw deviation, and the average yaw deviation is calculated at the same time, that is, the total yaw deviation is divided by the number of trajectory points to obtain the average state of the UAV yaw.

[0087] S2. Based on the yaw state judgment, the target image is analyzed using recognition processing technology to determine the flight yaw angle of the UAV, and the yaw control amount is generated according to the flight yaw angle to control the flight state of the UAV.

[0088] In this embodiment, when the flight yaw angle of the UAV is determined by analyzing the target image using recognition processing technology based on yaw state judgment, and the yaw control amount is generated according to the flight yaw angle to control the flight state of the UAV, a corresponding control algorithm, such as a PID controller, can be designed according to the size and direction of the yaw angle to calculate the yaw control amount that needs to be adjusted, and the calculated yaw control amount is converted into a control signal and input into the flight control system of the UAV; the yaw state of the UAV is adjusted according to the yaw control amount to ensure the correctness of the flight direction and the accuracy of the route; the yaw state is continuously monitored during the flight, and the control amount is adjusted according to the real-time data to cope with possible environmental changes and flight dynamics.

[0089] In order to facilitate understanding of the technical solution of the present invention, the control operation of the UAV flight state during the actual operation of the present invention is described in detail below.

[0090] Step 1: Target image acquisition and processing;

[0091] (1) Image capture and processing:

[0092] The focal length of the camera configuration is 35 mm; the exposure time is 1 / 1000 second; the frame rate is 30fps; the capture period is from 10 seconds to 20 seconds after the start of flight; the color model is converted from RGB to HSV; the hue (H) threshold is 100° to 140° (green range); the saturation (S) threshold is 0.25 to 1.0;

[0093] (2) Image binarization and morphological processing:

[0094] Based on the binary threshold definition, the pixels within the hue and saturation thresholds are set to 1 (white), and the others are set to 0 (black). The morphological processing includes erosion and expansion. The structural element of the erosion is a 3x3 square, the number of iterations is 1, and the structural element of the expansion is a 3x3 square, the number of iterations is 1. The geometric center of the connected area is calculated to determine the flight point position.

[0095] Step 2: Calculate yaw deviation;

[0096] (1) Position solution and horizontal distance calculation: The position solution algorithm uses triangulation. The coordinates of the ground reference points are set to (50m, 50m, 0m), the flight altitude is 30m, and the horizontal distance is calculated using the Euclidean distance formula, ignoring the altitude information.

[0097] (2) Flight trajectory restoration and yaw judgment: After the relative position values ​​at each time point are connected to achieve trajectory restoration, the trajectory coincidence is calculated. The yaw angle threshold is set to 5°, and the yaw angle is estimated based on the degree of deviation between the flight point and the set trajectory. The yaw angle estimation values ​​of all trajectory points are summed to obtain the yaw deviation, and the average yaw deviation is calculated.

[0098] Step 3: Yaw control;

[0099] The yaw angle control algorithm uses a PID controller: the P (proportional) coefficient is 0.1; the I (integral) coefficient is 0.01; the D (differential) coefficient is 0.05; the required yaw control amount is calculated based on the yaw angle size and direction;

[0100] Control signal application: Apply control signals to the servos or other related mechanical parts of the drone to adjust the heading.

[0101] S3. Collect the real-time target image of the UAV after the flight status is adjusted, analyze the flight altitude of the UAV according to the real-time target image, and plan the landing trajectory of the UAV based on the flight altitude.

[0102] In this embodiment, collecting a real-time target image of the drone after the flight state is adjusted, analyzing the flight altitude of the drone according to the real-time target image, and planning the landing trajectory of the drone based on the flight altitude includes the following steps:

[0103] S31. Using a camera to capture a real-time target image of the UAV after the UAV is adjusted in flight state, and selecting a sample area for calculating the flight altitude from the real-time target image.

[0104] Specifically, when using a camera to capture the real-time target image of the UAV after the flight state is adjusted, and selecting the sample area for flight altitude calculation from the real-time target image, the real-time target image can be continuously captured after the flight state of the UAV is adjusted, and the captured image can be transmitted to the ground control station or processing system in real time; the real-time target image is preliminarily processed, including filtering, contrast enhancement, etc. to improve the image quality, and image recognition technology, such as edge detection, color segmentation, etc., is used to identify significant features or specific areas in the image; the size and position of the sample area are defined according to the requirements of flight altitude calculation, and the specific sample area should cover the specific area under the UAV so as to accurately calculate the flight altitude; and the defined sample area is selected from the real-time target image to ensure that the area contains sufficient information for accurate calculation of the flight altitude.

[0105] S32, obtaining the pixel value of each pixel point in the flight height calculation sample area, and converting the pixel value into a linear color system value based on a linear system calculation formula.

[0106] Specifically, when obtaining the pixel value of each pixel point in the flight altitude calculation sample area and converting the pixel value into a linear color system value based on a linear system calculation formula, the color data of each pixel point can be extracted from the sample area, which usually involves reading RGB (red, green, blue) values ​​or corresponding values ​​of other color models, and converting the pixel value into a linear color system value using a nonlinear to linear conversion formula.

[0107] The expression of the nonlinear to linear conversion formula is: ;

[0108] In the formula, C Represents RGB color value, Y Represents output values ​​in linear color space.

[0109] S33. Determine the type of the pixel point based on the linear color system value, and establish a UAV flight altitude algorithm according to the determination result to obtain the UAV flight altitude.

[0110] Specifically, judging the type of pixel points based on the linear color system value, and establishing a UAV flight altitude algorithm to obtain the UAV flight altitude according to the judgment result includes the following steps:

[0111] S331, determining the type of the pixel point based on the linear color system value and the preset flight height threshold range, the type including building area, ground area and vegetation area;

[0112] S332, obtaining the types of all pixel points, and determining the proportion of each type, dividing the sample area in the east-west, south-north directions using the longitude and latitude grid division technology and the size of the sample area, and performing a positioning operation on the pixel points according to the division results to obtain positioning points;

[0113] S333, setting a ruler position point for the pixel point at the center position of the sample area, and deriving a horizontal distance of the pixel point on the center straight line of the sample area according to the ruler position point and the positioning point;

[0114] S334. Determine the imaging height corresponding to the pixel points based on the horizontal distance, and generate an initial UAV flight height algorithm according to the imaging height and the horizontal distance.

[0115] It needs to be explained that the expression of the drone flight altitude algorithm is: ;

[0116] In the formula, H Indicates the flight altitude of the drone. x Indicates the horizontal distance of the center straight line of the sample area, T m Indicates m The imaging height of pixels, m Indicates the number of pixels. D m Indicates m The ruler position point of the pixel point at the center of the sample area.

[0117] S34. Characterize the UAV landing trajectory space based on the flight altitude and the dynamic window, and construct a target probability distribution model according to the landing trajectory space to generate the optimal landing trajectory of the UAV.

[0118] Specifically, characterizing the UAV landing trajectory space based on the flight altitude and the dynamic window, and constructing a target probability distribution model according to the landing trajectory space to generate the optimal landing trajectory of the UAV includes the following steps:

[0119] S341. Based on the descriptive coordinate system and the flight altitude of the UAV, the displacement of the UAV at each moment is decomposed into s Axis and t Axis in both directions, s The axis represents the direction of the drone parallel to the ground. t The axis represents the direction of the drone perpendicular to the ground;

[0120] S342, based on s Axis direction, t The three sets of parameters, namely, the axis direction and the flight speed range, generate a dynamic window, and characterize the feasible landing trajectory space of the UAV according to the dynamic window;

[0121] S343, constructing a target probability distribution based on the landing feasible trajectory space, and generating a set of candidate trajectories that meet the requirements for the landing of the UAV according to the target probability distribution and the stratified sampling strategy;

[0122] S344. A safety model based on the uncertainty of the UAV's flight speed selects an optimal trajectory from a set of candidate trajectories as a basis for generating an optimal landing trajectory for the UAV.

[0123] Among them, constructing the target probability distribution based on the landing feasible trajectory space, and generating a set of candidate trajectories that meet the landing requirements of the UAV according to the target probability distribution and the stratified sampling strategy include the following steps:

[0124] S3431, based on the feasible landing trajectory space s A Gaussian distribution centered on the highest point of the UAV’s flight altitude is established in the axial direction, and the cumulative probability within the dynamic window range is determined;

[0125] S3432, weighting the cumulative probability based on the maximum flight speed allowed by the dynamic window range, and normalizing the weighted cumulative probability to obtain the window selection probability;

[0126] S3433, constructing a probability distribution of the expected flight speed in the dynamic window according to the window selection probability, analyzing the probability distribution of the acceleration of the UAV based on the probability distribution result, and randomly sampling a set of height values ​​from the UAV flight altitude as the horizontal coordinate of the target point;

[0127] S3434. Based on the stratified sampling strategy of speed first and path later, the sampling order is set to the expected speed, acceleration and the horizontal coordinate of the target point to generate a set of candidate trajectories that meet the landing requirements of the UAV.

[0128] Specifically, based on the stratified sampling strategy of speed first and path second, setting the sampling order to be the expected speed, acceleration and target point horizontal coordinate in turn to generate a set of candidate trajectories that meet the requirements of drone landing includes the following steps:

[0129] S34341, taking the acceleration probability distribution, the expected flight speed probability distribution, the target point horizontal coordinate, the UAV flight altitude and the window selection probability as input, and the candidate trajectory set as output;

[0130] S34342, sampling the window selection probability, the probability distribution of the expected flight speed, the acceleration probability and the target point horizontal coordinate to generate a target window, a target flight speed, a target acceleration and a horizontal coordinate respectively;

[0131] S34343. Generate a speed curve according to the target flight speed and the target acceleration, and project the speed curve into time space to solve the quadratic programming problem to generate a smooth speed curve;

[0132] S34344. Calculate the target point's ordinate based on the target window and the abscissa, and generate a set of candidate trajectories that meet the requirements for UAV landing by combining the smooth speed curve and the Bezier curve.

[0133] Among them, before the candidate trajectory set is completed, it is necessary to determine whether the landing path curve meets the curvature constraint. If it meets the curvature constraint, the landing path curve is added to the candidate trajectory set, and steps S34342 to S34344 are repeated until the construction of the candidate trajectory set is completed. If it does not meet the curvature constraint, the landing path curve is eliminated.

[0134] It needs to be explained that when selecting the optimal trajectory from the candidate trajectory set based on the safety model of the uncertainty of the UAV's flight speed as the optimal landing trajectory for the UAV, an uncertainty parameter can be assigned to each candidate trajectory to reflect the uncertainty of the flight speed, and a safety assessment model can be established to consider the impact of the uncertainty of the UAV's flight speed on the trajectory safety. At the same time, a risk assessment is performed on each candidate trajectory, considering possible risk factors such as collision risk and flight speed fluctuations. The safety model is used to evaluate the safety of each candidate trajectory, and the evaluation results are output. The evaluation results are sorted, giving priority to trajectories with high safety, and then the trajectory with the highest safety is selected from the safety assessment results as the optimal landing trajectory.

[0135] S4. Generate an autonomous landing control command according to the landing trajectory of the UAV and the preset flight requirements, and send the autonomous landing control command to the controller to complete the autonomous landing flight control of the UAV.

[0136] In this embodiment, generating an autonomous landing control instruction according to the landing trajectory of the drone and the preset flight requirements, and sending the autonomous landing control instruction to the controller to complete the autonomous landing flight control of the drone includes the following steps:

[0137] S41. Obtain the environmental conditions when the drone is about to land based on the preset flight requirements, and analyze the thrust and yaw angle required by the drone during the landing process according to the environmental conditions and the landing trajectory of the drone.

[0138] Specifically, when obtaining the environmental conditions at the time of landing based on the preset flight requirements of the UAV, and analyzing the thrust and yaw angle required for the UAV during the landing process according to the environmental conditions and the landing trajectory of the UAV, meteorological sensors, GPS, radar, etc. can be used to collect environmental data of the landing area, including wind speed, wind direction, temperature, humidity, etc., analyze the collected data, and evaluate the environmental conditions at the time of landing, such as the impact of wind on the stability of the UAV; use flight simulation software to simulate the flight state according to the environmental conditions and the landing trajectory to predict the performance of the UAV in the actual environment; based on the dynamic model of the UAV and the current flight state, calculate the thrust and yaw angle required to realize the flight of the UAV (calculate the thrust according to the mass of the UAV, wind force and other environmental factors, and at the same time calculate the yaw angle adjustment required to maintain the predetermined trajectory under the influence of wind force).

[0139] S42. Generate an autonomous landing control instruction using a model predictive control algorithm based on the thrust and yaw angle analysis results, and send the autonomous landing control instruction to a controller.

[0140] Specifically, when the autonomous landing control command is generated by the model predictive control algorithm based on the thrust and yaw angle analysis results and the autonomous landing control command is sent to the controller, the dynamic model of the UAV can be pre-defined, including state variables (such as position, velocity, acceleration) and control variables (such as thrust and yaw angle), and environmental data (such as wind speed and wind direction) are used as external inputs of the model to affect the dynamic behavior of the UAV; performance indicators are set, usually minimizing the deviation and energy consumption of the landing trajectory, and physical constraints (such as maximum thrust, maximum yaw angle) and operational constraints (such as safety distance, obstacle avoidance rules) are set, and a prediction range (such as the flight trajectory in the next few seconds) is set by using the model predictive control algorithm MPC, and an optimization algorithm (such as linear programming, quadratic programming) is used to calculate the optimal control input that satisfies all constraints within a given prediction window; according to the results of the MPC algorithm, the thrust and yaw angle adjustment amount for each time step is calculated, and the calculated control command is formatted into a form that can be recognized by the UAV control system; the control command is sent to the flight controller of the UAV in real time through a wireless communication interface (such as Wi-Fi or radio signals).

[0141] S43, the controller receives and executes the autonomous landing control command to adjust the flight attitude of the UAV, implements the autonomous landing flight control of the UAV, and monitors the actual flight status of the UAV during the landing process.

[0142] S44, determining the deviation between the actual flight status result and the landing trajectory of the UAV, and adjusting the control command based on the deviation to complete the autonomous landing flight control of the UAV.

[0143] In order to facilitate understanding of the technical solution of the present invention, the control operation of the drone landing during actual operation of the present invention is described in detail below.

[0144] Step 1: Real-time target image acquisition and sample area selection;

[0145] (1) Camera configuration and image capture: The camera model is set to Sony Alpha a7; the resolution is 6000x4000 pixels; the frame rate is 30 frames per second; the focal length is 35 mm, providing a wide-angle field of view; the image capture time is 5 seconds after the flight state is adjusted;

[0146] (2) Image transmission and preliminary processing: The image is transmitted to the ground control station in real time via the 5G network. Image processing includes denoising using a Gaussian filter (5x5 kernel) and real-time contrast enhancement.

[0147] (3) Sample area selection: The sample area size is 1000x1000 pixels and is located in the center of the image. The function of the sample area is to estimate the flight altitude and analyze the ground features.

[0148] Step 2: Linear color system conversion and pixel value acquisition;

[0149] (1) Pixel value reading and color space conversion: A color model is selected to convert RGB to CIEXYZ and then linearize it. In the pixel processing quantity, color data is extracted and converted from 1,000,000 pixels in the sample area.

[0150] (2) Flight altitude algorithm and pixel type judgment: the ground brightness threshold is 50% of the XYZ brightness value, the building is 75%, and the vegetation is 30%. The ground, building, or vegetation is judged based on the XYZ brightness value;

[0151] Height calculation method: By comparing a known object (such as a building with a height of 20 meters) with its visual size in the image, the drone's height is estimated using trigonometric functions.

[0152] Step 3: UAV landing trajectory space;

[0153] (1) Dynamic window setting and trajectory planning: The dynamic window size is set to 20 meters ahead according to the current speed of the drone (10 m / s). When constructing the probability model, the probability weight is calculated based on the Gaussian distribution set based on the flight altitude and speed.

[0154] (2) Trajectory generation and selection: A sampling strategy is formulated based on the probability sampling of acceleration and velocity according to the set Gaussian distribution, and the trajectory corresponding to the maximum probability value is selected as the optimal landing path.

[0155] Step 4: Obtain environmental conditions and analyze thrust and yaw angle;

[0156] (1) Environmental data collection: The sensor types used are GPS, radar, and anemometer. The data collection frequency is set to collect data once per second. The collected data include wind speed (average 5 m / s, direction northeast), temperature (15°C), and humidity (60%).

[0157] (2) Flight simulation and thrust yaw angle prediction: ANSYS Fluent was used as the simulation software, and the prediction accuracy was set to 10 milliseconds as the calculation step. Based on the drone mass of 1.5 kg, the required thrust = 1.5 kg × 9.81 m / s 2 ×1.2 (safety factor); when the wind speed is 5m / s, the yaw angle needs to be adjusted to 5°.

[0158] Step 5: Generate autonomous landing control instructions;

[0159] (1) MPC settings: control period is 100 milliseconds; prediction window is 2 seconds in the future; optimization algorithm is quadratic programming;

[0160] (2) Control command generation: The thrust is 16.5 Newtons, the yaw angle is adjusted by 0.5° every 100 milliseconds, and the total adjustment takes 10 seconds.

[0161] Therefore, the present invention can accurately plan the landing trajectory of the UAV by collecting target images in real time and analyzing the flight altitude, generate autonomous landing control instructions according to the landing trajectory and preset flight requirements, and realize autonomous landing flight control of the UAV without human intervention, thereby improving the autonomous operation capability and application scope of the UAV.

[0162] See also Figure 2 The present invention also provides a UAV flight control system based on a target image, the UAV flight control system comprising:

[0163] The yaw state judgment module 1 is used to obtain the relative position of the UAV in the flight area based on the target image and the position solution algorithm, and to judge the yaw state of the UAV by the position comparison method;

[0164] The flight state control module 2 is used to determine the flight yaw angle of the UAV by analyzing the target image using recognition processing technology based on the yaw state judgment, and to generate a yaw control amount according to the flight yaw angle to control the flight state of the UAV;

[0165] The landing trajectory planning module 3 is used to collect the real-time target image of the UAV after the flight state is adjusted, analyze the flight altitude of the UAV according to the real-time target image, and plan the landing trajectory of the UAV based on the flight altitude;

[0166] The flight landing control module 4 is used to generate an autonomous landing control instruction according to the landing trajectory of the UAV and the preset flight requirements, and send the autonomous landing control instruction to the controller to complete the autonomous landing flight control of the UAV.

[0167] In summary, with the help of the above technical solutions of the present invention, the present invention can judge the yaw state of the UAV in real time based on the position solution algorithm and the position comparison method, analyze the flight yaw angle according to the yaw state, and generate the yaw control amount accordingly, which is helpful for the UAV to quickly adjust the flight attitude, optimize the flight path, reduce unnecessary flight time and energy consumption, and improve flight efficiency. At the same time, by real-time acquisition of target images and analysis of flight altitude, the landing trajectory of the UAV can be accurately planned, and autonomous landing control instructions can be generated according to the landing trajectory and preset flight requirements, thereby realizing autonomous landing flight control of the UAV without manual intervention, thereby improving the autonomous operation ability of the UAV. The invention converts pixel values ​​into linear color system values ​​and utilizes scientific linear system calculation formulas to make data processing more standardized and systematic, providing a reliable basis for subsequent judgment and calculation. At the same time, the type of pixel points is judged based on the linear color system value, and the accuracy of flight altitude calculation is improved based on sophisticated classification processing. The flight altitude and dynamic window are used to characterize the landing trajectory space of the UAV, and a trajectory probability distribution model is constructed in combination with the Bayesian network to generate the optimal landing trajectory of the UAV. Through real-time image acquisition, precise data processing and intelligent trajectory planning, the landing safety of the UAV is significantly improved.

[0168] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0169] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A UAV flight control method based on target image, characterized in that: The UAV flight control method comprises the following steps: S1. Obtain the relative position of the UAV in the flight area based on the target image and position calculation algorithm, and determine the yaw state of the UAV by position comparison method; S2, based on the yaw state judgment, the target image is analyzed using recognition processing technology to determine the flight yaw angle of the UAV, and the yaw control amount is generated according to the flight yaw angle to control the flight state of the UAV; S3, collecting the real-time target image of the UAV after the flight state is adjusted, analyzing the flight altitude of the UAV according to the real-time target image, and planning the landing trajectory of the UAV based on the flight altitude; S4. Generate an autonomous landing control command according to the landing trajectory of the UAV and the preset flight requirements, and send the autonomous landing control command to the controller to complete the autonomous landing flight control of the UAV; The collecting of the real-time target image of the UAV after the flight state is adjusted, analyzing the flight altitude of the UAV according to the real-time target image, and planning the landing trajectory of the UAV based on the flight altitude includes: S31, using a camera to capture a real-time target image of the drone after the flight state is adjusted, and selecting a sample area calculated by the flight altitude from the real-time target image; S32, obtaining the pixel value of each pixel point in the flight height calculation sample area, and converting the pixel value into a linear color system value based on a linear system calculation formula; S33, judging the type of the pixel point based on the linear color system value, and establishing a UAV flight altitude algorithm according to the judgment result to obtain the UAV flight altitude; S34. Characterize the UAV landing trajectory space based on the flight altitude and the dynamic window, and construct a target probability distribution model according to the landing trajectory space to generate the optimal landing trajectory of the UAV.

2. The method for controlling a UAV flight based on a target image according to claim 1, characterized in that: The method of obtaining the relative position of the UAV in the flight area based on the target image and the position calculation algorithm, and determining the yaw state of the UAV by the position comparison method includes the following steps: S11, using a camera to capture a flying target image of the UAV within a preset time period, and converting the target image into a color model, and the color model segments the target image according to a component threshold; S12, after the segmentation is completed, a binary image of the target image is obtained, and the binary image is subjected to corrosion and expansion processing respectively by using morphological processing technology to determine the position of the flight point in the binary image; S13, based on the position calculation algorithm, the flight point position is converted into a three-dimensional space coordinate, and the horizontal distance between the flight point position and the ground point is determined to obtain the relative position of the UAV and the ground; S14. Restore the flight trajectory of the UAV according to the relative position, determine the overlap between the relative position flight trajectory and the reference position flight trajectory, and analyze the yaw state of the UAV based on the overlap result.

3. The method for controlling a UAV flight based on a target image according to claim 2, characterized in that: The method of restoring the flight trajectory of the UAV according to the relative position, determining the overlap between the flight trajectory of the relative position and the flight trajectory of the set position, and analyzing the yaw state of the UAV based on the overlap result includes the following steps: S141, inputting the relative position values ​​of the UAV and the ground into the visualization analysis software, connecting the relative position values ​​in the order of timestamps, and restoring the flight trajectory of the UAV in the flight area; S142, using a matching positioning algorithm to determine the overlap distance between the relative position flight trajectory and the set position flight trajectory at the same flight position; S143, setting the trajectory point overlap distance as the estimated yaw angle of the UAV, and adding all the estimated yaw angle values ​​to obtain the yaw deviation state of the UAV; The calculation formula of the trajectory point coincidence distance is: ; In the formula, dis represents the overlap distance of trajectory points, p i Indicated in i Set the flight trajectory point under each flight position. q i Indicated in i The relative position flight trajectory points under the flight position, i Indicates the number of flight position points, tw max Indicates the preset maximum threshold of the trajectory point overlap distance.

4. The method for controlling a UAV flight based on a target image according to claim 1, characterized in that: The method of determining the type of pixel points based on the linear color system value and establishing a UAV flight altitude algorithm to obtain the UAV flight altitude according to the determination result includes the following steps: S331, determining the type of the pixel point based on the linear color system value and the preset flight height threshold range, wherein the type includes a building area, a ground area, and a vegetation area; S332, obtaining the types of all pixel points, and determining the proportion of each type, dividing the sample area in the east-west, south-north directions using the longitude and latitude grid division technology and the size of the sample area, and performing a positioning operation on the pixel points according to the division results to obtain positioning points; S333, setting a ruler position point for the pixel point at the center position of the sample area, and deriving a horizontal distance of the pixel point on the center straight line of the sample area according to the ruler position point and the positioning point; S334. Determine the imaging height corresponding to the pixel points based on the horizontal distance, and generate an initial UAV flight height algorithm according to the imaging height and the horizontal distance.

5. The method for controlling a UAV flight based on a target image according to claim 4, characterized in that: The method of characterizing the landing trajectory space of the UAV based on the flight altitude and the dynamic window, and constructing a target probability distribution model according to the landing trajectory space to generate the optimal landing trajectory of the UAV includes the following steps: S341. Based on the descriptive coordinate system and the flight altitude of the UAV, the displacement of the UAV at each moment is decomposed into s Axis and t Axis in both directions, s The axis represents the direction of the drone parallel to the ground. t The axis represents the direction of the drone perpendicular to the ground; S342, based on s Axis direction, t The three sets of parameters, namely, the axis direction and the flight speed range, generate a dynamic window, and characterize the feasible landing trajectory space of the UAV according to the dynamic window; S343, constructing a target probability distribution based on the landing feasible trajectory space, and generating a set of candidate trajectories that meet the requirements for the landing of the UAV according to the target probability distribution and the stratified sampling strategy; S344. A safety model based on the uncertainty of the UAV's flight speed selects an optimal trajectory from a set of candidate trajectories as a basis for generating an optimal landing trajectory for the UAV.

6. The method for controlling a UAV flight based on a target image according to claim 5, characterized in that: The method of constructing a target probability distribution based on the landing feasible trajectory space and generating a set of candidate trajectories that meet the requirements for the landing of the UAV according to the target probability distribution and the stratified sampling strategy includes the following steps: S3431, based on the feasible landing trajectory space s A Gaussian distribution centered on the highest point of the UAV’s flight altitude is established in the axial direction, and the cumulative probability within the dynamic window range is determined; S3432, weighting the cumulative probability based on the maximum flight speed allowed by the dynamic window range, and normalizing the weighted cumulative probability to obtain the window selection probability; S3433, constructing a probability distribution of the expected flight speed in the dynamic window according to the window selection probability, analyzing the probability distribution of the acceleration of the UAV based on the probability distribution result, and randomly sampling a set of height values ​​from the UAV flight altitude as the horizontal coordinate of the target point; S3434. Based on the stratified sampling strategy of speed first and path later, the sampling order is set to the expected speed, acceleration and the horizontal coordinate of the target point to generate a set of candidate trajectories that meet the landing requirements of the UAV.

7. The method for controlling a UAV flight based on a target image according to claim 6, characterized in that: The stratified sampling strategy based on speed first and path second sets the sampling order to generate a set of candidate trajectories that meet the landing requirements of the drone in the order of expected speed, acceleration and target point abscissa, including the following steps: S34341, taking the acceleration probability distribution, the expected flight speed probability distribution, the target point horizontal coordinate, the UAV flight altitude and the window selection probability as input, and the candidate trajectory set as output; S34342, sampling the window selection probability, the probability distribution of the expected flight speed, the acceleration probability and the target point horizontal coordinate to generate a target window, a target flight speed, a target acceleration and a horizontal coordinate respectively; S34343. Generate a speed curve according to the target flight speed and the target acceleration, and project the speed curve into time space to solve the quadratic programming problem to generate a smooth speed curve; S34344. Calculate the target point's ordinate based on the target window and the abscissa, and generate a set of candidate trajectories that meet the requirements for UAV landing by combining the smooth speed curve and the Bezier curve.

8. The method for controlling a UAV flight based on a target image according to claim 1, characterized in that: The step of generating an autonomous landing control instruction according to the landing trajectory of the UAV and the preset flight requirements, and sending the autonomous landing control instruction to the controller to complete the autonomous landing flight control of the UAV includes the following steps: S41, obtaining the environmental conditions when the drone is to land based on the preset flight requirements, and analyzing the thrust and yaw angle required by the drone during the landing process according to the environmental conditions and the landing trajectory of the drone; S42, generating an autonomous landing control instruction using a model predictive control algorithm based on the thrust and yaw angle analysis results, and sending the autonomous landing control instruction to a controller; S43, the controller receives and executes the autonomous landing control command to adjust the flight attitude of the UAV, implements the autonomous landing flight control of the UAV, and monitors the actual flight status of the UAV during the landing process; S44, determining the deviation between the actual flight status result and the landing trajectory of the UAV, and adjusting the control command based on the deviation to complete the autonomous landing flight control of the UAV.

9. A target image-based UAV flight control system, used to implement the target image-based UAV flight control method according to any one of claims 1 to 8, characterized in that: The UAV flight control system includes: The yaw state judgment module is used to obtain the relative position of the UAV in the flight area based on the target image and the position solution algorithm, and to judge the yaw state of the UAV by the position comparison method; The flight state control module is used to determine the flight yaw angle of the UAV by analyzing the target image using recognition and processing technology based on the yaw state judgment, and to generate a yaw control amount according to the flight yaw angle to control the flight state of the UAV; The landing trajectory planning module is used to collect the real-time target image of the UAV after the flight status is adjusted, analyze the flight altitude of the UAV according to the real-time target image, and plan the landing trajectory of the UAV based on the flight altitude; The flight landing control module is used to generate autonomous landing control instructions according to the UAV landing trajectory and preset flight requirements, and send the autonomous landing control instructions to the controller to complete the autonomous landing flight control of the UAV.

Images