UAV landing control method and system based on visual recognition

Abstract

The present invention relates to the technical field of unmanned aerial vehicle control. The present invention discloses a method and system for controlling the landing of an unmanned aerial vehicle based on visual recognition, comprising: screening out a target forced landing ground area, controlling the unmanned aerial vehicle to travel to and hover over the target forced landing ground area, and collecting the hovering position and a first overhead image of the unmanned aerial vehicle; obtaining the hovering height, and obtaining the vertical distance of all obstacles, and obtaining the height data according to the vertical distance and the hovering height; obtaining a preset second safe forced landing speed, inputting the height data and the second safe forced landing speed into a pre-built speed prediction model to obtain the first safe forced landing speed of the unmanned aerial vehicle; judging whether the unmanned aerial vehicle has left the target forced landing ground area according to the first overhead image and the second overhead image; controlling the unmanned aerial vehicle to land smoothly from an updated position to the ground according to the second safe forced landing speed; or controlling the unmanned aerial vehicle to land smoothly from an adjusted position to the ground; the present invention can achieve rapid and safe landing.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and more specifically, to a method and system for controlling landing of an unmanned aerial vehicle based on visual recognition. Background Art

[0002] With the rapid development of modern drone technology, the application areas of drones have expanded to various industries including urban management, disaster relief, agricultural monitoring and industrial inspection. However, no matter what the application scenario is, the safe and stable landing of drones is always a crucial technical challenge. Especially in complex environments such as mountains, densely populated urban areas, and indoor environments, how to ensure that drones can land accurately and safely has become an urgent problem to be solved.

[0003] At present, there is a lack of UAV landing control solutions for complex scenes. Although there are some relevant technical documents, for example, the Chinese patent with authorization announcement number CN105353765B discloses a method and device for controlling the landing of a UAV. This method uses the judgment of image clarity in a focusing mode to measure the distance from the landing plane, and adjusts the speed according to the height from the landing plane, thereby achieving the landing of the UAV. However, research and practical application of the above method and the prior art have found that the above method and the prior art have at least the following defects:

[0004] (1) The impact on the environment and equipment lacks sufficient anti-interference ability. When the drone is affected by environmental factors (such as light changes, weather conditions) or equipment failures (such as camera blur, sensor failure), the image may become unclear, which will directly affect the landing effect and speed, thereby increasing the risk of landing failure;

[0005] (2) The control process is relatively complicated. If the drone needs to make an emergency landing (e.g., due to insufficient battery) in a complex scene (such as a mountainous area or forest), such multiple adjustments may cause the drone to consume unnecessary power, thereby further increasing the risk of a crash. In addition, frequent speed adjustments also make the landing process more complicated, making it difficult to complete a safe landing in a short time.

[0006] (3) It is impossible to accurately determine the closest distance between the UAV and the obstacle, nor is it possible to obtain the UAV's initial descent speed based on this distance while ensuring flight stability. This makes it difficult to increase the landing speed in complex scenarios. In addition, it is impossible to adjust the orientation of the UAV, which makes it impossible for the UAV to avoid obstacles in real time during landing, increasing the risk of collision. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a UAV landing control method and system based on visual recognition.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for controlling a drone's landing based on visual recognition, the method comprising:

[0010] S110: Acquire a first ground image set, filter out a target forced landing ground area according to the first ground image set, control the UAV to fly to and hover over the target forced landing ground area, and collect a hovering position and a first overhead image of the UAV during the hovering; the first ground image set includes K first ground images, where K is an integer greater than zero;

[0011] S120: Obtaining a hovering height of the UAV above the target forced landing ground area, and obtaining a vertical distance between the UAV and all obstacles around the target forced landing ground area, and obtaining altitude data according to the vertical distance and the hovering height, wherein the altitude data includes the target forced landing altitude, a first altitude difference, and a second altitude difference;

[0012] S130: Obtain a preset second safe forced landing speed, input the altitude data and the second safe forced landing speed into a pre-built speed prediction model to obtain a first safe forced landing speed of the UAV, and control the UAV to reach a target forced landing height at the first safe forced landing speed; and collect an updated position and a second overhead image of the UAV when reaching the target forced landing height;

[0013] S140: judging whether the UAV has left the target forced landing ground area according to the first bird's-eye view image and the second bird's-eye view image; if not, jumping to step S150; if so, adjusting the updated position based on the hovering position to obtain an adjusted position;

[0014] S150: Control the UAV to land smoothly from the updated position to the ground according to the second safe forced landing speed; or control the UAV to land smoothly from the adjusted position to the ground.

[0015] Furthermore, before selecting the target forced landing ground area according to the first ground image set, the method includes:

[0016] S111: extracting the i-th first ground image in the first ground image set, where i is an integer greater than zero;

[0017] S112: Calculate the image quality coefficient of the i-th first ground image, and compare the image quality coefficient with a preset image quality coefficient threshold. If the image quality coefficient is greater than or equal to the preset image quality coefficient threshold, take the i-th first ground image as the target first ground image; if the image quality coefficient is less than the preset image quality coefficient threshold, set i=i+1, and return to step a1;

[0018] The calculation logic of the image quality coefficient is as follows:

[0019] Obtaining the grayscale value and image size of the i-th first ground image;

[0020] Inputting the grayscale value and the image size into a pre-built mathematical calculation model for calculating the quality coefficient to obtain the quality coefficient of the i-th first ground image;

[0021] The mathematical calculation model is expressed as follows:

[0022] ;

[0023] Where: is the quality coefficient; and is the image size; is the edge strength of each pixel (x, y); V is the overall variance of the image; log is the logarithmic function;

[0024] S113: Repeat the above S111 to S112 until i=K, then end the loop and obtain the target first ground image.

[0025] Furthermore, the step of screening out the target forced landing ground area according to the first ground image set includes:

[0026] Using the K-means clustering algorithm to distinguish pixels of the target first ground image, and taking the area formed by the pixel clustering as the clustering area;

[0027] Extract the pixel area of ​​each cluster region, and take the corresponding cluster regions whose pixel areas are greater than or equal to a preset pixel area threshold as candidate regions, and obtain R candidate regions, where R is an integer greater than zero;

[0028] Input the candidate region into a pre-trained depth estimation model to obtain a depth map, where the value of each pixel in the depth map represents the depth relative to the camera;

[0029] Select a pixel point P , and the adjacent pixel point P and P ;

[0030] Calculate pixel point P With adjacent pixel point P and P , the depth difference in the horizontal and vertical directions, which is calculated as follows:

[0031] ;

[0032] Where: For a pixel P The depth of is a neighboring pixel point P The depth of is a neighboring pixel point P The depth of is the horizontal depth difference, is the depth difference in the vertical direction;

[0033] Get pixel point P With adjacent pixel point P and P The actual distance between the pixels is obtained by substituting the actual distance, the depth difference in the horizontal direction, and the depth difference in the vertical direction into the preset slope calculation model to obtain the pixel point P The slope of

[0034] The expression of the preset slope calculation model is as follows:

[0035] ;

[0036] Where: is the slope, and is the actual distance between adjacent pixels, is the inverse tangent function;

[0037] For each pixel in the depth map, repeat the above slope calculation process to obtain the slope of each pixel in the depth map;

[0038] The slopes of all pixels are averaged to obtain the average slope value, and the average slope value of the candidate area corresponding to each depth map is counted;

[0039] Sort all average slope values ​​from large to small, and mark the average slope value corresponding to the first one in the sort as the target forced landing ground area.

[0040] Further, the first height difference is the height distance between the target forced landing height and the hovering height, and the second height difference is the height distance between the target forced landing height and the ground;

[0041] The obtaining of height data according to the vertical distance and the hovering height includes:

[0042] Extract the vertical distances of the drone from all obstacles around the target forced landing ground area to obtain U vertical distances, where U is an integer greater than zero;

[0043] Sort the U vertical distances from small to large, and make the vertical distance corresponding to the first one in the sorting the closest vertical distance;

[0044] Substitute the nearest vertical distance and hovering height into the preset height data calculation model to obtain the target forced landing height; the specific calculation formula is as follows: ; Where: is the hovering height, is the closest vertical distance, To achieve the target altitude, is a safety factor greater than zero.

[0045] Furthermore, the training logic of the pre-built speed prediction model is as follows:

[0046] Acquire historical UAV speed prediction training data, and divide the historical UAV speed prediction training data into a speed prediction training set and a speed prediction test set, wherein the historical UAV speed prediction training data includes speed prediction features and their corresponding first safe forced landing speeds;

[0047] Wherein, the speed prediction features include altitude data and a second safe forced landing speed;

[0048] The second safe emergency landing speed is obtained by optimizing based on a preconfigured genetic optimization algorithm, and the optimization logic of the genetic optimization algorithm is as follows:

[0049] S311: Initialize the population: randomly generate an original population, wherein the original population contains Y chromosomes, each chromosome represents the initial forced landing speed of a UAV, and Y is an integer greater than zero;

[0050] S312: Fitness evaluation: Under each chromosome, the landing time and vibration frequency of the drone are obtained, and the landing time and vibration frequency are input into the pre-built fitness function to calculate the fitness of each chromosome;

[0051] S313: Selection: Use the roulette wheel method to select two chromosomes with high fitness in the original population as the father and mother;

[0052] S314: Crossover: Perform a crossover operation on the father and mother to generate new chromosomes;

[0053] S315: mutation: performing mutation operation on the new chromosome to obtain Y new chromosomes, combining the Y new chromosomes into a new population, replacing the original population with the new population, and returning to step S312;

[0054] S316: Repeat the above steps S312 to S315 until the fitness of the chromosome in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, and output the initial forced landing speed represented by the corresponding chromosome as the first safe forced landing speed.

[0055] Furthermore, the logic for obtaining the vibration frequency is as follows:

[0056] Get the initial forced landing speed of the drone, and the wind force value of the drone's environment;

[0057] The wind force value and the initial forced landing speed are input into the pre-built vibration frequency prediction model to obtain the vibration frequency.

[0058] Furthermore, the training logic of the pre-built vibration frequency prediction model is as follows:

[0059] Acquire historical drone vibration frequency training data, and divide the historical drone vibration frequency training data into a vibration frequency training set and a vibration frequency test set; the vibration frequency training includes vibration frequency prediction features and their corresponding vibration frequencies;

[0060] Wherein, the vibration frequency prediction characteristics include wind force value and initial forced landing speed;

[0061] Constructing a first regression network, taking the vibration frequency prediction features in the vibration frequency training set as input of the first regression network; and taking the vibration frequency as output of the first regression network; training the first regression network to obtain an initial vibration frequency prediction network;

[0062] The initial vibration frequency prediction network is model-verified using the vibration frequency test set, and an initial vibration frequency prediction network that is less than or equal to a first test error threshold is output as a pre-constructed vibration frequency prediction model.

[0063] Furthermore, the calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is the landing time, and vibration frequency.

[0064] A UAV landing control system based on visual recognition, comprising:

[0065] a data collection module, configured to obtain a first ground image set, filter out a target forced landing ground area according to the first ground image set, control the UAV to fly to and hover over the target forced landing ground area, and collect a hovering position and a first overhead image of the UAV during the hovering; the first ground image set includes K first ground images, where K is an integer greater than zero;

[0066] The height detection module is used to obtain the hovering height of the UAV above the target forced landing ground area, and obtain the vertical distance between the UAV and all obstacles around the target forced landing ground area, and obtain height data according to the vertical distance and the hovering height, wherein the height data includes the target forced landing height, the first height difference and the second height difference;

[0067] The speed analysis module is used to obtain a preset second safe forced landing speed, input the altitude data and the second safe forced landing speed into a pre-built speed prediction model to obtain a first safe forced landing speed of the UAV, and control the UAV to reach a target forced landing height at the first safe forced landing speed; and collect an updated position and a second overhead view image of the UAV when reaching the target forced landing height;

[0068] The orientation judgment module is used to judge whether the UAV has left the target forced landing ground area according to the first bird's-eye view image and the second bird's-eye view image, and if not, trigger the landing control module; if not, adjust the updated position based on the hovering position to obtain the adjusted position;

[0069] The landing control module is used to control the UAV to land smoothly from the updated position to the ground according to the second safe forced landing speed; or control the UAV to land smoothly from the adjusted position to the ground.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] The present application discloses a method and system for controlling a drone landing based on visual recognition, including: screening out a target forced landing ground area, controlling the drone to travel to and hover over the target forced landing ground area, and collecting the drone's hovering position and a first overhead image; obtaining the hovering height, and obtaining the vertical distance of all obstacles, and obtaining the height data according to the vertical distance and the hovering height; obtaining a preset second safe forced landing speed, and inputting the height data and the second safe forced landing speed into a pre-built speed prediction model to obtain the drone's first safe forced landing speed; judging whether the drone has left the target forced landing ground area according to the first overhead image and the second overhead image; controlling the drone to land smoothly from an updated position to the ground according to the second safe forced landing speed; or controlling the drone to land smoothly from an adjusted position to the ground; and After comparing the technical features with those of the prior art, it is found that the present invention uses the target forced landing altitude as a turning condition for the secondary control of the UAV, which not only can reduce the UAV's unnecessary power consumption, but also makes it suitable for complex scenes, has strong anti-interference ability, and is beneficial to improving the UAV's landing efficiency and shortening the UAV's landing; in addition, by determining the closest distance between the UAV and the obstacle, and obtaining the UAV's initial descent speed while ensuring flight stability; this is beneficial to improving the UAV's landing speed in complex scenes; in addition, by dynamically adjusting the UAV's orientation, the UAV can be kept in the target forced landing ground area during the landing process and avoid obstacles, thereby reducing the risk of UAV collision and greatly improving the UAV's landing safety in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1A flowchart of the UAV landing control method based on visual recognition provided by the present invention;

[0073] Figure 2 A module diagram of the UAV landing control system based on visual recognition provided by the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] Example 1

[0076] See also Figure 1 As shown, this embodiment discloses a UAV landing control method based on visual recognition, and the method includes:

[0077] S110: Acquire a first ground image set, filter out a target forced landing ground area according to the first ground image set, control the UAV to fly to and hover over the target forced landing ground area, and collect a hovering position and a first overhead image of the UAV during the hovering; the first ground image set includes K first ground images, where K is an integer greater than zero;

[0078] It should be noted that: the first ground image is obtained by photographing and collecting the ground below the drone with a high-definition camera pre-set on the drone. The high-definition camera will only photograph and collect from the ground angle when the drone triggers an emergency landing condition. The emergency landing condition includes but is not limited to the drone power being lower than a preset power threshold (such as 20%) or the air humidity being greater than a humidity threshold, etc.;

[0079] In implementation, before selecting the target forced landing ground area according to the first ground image set, the following steps are included:

[0080] S111: extracting the i-th first ground image in the first ground image set, where i is an integer greater than zero;

[0081] S112: Calculate the image quality coefficient of the i-th first ground image, and compare the image quality coefficient with a preset image quality coefficient threshold. If the image quality coefficient is greater than or equal to the preset image quality coefficient threshold, take the i-th first ground image as the target first ground image; if the image quality coefficient is less than the preset image quality coefficient threshold, set i=i+1, and return to step a1;

[0082] The calculation logic of the image quality coefficient is as follows:

[0083] Obtaining the grayscale value and image size of the i-th first ground image;

[0084] Inputting the grayscale value and the image size into a pre-built mathematical calculation model for calculating the quality coefficient to obtain the quality coefficient of the i-th first ground image;

[0085] The mathematical calculation model is expressed as follows:

[0086] ;

[0087] Where: is the quality coefficient; and is the image size; is the edge strength of each pixel (x, y); V is the overall variance of the image; log is the logarithmic function;

[0088] It should be noted that the edge strength is calculated by applying the Sobel operator: ; Where: and is the Sobel response of the image in the x and y directions; where, ; Where: ; is the grayscale value of the pixel at coordinate (x, y) in the image;

[0089] S113: repeat the above S111 to S112 until i=K, then end the loop and obtain the first ground image of the target;

[0090] In implementation, screening out the target forced landing ground area according to the first ground image set includes:

[0091] Using the K-means clustering algorithm to distinguish pixels of the target first ground image, and taking the area formed by the pixel clustering as the clustering area;

[0092] Extract the pixel area of ​​each cluster region, and take the corresponding cluster regions whose pixel areas are greater than or equal to a preset pixel area threshold as candidate regions, and obtain R candidate regions, where R is an integer greater than zero;

[0093] Input the candidate region into a pre-trained depth estimation model to obtain a depth map, where the value of each pixel in the depth map represents the depth relative to the camera;

[0094] It should be noted that the pre-trained depth estimation model is an existing model, including but not limited to MiDaS and Monodepth2, which can estimate the depth of a scene from a single image;

[0095] Select a pixel point P , and the adjacent pixel point P and P ;

[0096] Calculate pixel point P With adjacent pixel point P and P , the depth difference in the horizontal and vertical directions, which is calculated as follows:

[0097] ;

[0098] Where: For a pixel P The depth of is a neighboring pixel point P The depth of is a neighboring pixel point P The depth of is the horizontal depth difference, is the depth difference in the vertical direction;

[0099] Get pixel point P With adjacent pixel point P and P The actual distance between the pixels is obtained by substituting the actual distance, the depth difference in the horizontal direction, and the depth difference in the vertical direction into the preset slope calculation model to obtain the pixel point P The slope of

[0100] The expression of the preset slope calculation model is as follows:

[0101] ;

[0102] Where: is the slope, and is the actual distance between adjacent pixels, is the inverse tangent function;

[0103] It should be noted that when the slope Slope is obtained, it can also be converted into degrees for easy understanding. The specific conversion formula is as follows: ; Where: is the degree, It is a mathematical constant, approximately equal to 3.14159;

[0104] For each pixel in the depth map, repeat the above slope calculation process to obtain the slope of each pixel in the depth map;

[0105] The slopes of all pixels are averaged to obtain the average slope value, and the average slope value of the candidate area corresponding to each depth map is counted;

[0106] Sort all average slope values ​​from large to small, and mark the average slope value corresponding to the first one in the sort as the target forced landing ground area.

[0107] S120: Obtaining a hovering height of the UAV above the target forced landing ground area, and obtaining a vertical distance between the UAV and all obstacles around the target forced landing ground area, and obtaining altitude data according to the vertical distance and the hovering height, wherein the altitude data includes the target forced landing altitude, a first altitude difference, and a second altitude difference;

[0108] The first height difference is the height distance between the target forced landing height and the hovering height, and the second height difference is the height distance between the target forced landing height and the ground;

[0109] It should be noted that the hovering height of the UAV above the target forced landing ground area can be obtained through positioning systems such as GPS or Beidou, or can be directly measured based on laser radar;

[0110] It should also be noted that the drone is also equipped with a laser radar or other ranging equipment, which can be used to measure the vertical distance between the drone and all obstacles around the target forced landing ground area. It should be further noted that the measurement range of the laser radar or other ranging equipment can be manually set in advance. In other words, the volume of vertical distance measurement data is determined according to the measurement range of the laser radar or other ranging equipment, that is, all obstacles around the measurement range can be identified as all obstacles around the target forced landing ground area.

[0111] In a specific implementation, the acquiring of altitude data according to the vertical distance and the hovering altitude includes:

[0112] Extract the vertical distances of the drone from all obstacles around the target forced landing ground area to obtain U vertical distances, where U is an integer greater than zero;

[0113] Sort the U vertical distances from small to large, and make the vertical distance corresponding to the first one in the sorting the closest vertical distance;

[0114] Substitute the nearest vertical distance and hovering height into the preset height data calculation model to obtain the target forced landing height; the specific calculation formula is as follows: ; Where: is the hovering height, is the closest vertical distance, To achieve the target altitude, is a safety factor greater than zero;

[0115] It can be understood that: the height of the highest obstacle can be obtained by subtracting the closest vertical distance from the hovering height. When the height of the highest obstacle is obtained, the first height difference can be obtained by calculating the height distance between the target forced landing height and the hovering height, and the distance from the hovering height to the target forced landing height is used as the first descent stage of the drone. At this stage, the drone can execute a rapid descent. However, it is worth noting that if the The result is used as the target forced landing height. Due to the influence of the drone's own volume and other factors, there is a risk of collision in the first descent stage. In order to ensure the descent speed and landing safety in the first stage, a safety factor is introduced. , the probability of this collision risk occurring in the first descent stage can be avoided; thus, it is beneficial to ensure the descent safety of the UAV in the first descent stage; the safety factor Determined by technical personnel based on historical experimental data.

[0116] In a specific embodiment, the obtaining of altitude data according to the vertical distance and the hovering altitude further includes: obtaining the altitude distance between the target forced landing altitude and the ground, and using the altitude distance between the target forced landing altitude and the ground as the second altitude difference;

[0117] It is understandable that when the drone reaches the target forced landing altitude, it means that the drone has entered the second descent stage of steady descent after executing the first descent stage of rapid descent. Since the drone is close to the ground in the second descent stage, if the rapid descent strategy is still adopted, it is easy to cause the drone to land hard, and thus the drone cannot be guaranteed to land smoothly.

[0118] S130: Obtain a preset second safe forced landing speed, input the altitude data and the second safe forced landing speed into a pre-built speed prediction model to obtain a first safe forced landing speed of the UAV, and control the UAV to reach a target forced landing height at the first safe forced landing speed; and collect an updated position and a second overhead image of the UAV when reaching the target forced landing height;

[0119] It should be understood that: since the drone is close to the ground in the second descent stage, in order to ensure the safety of the drone's descent, the preset second safe forced landing speed is used to control the drone's landing; and the preset second safe forced landing speed is artificially set by technicians based on historical experiments, and the second safe forced landing speed is less than the first safe forced landing speed;

[0120] In implementation, the training logic of the pre-built speed prediction model is as follows:

[0121] Acquire historical UAV speed prediction training data, and divide the historical UAV speed prediction training data into a speed prediction training set and a speed prediction test set, wherein the historical UAV speed prediction training data includes speed prediction features and their corresponding first safe forced landing speeds;

[0122] Wherein, the speed prediction features include altitude data and a second safe forced landing speed;

[0123] The second safe emergency landing speed is obtained by optimizing based on a preconfigured genetic optimization algorithm, and the optimization logic of the genetic optimization algorithm is as follows:

[0124] S311: Initialize the population: randomly generate an original population, wherein the original population contains Y chromosomes, each chromosome represents the initial forced landing speed of a UAV, and Y is an integer greater than zero;

[0125] S312: Fitness evaluation: Under each chromosome (i.e., each forced landing speed in the original population), the landing time and vibration frequency of the drone are obtained, and the landing time and vibration frequency are input into the pre-constructed fitness function to calculate the fitness of each chromosome;

[0126] The logic for obtaining the vibration frequency is as follows:

[0127] Get the initial forced landing speed of the drone, and the wind force value of the drone's environment;

[0128] The wind force value and the initial forced landing speed are input into a pre-built vibration frequency prediction model to obtain the vibration frequency;

[0129] The training logic of the pre-built vibration frequency prediction model is as follows:

[0130] Acquire historical drone vibration frequency training data, and divide the historical drone vibration frequency training data into a vibration frequency training set and a vibration frequency test set; the vibration frequency training includes vibration frequency prediction features and their corresponding vibration frequencies;

[0131] Wherein, the vibration frequency prediction characteristics include wind force value and initial forced landing speed;

[0132] It should be noted that: the wind force value in the historical drone vibration frequency training data is collected in real time by a wind sensor, the initial forced landing speed is randomly generated, and the vibration frequency is experimentally recorded by technicians in advance;

[0133] Constructing a first regression network, taking the vibration frequency prediction features in the vibration frequency training set as input of the first regression network; and taking the vibration frequency as output of the first regression network; training the first regression network to obtain an initial vibration frequency prediction network;

[0134] Using the vibration frequency test set to perform model verification on the initial vibration frequency prediction network, and outputting an initial vibration frequency prediction network that is less than or equal to a first test error threshold as a pre-built vibration frequency prediction model;

[0135] The calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is the landing time, and vibration frequency;

[0136] S313: Selection: Use the roulette wheel method to select two chromosomes with high fitness in the original population as the father and mother;

[0137] S314: Crossover: Perform a crossover operation on the father and mother to generate new chromosomes;

[0138] It should be noted that: the crossover operation on the paternal parent and the maternal parent is implemented based on a crossover operation, and the crossover operation includes but is not limited to one of a single-point crossover, a uniform crossover or a sequential crossover, etc.;

[0139] S315: mutation: performing mutation operation on the new chromosome to obtain Y new chromosomes, combining the Y new chromosomes into a new population, replacing the original population with the new population, and returning to step S312;

[0140] In the genetic algorithm, the mutation operation is to introduce gene diversity and prevent the algorithm from falling into the local optimum; the mutation operation on the new chromosome is implemented by uniform mutation or Gaussian mutation;

[0141] S316: Repeat the above steps S312 to S315 until the fitness of the chromosome in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, and output the initial forced landing speed represented by the corresponding chromosome as the first safe forced landing speed;

[0142] For example: Assume that the maximum number of iterations is 100, and record the chromosome with the highest fitness in the current population and its fitness value after each iteration; if it is found that the fitness value has not changed significantly in a certain generation, it is considered that the convergence condition has been met, the iteration is stopped, and the initial forced landing speed represented by the corresponding chromosome is output;

[0143] A second regression network is constructed, the speed prediction feature in the speed prediction feature is used as an input of the second regression network, and the first safe emergency landing speed in the speed prediction feature is used as an output of the second regression network, and the input of the second regression network is trained to obtain an initial speed prediction network;

[0144] Using the speed prediction test set to perform model verification on the initial speed prediction network, outputting an initial speed prediction network that is less than or equal to a preset second test error threshold as a pre-built speed prediction model;

[0145] It should be noted that: like the above-mentioned first regression network, the second regression network is specifically one of the regression algorithms such as convolutional neural network, recurrent neural network, decision tree, random forest, support vector machine or polynomial regression.

[0146] S140: judging whether the UAV has left the target forced landing ground area according to the first bird's-eye view image and the second bird's-eye view image; if not, jumping to step S150; if so, adjusting the updated position based on the hovering position to obtain an adjusted position;

[0147] In implementation, the determination of whether the drone has left the target forced landing ground area includes:

[0148] Extracting a first bird's-eye view image and a second bird's-eye view image, and dividing the first bird's-eye view image and the second bird's-eye view image into a plurality of equally divided regions according to the same rule;

[0149] Compare the pixels of the equally divided regions at the same position in the first overhead image and the second overhead image one by one, and record the similar equally divided regions in the first overhead image and the second overhead image;

[0150] It should be noted that: the first overhead image and the second overhead image are both divided into a plurality of equally divided regions in the same manner and in the same size; in addition, the similar equally divided regions are obtained by processing with a similarity algorithm (including but not limited to cosine similarity, etc.);

[0151] Count the number of similar equally divided areas and the proportion of similar equally divided areas in all equally divided areas; if the proportion of similar equally divided areas in all equally divided areas is greater than or equal to the set proportion threshold, it is determined that the drone has not left the target forced landing ground area; if the proportion of similar equally divided areas in all equally divided areas is less than the set proportion threshold, it is determined that the drone has left the target forced landing ground area;

[0152] It should be noted that the update position is adjusted based on the hovering position by uniformly controlling the Z-axis vertical axis (i.e., aligning the Z-axis coordinates). For example, assuming that the hovering position of the drone is (26, 95, 82) and the updated coordinates are (30, 82, 85), then after controlling the Z axis, (30, 82, 82) is obtained. At this time, the drone will not deviate from the target forced landing ground area.

[0153] S150: Control the UAV to land smoothly from the updated position to the ground according to the second safe forced landing speed; or control the UAV to land smoothly from the adjusted position to the ground;

[0154] It can be understood that: when the UAV has not left the target forced landing ground area, the UAV is deemed to be inside the airspace of the target forced landing ground area and at the target forced landing altitude, and the UAV is directly controlled to land smoothly from the updated position to the ground according to the second safe forced landing speed; and when the UAV leaves the target forced landing ground area, the UAV is deemed to be outside the airspace of the target forced landing ground area, and the position of the UAV needs to be adjusted to prevent it from colliding with obstacles during landing.

[0155] Example 2

[0156] See also Figure 2 As shown, this embodiment discloses a drone landing control system based on visual recognition, including:

[0157] The data collection module 210 is used to obtain a first ground image set, filter out a target forced landing ground area according to the first ground image set, control the UAV to fly to the target forced landing ground area and hover, and collect the hovering position and the first overhead image of the UAV during the hovering; the first ground image set includes K first ground images, where K is an integer greater than zero;

[0158] The height detection module 220 is used to obtain the hovering height of the UAV above the target forced landing ground area, and obtain the vertical distance between the UAV and all obstacles around the target forced landing ground area, and obtain height data according to the vertical distance and the hovering height, wherein the height data includes the target forced landing height, the first height difference and the second height difference;

[0159] The speed analysis module 230 is used to obtain a preset second safe forced landing speed, input the altitude data and the second safe forced landing speed into a pre-built speed prediction model to obtain a first safe forced landing speed of the UAV, and control the UAV to reach a target forced landing height at the first safe forced landing speed; and collect an updated position and a second overhead view image of the UAV when reaching the target forced landing height;

[0160] The orientation judgment module 240 is used to judge whether the UAV has left the target forced landing ground area according to the first bird's-eye view image and the second bird's-eye view image, and if not, trigger the landing control module 250; if not, adjust the updated position based on the hovering position to obtain the adjusted position;

[0161] The landing control module 250 is used to control the UAV to land smoothly from the updated position to the ground according to the second safe forced landing speed; or control the UAV to land smoothly from the adjusted position to the ground.

[0162] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters, weights and thresholds in the formula are set by technicians in this field according to actual conditions.

[0163] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired network or a wireless network. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD) or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0164] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0170] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A UAV landing control method based on visual recognition, characterized in that: The method comprises: S110: Acquire a first ground image set, filter out a target forced landing ground area according to the first ground image set, control the UAV to fly to and hover over the target forced landing ground area, and collect a hovering position and a first overhead image of the UAV during the hovering; the first ground image set includes K first ground images, where K is an integer greater than zero; S120: Obtaining a hovering height of the UAV above the target forced landing ground area, and obtaining a vertical distance between the UAV and all obstacles around the target forced landing ground area, and obtaining altitude data according to the vertical distance and the hovering height, wherein the altitude data includes the target forced landing altitude, a first altitude difference, and a second altitude difference; The first height difference is the height distance between the target forced landing height and the hovering height, and the second height difference is the height distance between the target forced landing height and the ground; The obtaining of height data according to the vertical distance and the hovering height includes: Extract the vertical distances of the drone from all obstacles around the target forced landing ground area to obtain U vertical distances, where U is an integer greater than zero; Sort the U vertical distances from small to large, and make the vertical distance corresponding to the first one in the sorting the closest vertical distance; Substitute the nearest vertical distance and hovering height into the preset height data calculation model to obtain the target forced landing height; the specific calculation formula is as follows: ; Where: is the hovering height, is the closest vertical distance, To achieve the target altitude, is a safety factor greater than zero; S130: Obtain a preset second safe forced landing speed, input the altitude data and the second safe forced landing speed into a pre-built speed prediction model to obtain a first safe forced landing speed of the UAV, and control the UAV to reach a target forced landing height at the first safe forced landing speed; and collect an updated position and a second overhead image of the UAV when reaching the target forced landing height; S140: judging whether the UAV has left the target forced landing ground area according to the first bird's-eye view image and the second bird's-eye view image; if not, jumping to step S150; if so, adjusting the updated position based on the hovering position to obtain an adjusted position; S150: Control the UAV to land smoothly from the updated position to the ground according to the second safe forced landing speed; or control the UAV to land smoothly from the adjusted position to the ground.

2. The method for controlling the landing of a UAV based on visual recognition according to claim 1, characterized in that: Before selecting the target forced landing ground area based on the first ground image set, the following steps are included: S111: extracting the i-th first ground image in the first ground image set, where i is an integer greater than zero; S112: Calculate the image quality coefficient of the i-th first ground image, and compare the image quality coefficient with a preset image quality coefficient threshold. If the image quality coefficient is greater than or equal to the preset image quality coefficient threshold, take the i-th first ground image as the target first ground image; if the image quality coefficient is less than the preset image quality coefficient threshold, set i=i+1, and return to step a1; The calculation logic of the image quality coefficient is as follows: Obtaining the grayscale value and image size of the i-th first ground image; Inputting the grayscale value and the image size into a pre-built mathematical calculation model for calculating the quality coefficient to obtain the quality coefficient of the i-th first ground image; The mathematical calculation model is expressed as follows: ； Where: is the quality coefficient; and is the image size; is the edge strength of each pixel (x, y); V is the overall variance of the image; log is the logarithmic function; S113: Repeat the above S111 to S112 until i=K, then end the loop and obtain the target first ground image.

3. The method for controlling the landing of a UAV based on visual recognition according to claim 2, characterized in that: The step of screening out the target forced landing ground area according to the first ground image set includes: The K-means clustering algorithm is used to distinguish pixels of the target first ground image, and the area formed by the pixel clustering is used as the clustering area; Extract the pixel area of ​​each cluster region, and take the corresponding cluster regions whose pixel areas are greater than or equal to a preset pixel area threshold as candidate regions, and obtain R candidate regions, where R is an integer greater than zero; Input the candidate region into a pre-trained depth estimation model to obtain a depth map, where the value of each pixel in the depth map represents the depth relative to the camera; Select a pixel point P , and the adjacent pixel point P and P ; Calculate pixel point P With adjacent pixel point P and P , the depth difference in the horizontal and vertical directions, which is calculated as follows: ； Where: For a pixel P The depth of is a neighboring pixel point P The depth of is a neighboring pixel point P The depth of is the horizontal depth difference, is the depth difference in the vertical direction; Get pixel point P With adjacent pixel point P and P The actual distance between the pixels is obtained by substituting the actual distance, the depth difference in the horizontal direction, and the depth difference in the vertical direction into the preset slope calculation model to obtain the pixel point P The slope of The expression of the preset slope calculation model is as follows: ； Where: is the slope, and is the actual distance between adjacent pixels, is the inverse tangent function; For each pixel in the depth map, repeat the above slope calculation process to obtain the slope of each pixel in the depth map; The slopes of all pixels are averaged to obtain the average slope value, and the average slope value of the candidate area corresponding to each depth map is counted; Sort all average slope values ​​from large to small, and mark the average slope value corresponding to the first one in the sort as the target forced landing ground area.

4. The method for controlling the landing of a UAV based on visual recognition according to claim 3 is characterized in that: The training logic of the pre-built speed prediction model is as follows: Acquire historical UAV speed prediction training data, and divide the historical UAV speed prediction training data into a speed prediction training set and a speed prediction test set, wherein the historical UAV speed prediction training data includes speed prediction features and their corresponding first safe forced landing speeds; Wherein, the speed prediction features include altitude data and a second safe forced landing speed; The second safe emergency landing speed is obtained by optimizing based on a preconfigured genetic optimization algorithm, and the optimization logic of the genetic optimization algorithm is as follows: S311: Initialize the population: randomly generate an original population, wherein the original population contains Y chromosomes, each chromosome represents the initial forced landing speed of a UAV, and Y is an integer greater than zero; S312: Fitness evaluation: Under each chromosome, the landing time and vibration frequency of the drone are obtained, and the landing time and vibration frequency are input into the pre-built fitness function to calculate the fitness of each chromosome; S313: Selection: Use the roulette wheel method to select two chromosomes with high fitness in the original population as the father and mother; S314: Crossover: Perform a crossover operation on the father and mother to generate new chromosomes; S315: mutation: performing mutation operation on the new chromosome to obtain Y new chromosomes, combining the Y new chromosomes into a new population, replacing the original population with the new population, and returning to step S312; S316: Repeat the above steps S312 to S315 until the fitness of the chromosome in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, and output the initial forced landing speed represented by the corresponding chromosome as the first safe forced landing speed.

5. The method for controlling the landing of a UAV based on visual recognition according to claim 4, characterized in that: The logic for obtaining the vibration frequency is as follows: Obtain the randomly generated initial forced landing speed of the drone, and obtain the wind force value of the drone's environment; The wind force value and the initial forced landing speed are input into the pre-built vibration frequency prediction model to obtain the vibration frequency.

6. The method for controlling the landing of a UAV based on visual recognition according to claim 5, characterized in that: The training logic of the pre-built vibration frequency prediction model is as follows: Acquire historical drone vibration frequency training data, and divide the historical drone vibration frequency training data into a vibration frequency training set and a vibration frequency test set; the vibration frequency training includes vibration frequency prediction features and their corresponding vibration frequencies; Wherein, the vibration frequency prediction characteristics include wind force value and initial forced landing speed; Constructing a first regression network, taking the vibration frequency prediction features in the vibration frequency training set as input of the first regression network; and taking the vibration frequency as output of the first regression network; training the first regression network to obtain an initial vibration frequency prediction network; The initial vibration frequency prediction network is model-verified using the vibration frequency test set, and an initial vibration frequency prediction network that is less than or equal to a first test error threshold is output as a pre-constructed vibration frequency prediction model.

7. The method for controlling the landing of a UAV based on visual recognition according to claim 6, characterized in that: The calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is the landing time, and vibration frequency.

8. A UAV landing control system based on visual recognition, characterized in that: include: A data collection module is used to obtain a first ground image set, filter out a target forced landing ground area according to the first ground image set, control the UAV to fly to hover over the target forced landing ground area, and collect a hovering position and a first overhead image of the UAV during the hovering; The first ground image set includes K first ground images, where K is an integer greater than zero; The height detection module is used to obtain the hovering height of the UAV above the target forced landing ground area, and obtain the vertical distance between the UAV and all obstacles around the target forced landing ground area, and obtain height data according to the vertical distance and the hovering height, wherein the height data includes the target forced landing height, the first height difference and the second height difference; The first height difference is the height distance between the target forced landing height and the hovering height, and the second height difference is the height distance between the target forced landing height and the ground; The obtaining of height data according to the vertical distance and the hovering height includes: Extract the vertical distances of the drone from all obstacles around the target forced landing ground area to obtain U vertical distances, where U is an integer greater than zero; Sort the U vertical distances from small to large, and make the vertical distance corresponding to the first one in the sorting the closest vertical distance; Substitute the nearest vertical distance and hovering height into the preset height data calculation model to obtain the target forced landing height; the specific calculation formula is as follows: ; Where: is the hovering height, is the closest vertical distance, To achieve the target altitude, is a safety factor greater than zero; The speed analysis module is used to obtain a preset second safe forced landing speed, input the altitude data and the second safe forced landing speed into a pre-built speed prediction model to obtain a first safe forced landing speed of the UAV, and control the UAV to reach a target forced landing height at the first safe forced landing speed; and collect an updated position and a second overhead view image of the UAV when reaching the target forced landing height; The orientation judgment module is used to judge whether the UAV has left the target forced landing ground area according to the first bird's-eye view image and the second bird's-eye view image, and if not, trigger the landing control module; if not, adjust the updated position based on the hovering position to obtain the adjusted position; The landing control module is used to control the UAV to land smoothly from the updated position to the ground according to the second safe forced landing speed; or control the UAV to land smoothly from the adjusted position to the ground.

Images