A method and system for accurately landing a UAV based on image recognition

Abstract

The present invention discloses a method and system for accurately landing a drone based on image recognition, including S1: using the GPS satellite navigation system to place the drone near a landing beacon; S2: using a camera on the drone to take photos of the ground vertically downward, and constructing a target recognition network to identify the landing beacon; S3: constructing a height estimation network, and estimating the height of the drone from the ground in combination with a barometric altimeter; S4: jointly optimizing the target recognition network and the height estimation network; S5: controlling the drone to accurately land through a PID control algorithm. The present invention combines multi-resolution image features, the number of pixels of the landing beacon, and the data of the barometric altimeter to improve the accuracy of the drone's altitude estimation, thereby improving the effect of the drone's accurate landing, and solving the problems of high cost due to reliance on a high-precision GPS system and poor landing effect due to inaccurate altitude estimation when landing a drone based on traditional image recognition.

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicle control, and relates to a method and system for accurately positioning and landing an unmanned aerial vehicle based on image recognition. Background Art

[0002] Drone precision landing refers to the technology that enables drones to land at a predetermined location with high accuracy and reliability. Traditional drone precision landing technology mainly relies on high-precision GPS systems for positioning, but this method has a high overall cost due to the high price of high-precision GPS equipment.

[0003] In recent years, image-based drone precision landing methods have achieved precision landing by identifying target patterns or specific signs on the ground, and then combining barometric altimeters to calculate the relative position between the drone and the target. However, barometric altimeters rely on changes in air pressure to measure altitude, and air pressure is greatly affected by weather and environmental factors, resulting in unstable measurements, which in turn affects the precision landing effect. Summary of the invention

[0004] In view of this, the present invention aims to provide a method and system for precise positioning and landing of UAVs based on image recognition, aiming to solve the problems of high cost and inaccurate height estimation of traditional UAV precise landing methods, which leads to poor precise landing effect.

[0005] A method for accurately landing a UAV at a certain position based on image recognition, comprising:

[0006] S1: Use the GPS satellite navigation system to place the drone near the landing beacon;

[0007] S2: Use the camera on the drone to take photos of the ground vertically downward, and build a target recognition network to identify the landing beacon;

[0008] S3: Build a height estimation network and estimate the height of the drone from the ground by combining the barometric altimeter.

[0009] S4: Jointly optimize the target recognition network and the height estimation network;

[0010] S5: Control the drone to land accurately through PID control algorithm;

[0011] Furthermore, the step S1 uses the GPS satellite navigation system to place the drone near a landing beacon, including:

[0012] S11: The drone receives signals from satellites through a GPS receiver to determine its current location;

[0013] S12: Calculate the relative distance and direction between the UAV and the landing beacon using the signal from the satellite;

[0014] S13: According to the calculated relative position, the flight path of the UAV is adjusted so that the camera of the UAV can capture the landing beacon.

[0015] Furthermore, the step S2 uses the camera on the drone to take photos of the ground vertically downward, and constructs a target recognition network to identify the landing beacon, including:

[0016] S21: grayscale and binarize the image taken by the drone to obtain a grayscale image and a binarized image. The calculation method is:

[0017] Y=0.299R+0.587G+0.114B

[0018] Among them, Y is the calculated grayscale value, and R, G, and B are the values ​​of the red, green, and blue channels respectively;

[0019] Grayscale processing converts color images into grayscale images by reducing the color channels of the image, thus reducing the amount of calculation for subsequent processing;

[0020]

[0021] Among them, Y(x,y) is the grayscale value at the position (x,y), x and y are the horizontal and vertical coordinates of the image pixel, respectively, and f(x,y) is the value of the binarized image at the position (x,y); the binarization process further simplifies the image, converting the grayscale image into an image containing only black and white colors, highlighting the characteristics of the landing beacon;

[0022] S22: Construct a low-computation convolutional layer, the calculation method is:

[0023] For each input channel, use the first convolution kernel to perform a convolution operation to obtain a feature map; then use the second convolution kernel with a size of 1 to perform a convolution operation on the feature map;

[0024] The second convolution kernel of size 1 covers only one pixel at a time on the input feature map. Although the convolution kernel size is small, it can be applied to each position of the feature map and perform independent convolution on each channel of the input feature map, allowing the network to learn complex interactions between channels without increasing the computational burden. The first convolution kernel is responsible for applying filters independently within each channel, while the second convolution kernel is responsible for merging the outputs of these channels.

[0025] Compared with the traditional convolution layer, the low-computation convolution layer decomposes the convolution operation so that each convolution kernel only operates on a single input channel instead of all input channels, reducing the number of parameters and making the network more efficient and more suitable for real-time image processing of drones.

[0026] S23: Use low-computation convolutional layers to replace the convolutional layers in the YOLOv8 network to obtain an improved YOLOv8 network;

[0027] S24: The image taken by the drone, the grayscale image, and the binary image are channel-superimposed, and the improved YOLOv8 network is used to identify the landing beacon. The calculation method is:

[0028] predictions=YOLOv8(stack[RGB(x,y),Y(x,y),f(x,y)])

[0029] Among them, predictions is the output of the improved YOLOv8 network, including the center coordinates of the bounding box, the width and height of the bounding box, and the confidence that the bounding box contains the target; RGB(x,y) is the picture taken by the drone, and stack[,] is the channel stacking operation;

[0030] Compared with the traditional target detection network, by superimposing RGB(x,y), Y(x,y), and f(x,y) channels, the improved YOLOv8 network can utilize these three different types of data, while considering color information, grayscale levels, and obvious shape or edge features to enhance the performance of landing beacon recognition, which is especially beneficial for target detection under complex backgrounds or changing lighting conditions. Moreover, since the convolutional layers in the YOLOv8 network are replaced with low-computational convolutional layers, the number of parameters is controllable, which also ensures the real-time nature of landing beacon recognition. Even if real-time changes are encountered during flight, adjustments can be made quickly to ensure accurate landing.

[0031] Furthermore, the step S3 constructs a height estimation network, and estimates the height of the drone from the ground by combining the barometric altimeter, including:

[0032] S31: Count the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone as an input feature of the height estimation network, and use the picture taken by the drone as another input feature of the height estimation network;

[0033] S32: Construct a first path, and gradually reduce the resolution of the feature map through the convolution layer. For each convolution layer, the calculation method of the feature map output by the first path is:

[0034] C0=Conv(RGB(x,y))

[0035]

[0036] Among them, C0 is the feature map after preliminary convolution processing, is the output feature map of the first path, d is the convolutional layer index, is the feature map of the previous convolutional layer, and Conv() is the convolution operation;

[0037] S33: Construct the second path, use upsampling to increase the resolution of the feature map, and combine it with the feature map output by the first path. The calculation method is:

[0038]

[0039] in, Output feature map for the second path, UpSample() is the upsampling operation, Output feature map for the first path of the next convolutional layer;

[0040] The traditional convolution method does not consider the strategy of reducing the resolution layer by layer. In contrast, the present invention allows the network to capture features at different resolution levels. First, the resolution of the feature map is gradually reduced through the first path to extract the global information of the image, such as shape and structure, so as to obtain a feature map containing macro features. Then, the resolution of the feature map is increased through the upsampling technology in the second path to refine the micro features such as texture and edge of the image. Finally, the two feature maps of different granularity are fused together to provide a multi-scale information perspective for the altitude estimation network of the drone, enrich the perception ability of the network, and enhance the accuracy and reliability of altitude estimation.

[0041] S34: Use the fully connected layer to concatenate the number of pixels of the landing beacon in the picture taken by the drone and the feature map output by the second path, and input them into the height estimation network to output the predicted height result. The calculation method is:

[0042]

[0043] Among them, H pred is the height result predicted by the height estimation network, sigmoid() is the sigmoid function, W1 is the weight matrix, (m,n) is the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone, b1 is the bias vector, and Concat{,} is the concatenation operation;

[0044] The number of pixels of the landing beacon will change with the change of the altitude of the UAV, and thus directly reflects the size of the landing beacon in the image. In traditional altitude estimation networks, altitude estimation is usually based only on the visual features of the image, without considering quantitative information such as the number of pixels. In contrast, the present invention uses the number of pixels of the landing beacon as an explicit input feature and splices it with the feature map output by the second path, thereby providing an intuitive scale reference for the network, which is conducive to the network estimating the relative altitude between the UAV and the target more accurately. At the same time, using the number of pixels as an independent feature input makes the model more sensitive to small changes in the number of pixels, thereby improving the response speed to altitude changes.

[0045] S35: The altitude data measured by the barometric altimeter is integrated with the network prediction result to obtain the altitude result of the drone. The calculation method is:

[0046] H=sigmoid(W2·Concat{H pred ,h}+b2)

[0047] Among them, H is the final predicted drone height, W2 is the weight matrix, b2 is the bias vector, and h is the height data measured by the barometric altimeter;

[0048] The barometric altimeter is greatly affected by changes in air pressure. Under complex environmental conditions, such as unstable air pressure or climate change, a single barometric altimeter may not be able to provide accurate altitude data. The altitude estimation network provides visual information. The combination of the two complements each other, improving the accuracy of altitude estimation and enhancing the robustness of the system.

[0049] Furthermore, the step S4 sets the input, output, and optimization target of the height estimation network, including:

[0050] S41: Set the input of the target recognition network to be the picture taken by the drone, and the output to be the center coordinate of the bounding box, the width and height of the bounding box, and the confidence that the bounding box contains the target;

[0051] S42: The input of the height estimation network is set to be the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone, the picture taken by the drone, and the height data measured by the barometric altimeter; the output is the final drone height;

[0052] S43: The calculation method of the height estimation loss function is:

[0053]

[0054] Among them, L height is the height estimation loss function, N is the number of samples, i is the sample number index, H iis the final predicted drone height of the i-th data, is the actual height of the drone for the i-th data;

[0055] S44: The target detection loss function uses the loss function L of YOLOv8 detection , including classification loss, bounding box coordinate loss and confidence loss;

[0056] S45: The final joint loss function is calculated as:

[0057] L joint =λ1L height +λ2L detection

[0058] Among them, L joint is the joint loss function, λ1 and λ2 are the weight coefficients of the loss function;

[0059] The object recognition network focuses on locating and identifying objects in the image, while the altitude estimation network uses this information to predict the altitude of the drone; joint The height estimation loss and target detection loss are combined. By adjusting the weight coefficients λ1 and λ2, the optimization objectives of the two networks can be balanced, thereby improving the accuracy of height estimation while ensuring the accuracy of target detection.

[0060] Furthermore, the step S5 controls the drone to land precisely by using a PID control algorithm, including:

[0061] S51: setting proportional, integral and differential gain parameters in the PID controller, and adjusting the PID parameters according to the real-time position and ground clearance data;

[0062] The control system monitors the position and height of the drone in real time, and based on the real-time monitoring data, the control system dynamically adjusts the proportional, integral and differential gain parameters in the PID controller. If the drone deviates from the planned path, the proportional gain increases to generate stronger control force and quickly correct the deviation. In this way, the PID controller can flexibly adjust the control strategy to ensure that the drone can achieve precise landing under various flight conditions.

[0063] S52: Calculate the position error between the current position of the UAV and the landing beacon in real time, and decompose the position error into position errors in the lateral, longitudinal and height directions;

[0064] S53: Perform a PID control loop to adjust the control input of the UAV according to the position errors in the lateral, longitudinal and height directions;

[0065] S54: Generate thrust and attitude commands for controlling the landing of the UAV according to the result of the PID control loop, and control the UAV to land accurately.

[0066] The present invention also provides a UAV precise position landing system based on image recognition, comprising:

[0067] Position coarse control module: Use the GPS satellite navigation system to keep the drone near the landing beacon;

[0068] Landing beacon identification module: Use the camera on the drone to take photos of the ground vertically downward, and build a target recognition network to identify the landing beacon;

[0069] Height estimation module: builds a height estimation network and estimates the height of the drone from the ground by combining the barometric altimeter;

[0070] Network joint optimization module: jointly optimizes the target recognition network and the height estimation network;

[0071] Position precision control module: Control the UAV to land precisely through the PID control algorithm, so as to realize the above-mentioned UAV precise position landing method based on image recognition.

[0072] Compared with the prior art, the advantages of the present invention are:

[0073] (1) In terms of the overall solution, the present invention adopts the idea of ​​image processing to achieve precise positioning of the UAV landing, which reduces the cost compared to the GPS system that relies on expensive hardware. At the same time, the present invention combines image features with the data of the barometric altimeter, does not rely on a single measurement technology, improves the accuracy of the UAV altitude estimation, and thus improves the effect of the UAV precise positioning landing.

[0074] (2) In terms of algorithm improvement, the present invention uses a low-parameter convolutional layer to reduce the complexity of the target recognition network and improve the recognition speed of the landing beacon; at the same time, the present invention combines two feature maps with different granularities, so that the model can capture information at more scales, thereby improving the accuracy of target detection and height estimation.

[0075] (3) In terms of data utilization, the present invention uses the number of pixels of the landing beacon as an input feature, which improves the model's sensitivity to image scale changes and integrates it with the altitude data of the barometric altimeter to provide a more comprehensive UAV altitude estimation method, reducing the uncertainty and error of relying on a single sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of the process flow of the method for accurately landing a UAV based on image recognition provided by the present invention.

[0077] Figure 2 A schematic diagram of the algorithm flow of the target recognition network provided by the present invention.

[0078] Figure 3 A schematic diagram of the algorithm flow of the height estimation network provided by the present invention. DETAILED DESCRIPTION

[0079] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0080] Embodiment 1: A method for accurately landing a drone based on image recognition, such as Figure 1 As shown, including:

[0081] S1: Use the GPS satellite navigation system to place the drone near the landing beacon;

[0082] S11: The drone receives signals from satellites through a GPS receiver to determine its current location;

[0083] S12: Calculate the relative distance and direction between the UAV and the landing beacon using the signal from the satellite;

[0084] S13: According to the calculated relative position, the flight path of the UAV is adjusted so that the camera of the UAV can capture the landing beacon.

[0085] S2: Use the camera on the drone to take photos of the ground vertically downward, and build a target recognition network to identify landing beacons, such as Figure 2 As shown;

[0086] S21: grayscale and binarize the image taken by the drone to obtain a grayscale image and a binarized image. The calculation method is:

[0087] Y=0.299R+0.587G+0.114B

[0088] Among them, Y is the calculated grayscale value, and R, G, and B are the values ​​of the red, green, and blue channels respectively;

[0089] Grayscale processing converts color images into grayscale images by reducing the color channels of the image, thus reducing the amount of calculation for subsequent processing;

[0090]

[0091] Among them, Y(x,y) is the grayscale value at the position (x,y), x and y are the horizontal and vertical coordinates of the image pixel, respectively, and f(x,y) is the value of the binarized image at the position (x,y); the binarization process further simplifies the image, converting the grayscale image into an image containing only black and white colors, highlighting the characteristics of the landing beacon;

[0092] S22: Construct a low-computation convolutional layer, the calculation method is:

[0093] For each input channel, use the first convolution kernel to perform a convolution operation to obtain a feature map; then use the second convolution kernel with a size of 1 to perform a convolution operation on the feature map;

[0094] The second convolution kernel of size 1 covers only one pixel at a time on the input feature map. Although the convolution kernel size is small, it can be applied to each position of the feature map and perform independent convolution on each channel of the input feature map, allowing the network to learn complex interactions between channels without increasing the computational burden. The first convolution kernel is responsible for applying filters independently within each channel, while the second convolution kernel is responsible for merging the outputs of these channels.

[0095] Compared with the traditional convolution layer, the low-computation convolution layer decomposes the convolution operation so that each convolution kernel only operates on a single input channel instead of all input channels, reducing the number of parameters and making the network more efficient and more suitable for real-time image processing of drones.

[0096] For example, assuming that the input feature map size is 32*32*3, it is convolved with a 3*3 convolution kernel, and the number of channels of the output feature map is 64, then:

[0097] Traditional convolutional layer:

[0098] The number of parameters for each convolution kernel is 3*3*3=27

[0099] The total number of convolution kernel parameters is 64*27=1728

[0100] Each pixel on the output feature map requires 27 multiplications and 26 additions.

[0101] For the entire 30*30*64 output feature map, the total computational effort is 30*30*64*(27+26)=3052800 low computational convolutional layers:

[0102] Each input channel uses a 3*3 convolution kernel, a total of 3 convolution kernels, each convolution kernel has 9 parameters, the total number of parameters is 3*9=27

[0103] Convolution is performed on this basis using a 1*1 convolution kernel. Each 1*1 convolution kernel has 3 parameters (corresponding to the number of output channels). For 64 output channels, the total number of parameters is 64*3=192.

[0104] The total number of convolution kernel parameters is 27+192=219

[0105] Each pixel on the output feature map requires 9 multiplications and 8 additions, plus 3 multiplications and 2 additions.

[0106] For the entire 30*30*64 output feature map, the total amount of computation is (30*30*(9+8)+(30*30*64*(3+2))=303300

[0107] After adopting the low-computation convolution layer, the amount of computation is reduced to about one-tenth of the original amount;

[0108] S23: Use low-computation convolutional layers to replace the convolutional layers in the YOLOv8 network to obtain an improved YOLOv8 network;

[0109] S24: The image taken by the drone, the grayscale image, and the binary image are channel-superimposed, and the improved YOLOv8 network is used to identify the landing beacon. The calculation method is:

[0110] predictions=YOLOv8(stack[RGB(x,y),Y(x,y),f(x,y)])

[0111] Among them, predictions is the output of the improved YOLOv8 network, including the center coordinates of the bounding box, the width and height of the bounding box, and the confidence that the bounding box contains the target; RGB(x,y) is the picture taken by the drone, and stack[,] is the channel stacking operation;

[0112] Compared with the traditional target detection network, by superimposing RGB(x,y), Y(x,y), and f(x,y) channels, the improved YOLOv8 network can utilize these three different types of data, while considering color information, grayscale levels, and obvious shape or edge features to enhance the performance of landing beacon recognition, which is especially beneficial for target detection under complex backgrounds or changing lighting conditions. Moreover, since the convolutional layers in the YOLOv8 network are replaced with low-computational convolutional layers, the number of parameters is controllable, which also ensures the real-time nature of landing beacon recognition. Even if real-time changes are encountered during flight, adjustments can be made quickly to ensure accurate landing.

[0113] S3: Build a height estimation network and estimate the height of the drone from the ground by combining the barometric altimeter. Figure 3 As shown;

[0114] S31: Count the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone as an input feature of the height estimation network, and use the picture taken by the drone as another input feature of the height estimation network;

[0115] S32: Construct a first path, and gradually reduce the resolution of the feature map through the convolution layer. For each convolution layer, the calculation method of the feature map output by the first path is:

[0116] C0=Conv(RGB(x,y))

[0117]

[0118] Among them, C0 is the feature map after preliminary convolution processing, is the output feature map of the first path, d is the convolutional layer index, is the feature map of the previous convolutional layer, and Conv() is the convolution operation;

[0119] S33: Construct the second path, use upsampling to increase the resolution of the feature map, and combine it with the feature map output by the first path. The calculation method is:

[0120]

[0121] in, Output feature map for the second path, UpSample() is the upsampling operation, Output feature map for the first path of the next convolutional layer;

[0122] The traditional convolution method does not consider the strategy of reducing the resolution layer by layer. In contrast, the present invention allows the network to capture features at different resolution levels. First, the resolution of the feature map is gradually reduced through the first path to extract the global information of the image, such as shape and structure, so as to obtain a feature map containing macro features. Then, the resolution of the feature map is increased through the upsampling technology in the second path to refine the micro features such as texture and edge of the image. Finally, the two feature maps of different granularity are fused together to provide a multi-scale information perspective for the altitude estimation network of the drone, enrich the perception ability of the network, and enhance the accuracy and reliability of altitude estimation.

[0123] S34: Use the fully connected layer to concatenate the number of pixels of the landing beacon in the picture taken by the drone and the feature map output by the second path, and input them into the height estimation network to output the predicted height result. The calculation method is:

[0124]

[0125] Among them, H pred is the height result predicted by the height estimation network, sigmoid() is the sigmoid function, W1 is the weight matrix, (m,n) is the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone, b1 is the bias vector, and Concat{,} is the concatenation operation;

[0126] The number of pixels of the landing beacon will change with the change of the altitude of the UAV, and thus directly reflects the size of the landing beacon in the image. In traditional altitude estimation networks, altitude estimation is usually based only on the visual features of the image, without considering quantitative information such as the number of pixels. In contrast, the present invention uses the number of pixels of the landing beacon as an explicit input feature and splices it with the feature map output by the second path, thereby providing an intuitive scale reference for the network, which is conducive to the network estimating the relative altitude between the UAV and the target more accurately. At the same time, using the number of pixels as an independent feature input makes the model more sensitive to small changes in the number of pixels, thereby improving the response speed to altitude changes.

[0127] As the altitude of the drone changes, the size of the landing beacon in the captured image will also change accordingly, and this change will be reflected in the increase or decrease in the number of pixels. For example, when the drone drops from an altitude of 100 meters to 50 meters, the number of pixels of the landing beacon may increase from 200*200 to 400*400. This change provides real-time dynamic input data for the altitude estimation network, allowing the network to adjust the predicted altitude value according to the change in the number of pixels.

[0128] S35: The altitude data measured by the barometric altimeter is integrated with the network prediction result to obtain the altitude result of the drone. The calculation method is:

[0129] H=sigmoid(W2·Concat{H pred ,h}+b2)

[0130] Among them, H is the final predicted drone height, W2 is the weight matrix, b2 is the bias vector, and h is the height data measured by the barometric altimeter;

[0131] The barometric altimeter is greatly affected by changes in air pressure. Under complex environmental conditions, such as unstable air pressure or climate change, a single barometric altimeter may not be able to provide accurate altitude data. The altitude estimation network provides visual information. The combination of the two complements each other, improving the accuracy of altitude estimation and enhancing the robustness of the system.

[0132] S4: Jointly optimize the target recognition network and the height estimation network;

[0133] S41: Set the input of the target recognition network to be the picture taken by the drone, and the output to be the center coordinate of the bounding box, the width and height of the bounding box, and the confidence that the bounding box contains the target;

[0134] S42: The input of the height estimation network is set to be the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone, the picture taken by the drone, and the height data measured by the barometric altimeter; the output is the final drone height;

[0135] S43: The calculation method of the height estimation loss function is:

[0136]

[0137] Among them, L height is the height estimation loss function, N is the number of samples, i is the sample number index, H i is the final predicted drone height of the i-th data, is the actual height of the drone for the i-th data;

[0138] S44: The target detection loss function uses the loss function L of YOLOv8 detection , including classification loss, bounding box coordinate loss and confidence loss;

[0139] S45: The final joint loss function is calculated as:

[0140] L joint =λ1L height +λ2L detection

[0141] Among them, L joint is the joint loss function, λ1 and λ2 are the weight coefficients of the loss function;

[0142] The object recognition network focuses on locating and identifying objects in the image, while the altitude estimation network uses this information to predict the altitude of the drone; joint The height estimation loss and target detection loss are combined. By adjusting the weight coefficients λ1 and λ2, the optimization objectives of the two networks can be balanced, thereby improving the accuracy of height estimation while ensuring the accuracy of target detection.

[0143] S5: Control the drone to land accurately through PID control algorithm;

[0144] S51: setting proportional, integral and differential gain parameters in the PID controller, and adjusting the PID parameters according to the real-time position and ground clearance data;

[0145] The control system monitors the position and height of the drone in real time, and based on the real-time monitoring data, the control system dynamically adjusts the proportional, integral and differential gain parameters in the PID controller. If the drone deviates from the planned path, the proportional gain increases to generate stronger control force and quickly correct the deviation. In this way, the PID controller can flexibly adjust the control strategy to ensure that the drone can achieve precise landing under various flight conditions.

[0146] S52: Calculate the position error between the current position of the UAV and the landing beacon in real time, and decompose the position error into position errors in the lateral, longitudinal and height directions;

[0147] S53: Perform a PID control loop to adjust the control input of the UAV according to the position errors in the lateral, longitudinal and height directions;

[0148] S54: Generate thrust and attitude commands for controlling the landing of the UAV according to the result of the PID control loop, and control the UAV to land accurately.

[0149] The present invention is particularly suitable for situations where the GPS positioning accuracy is insufficient; the cost of high-precision GPS may be too high, and in certain environments, such as urban areas blocked by high-rise buildings, the GPS signal may be unstable, affecting the landing accuracy; the present invention can achieve high-precision position estimation and altitude control in the absence of high-precision GPS, thereby improving the landing accuracy and reliability of the UAV in a changing environment. Even in situations where the air pressure is greatly affected by weather changes, this solution can compensate for the shortcomings of the barometric altimeter through image recognition technology, ensuring that the UAV can accurately land at the predetermined location.

[0150] Embodiment 2: The present invention also discloses a UAV precise location landing system based on image recognition, which is used to implement the above-mentioned UAV precise location landing method based on image recognition, including:

[0151] Position coarse control module: Use the GPS satellite navigation system to keep the drone near the landing beacon;

[0152] Landing beacon identification module: Use the camera on the drone to take photos of the ground vertically downward, and build a target recognition network to identify the landing beacon;

[0153] Height estimation module: builds a height estimation network and estimates the height of the drone from the ground by combining the barometric altimeter;

[0154] Network joint optimization module: jointly optimizes the target recognition network and the height estimation network;

[0155] Position precision control module: Control the drone to land precisely through PID control algorithm.

[0156] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0158] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for accurately landing a UAV based on image recognition, characterized in that: include: S1: Use the GPS satellite navigation system to place the drone near the landing beacon; S2: Use the camera on the drone to take photos of the ground vertically downward, and build a target recognition network to identify the landing beacon; After graying and binarizing the pictures taken by the drone, the pictures taken by the drone, the grayed images, and the binarized images are channel-superimposed and input into the improved YOLOv8 network to identify the landing beacon. The improved YOLOv8 network is characterized in that a low-computation convolutional layer is reconstructed and the original convolutional layer is replaced; S3: Build a height estimation network and estimate the height of the drone from the ground by combining the barometric altimeter. The number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone is used as an input feature of the height estimation network, and the picture taken by the drone is used as another input feature of the height estimation network; two convolution paths are used to extract the picture features respectively and fuse the data with the height data measured by the barometric altimeter to output the drone height result; including: S31: Count the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone as an input feature of the height estimation network, and use the picture taken by the drone as another input feature of the height estimation network; S32: Construct a first path, and gradually reduce the resolution of the feature map through the convolution layer. For each convolution layer, the calculation method of the feature map output by the first path is: C0=Conv(RGB(x,y)) Among them, C0 is the feature map after preliminary convolution processing, is the output feature map of the first path, d is the convolutional layer index, is the feature map of the previous convolutional layer, Conv() is the convolution operation; RGB(x,y) is the picture taken by the drone; S33: Construct the second path, use upsampling to increase the resolution of the feature map, and combine it with the feature map output by the first path. The calculation method is: in, Output feature map for the second path, UpSample() is the upsampling operation, Output feature map for the first path of the next convolutional layer; S34: Use the fully connected layer to concatenate the number of pixels of the landing beacon in the picture taken by the drone and the feature map output by the second path, and input them into the height estimation network to output the predicted height result. The calculation method is: Among them, H pred is the height result predicted by the height estimation network, sigmoid() is the sigmoid function, W1 is the weight matrix, (m,n) is the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone, b1 is the bias vector, and Concat{,} is the concatenation operation; S35: The altitude data measured by the barometric altimeter is integrated with the network prediction result to obtain the altitude result of the drone. The calculation method is: H=sigmoid(W2·Concat{H pred ,h}+b2) Among them, H is the final predicted drone height, W2 is the weight matrix, b2 is the bias vector, and h is the height data measured by the barometric altimeter; S4: Jointly optimize the target recognition network and the height estimation network; S5: Control the drone to land precisely through PID control algorithm.

2. The method for accurately landing a UAV based on image recognition according to claim 1, characterized in that: The step S1 comprises: S11: The drone receives signals from satellites through a GPS receiver to determine its current location; S12: Calculate the relative distance and direction between the UAV and the landing beacon using the signal from the satellite; S13: According to the calculated relative position, the flight path of the UAV is adjusted so that the camera of the UAV can capture the landing beacon.

3. The method for accurately landing a UAV based on image recognition according to claim 2 is characterized in that: The step S2 comprises: S21: grayscale and binarize the image taken by the drone to obtain a grayscale image and a binarized image. The calculation method is: Y=0.299R+0.587G+0.114B Among them, Y is the calculated grayscale value, and R, G, and B are the values ​​of the red, green, and blue channels respectively; Among them, Y(x,y) is the grayscale value at the position (x,y), x and y are the horizontal and vertical coordinates of the image pixel respectively, and f(x,y) is the value of the binary image at the position (x,y); S22: Construct a low-computation convolutional layer, the calculation method is: For each input channel, use the first convolution kernel to perform a convolution operation to obtain a feature map; then use the second convolution kernel with a size of 1 to perform a convolution operation on the feature map; S23: Use low-computation convolutional layers to replace the convolutional layers in the YOLOv8 network to obtain an improved YOLOv8 network; S24: The image taken by the drone, the grayscale image, and the binary image are channel-superimposed, and the improved YOLOv8 network is used to identify the landing beacon. The calculation method is: predictions=YOLOv8(stack[RGB(x,y),Y(x,y),f(x,y)]) Among them, predictions is the output of the improved YOLOv8 network, including the center coordinates of the bounding box, the width and height of the bounding box, and the confidence that the bounding box contains the target; RGB(x,y) is the picture taken by the drone, and stack[,] is the channel stacking operation.

4. The method for accurately landing a UAV based on image recognition according to claim 3 is characterized in that: The step S4 comprises: S41: Set the input of the target recognition network to be the picture taken by the drone, and the output to be the center coordinate of the bounding box, the width and height of the bounding box, and the confidence that the bounding box contains the target; S42: The input of the height estimation network is set to be the number of pixels in the horizontal and vertical directions of the landing beacon in the picture taken by the drone, the picture taken by the drone, and the height data measured by the barometric altimeter; the output is the final drone height; S43: The calculation method of the height estimation loss function is: Among them, L height is the height estimation loss function, N is the number of samples, i is the sample number index, H i is the final predicted drone height of the i-th data, is the actual height of the drone for the i-th data; S44: The target detection loss function uses the loss function L of YOLOv8 detection , including classification loss, bounding box coordinate loss and confidence loss; S45: The final joint loss function is calculated as: L joint =λ1L height +λ2L detection Among them, L joint is the joint loss function, λ1 and λ2 are the weight coefficients of the loss function.

5. The method for accurately landing a UAV based on image recognition according to claim 4 is characterized in that: The step S5 comprises: S51: setting proportional, integral and differential gain parameters in the PID controller, and adjusting the PID parameters according to the real-time position and ground clearance data; S52: Calculate the position error between the current position of the UAV and the landing beacon in real time, and decompose the position error into position errors in the lateral, longitudinal and height directions; S53: Perform a PID control loop to adjust the control input of the UAV according to the position errors in the lateral, longitudinal and height directions; S54: Generate thrust and attitude commands for controlling the landing of the UAV according to the result of the PID control loop, and control the UAV to land accurately.

6. A UAV precise location landing system based on image recognition, characterized in that: include: Position coarse control module: Use the GPS satellite navigation system to keep the drone near the landing beacon; Landing beacon identification module: Use the camera on the drone to take photos of the ground vertically downward, and build a target recognition network to identify the landing beacon; Height estimation module: builds a height estimation network and estimates the height of the drone from the ground by combining the barometric altimeter; Network joint optimization module: jointly optimizes the target recognition network and the height estimation network; Position precision control module: Control the drone to land precisely through PID control algorithm; To realize the UAV precise location landing method based on image recognition as described in any one of claims 1-5.

Images