Unmanned aerial vehicle landing guidance method based on monocular depth estimation network

Abstract

The application relates to a UAV landing guidance method based on a monocular depth estimation network, and belongs to the technical field of UAV landing. The UAV landing is divided into three stages, the first stage is a global positioning stage, a GPS / IMU is used to coarsely position the UAV above a landing mark; the second stage is an identification positioning stage, target detection is carried out on the landing mark, and a monocular depth estimation network based on a DepthMaster model is used to carry out depth estimation, so that the UAV approaches the landing mark; the third stage is a fine positioning stage, three-dimensional information of a known landing mark is matched with two-dimensional information of the landing mark obtained through image processing by using a PnP algorithm, pose information is solved, and accurate landing of the UAV is realized in cooperation with depth estimation information, so that the safety and reliability of the system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle landing, in particular to an unmanned aerial vehicle landing guiding method based on a monocular depth estimation network. BACKGROUND

[0002] In the field of image processing, deep learning networks have achieved great success in object classification and detection. With the rapid development of deep learning and its combination with various fields, it has shown great potential and commercial value. Deep learning has shown strong analytical and expressive power in computer vision, making it possible to estimate depth from a single image. Monocular image depth estimation based on deep learning has become one of the hotspots of researchers in recent years. In the problem of monocular depth estimation, deep learning networks have great advantages over traditional image algorithms. For example, Chinese patent No. CN111402310B and Chinese patent application Nos. CN111539922A and CN116188555A.

[0003] In recent years, with the continuous progress of unmanned aerial vehicle technology, its practical application scenarios have become increasingly complex and diverse, which undoubtedly puts higher requirements on the precision and reliability of unmanned aerial vehicle landing technology. How to ensure precise landing of unmanned aerial vehicles has become a key technical problem that needs to be solved in the field of unmanned aerial vehicles, attracting attention and concern from all walks of life.

[0004] In the field of unmanned aerial vehicle landing guidance, traditional methods include instrument landing system (ILS), global satellite navigation system (GNSS), and inertial navigation system (INS), etc. Among them, ILS uses precise landing radar at the landing site to analyze the relative position data between the landing site and the unmanned aerial vehicle, and the positioning accuracy depends on the landing radar, which cannot meet the landing needs of unmanned aerial vehicles in complex and variable environments. GNSS is widely used in unmanned aerial vehicle landing guidance due to its stable signal and mature technology, but it has the disadvantages of low signal update rate and susceptibility to interference. INS realizes navigation through inertial sensors, but it will produce cumulative error during operation, and is usually combined with GNSS to improve the reliability and accuracy of unmanned aerial vehicle landing guidance.

[0005] Accurate position estimation is the key to precise landing of unmanned aerial vehicles. The binocular vision-based method can accurately estimate the spatial distance, but high-precision binocular depth cameras are expensive, and the measurement range is proportional to the baseline (the distance between the two cameras), which cannot be increased indefinitely, limiting its long-distance measurement capability. SUMMARY

[0006] In view of the above problems, the present application provides a UAV landing guidance method based on a monocular depth estimation network, which divides the UAV landing into three stages: the first stage is the global positioning stage, the UAV is coarsely positioned above the landing mark using GPS / IMU; the second stage is the identification positioning stage, the landing mark is detected, and the monocular depth estimation network based on the DepthMaster model is used for depth estimation, so that the UAV approaches the landing mark; the third stage is the fine positioning stage, the three-dimensional information of the known landing mark is matched with the two-dimensional information of the landing mark obtained by image processing by using the PnP algorithm, and the pose information is solved, and the depth estimation information is used to realize the accurate landing of the UAV and improve the safety and reliability of the system.

[0007] The present application provides a UAV landing guidance method based on a monocular depth estimation network, characterized in that it comprises:

[0008] Step S1, let i=1, when i=1, it represents the first UAV;

[0009] Step S2, globally positioning the i-th UAV to obtain the globally positioned i-th UAV;

[0010] Step S3, identifying the corresponding original landing area image S i based on the globally positioned i-th UAV;

[0011] Step S4, let n=1, when n=1, it represents the first landing mark;

[0012] Step S5, inputting the original landing area image S i into the YOLOv8 model A i to identify the n-th landing mark Y i,n , and adjusting the position of the globally positioned i-th UAV so that the landing mark Y i,n is in the center position of the landing area image, to obtain the i-th position updated UAV Z i,n ;

[0013] Based on the i-th position updated UAV Z i,n , the landing area image S' i,n is obtained;

[0014] Step S6, inputting the landing area image S' i,n into the monocular depth estimation network B i,n to obtain the depth estimation features of the i-th position updated UAV Z i,n ;

[0015] Adjusting the i-th position updated UAV Z i,nDistance to the landing area, the i-th position-updated drone Z i,n Distance to the landing area, the i-th position-updated drone Z i,n ;

[0016] Step S7, based on the i-th position-updated drone Z i,n Obtain updated landing area image S" i,n ; Based on the updated landing area image S" i,n Get the pose information T of the i-th position-updated drone i,n ;

[0017] Step S8, using the pose information T of the i-th position-updated drone i,n And the depth estimation feature of the i-th position-updated drone Z i,n Adjust the position of the i-th position-updated drone Z i,n Complete the i-th position-updated drone Z i,n Positioning of the n-th landing marker ;

[0018] Step S9, judge whether n is greater than or equal to N, N represents the total number of landing markers, if yes, complete the positioning of each landing marker by the i-th drone, if not, let n=n+1, return to step S5 ;

[0019] Step S10, judge whether i is greater than or equal to I, I represents the total number of drones in the training set, if yes, complete the positioning of each landing marker by each drone, and get the final monocular depth estimation network; if not, let i=i+1, return to step S2 ;

[0020] Step S11, based on the final monocular depth estimation network, guide the landing of the drone.

[0021] Optionally, the landing mark is an infrared LED.

[0022] Optionally, the specific steps of obtaining the depth estimation feature of the i-th position-updated drone Z i,n include:

[0023] Input the landing area image S' i,n to the monocular depth estimation network B i,n , encode the landing area image S' i,n in the hidden space, and then input it to the U-Net model D i,n to extract the corresponding image local feature F unet,i,n ;

[0024] Extract the image external feature F ext,i,n of the landing area image S' i,n ;

[0025] Based on the feature alignment loss, the local features F unet,i,n of the image are aligned with the external features F ext,i,n of the image to obtain the aligned features of the landing area image S′ i,n ;

[0026] The aligned features of the landing area image S′ i,n are input into a Fourier transform module F i,n to obtain the frequency domain features of the landing area image S′ i,n ;

[0027] The frequency domain features of the landing area image S′ i,n are input into a modulator G i,n to obtain the balanced frequency domain features of the landing area image S′ i,n ;

[0028] The balanced frequency domain features of the landing area image S′ i,n are transformed back to the spatial domain using an inverse fast Fourier transform to obtain the optimized landing area image F f,i,n ;

[0029] The spatial features F s,i,n of the landing area image are obtained;

[0030] The optimized landing area image F f,i,n is concatenated with the spatial features F s,i,n of the landing area image to perform feature complementation to obtain the complementary features of the landing area image S′ i,n ;

[0031] The complementary features of the landing area image S′ i,n are decoded to obtain the depth estimation features of the i-th position-updated UAV Z i,n .

[0032] Optionally, the feature alignment loss is expressed as:

[0033]

[0034] wherein l fa is the feature alignment loss, dist(.) represents the distance between two feature distributions, F ext,i,n is the projected feature of the n-th landing mark image local feature of the i-th position-updated UAV, and F is the external feature of the n-th landing mark of the i-th position-updated UAV.

[0035] Optionally, the specific steps of obtaining the pose information of the i-th position-updated UAV include:

[0036] Step S71, obtaining an updated landing area image S" i,n pixel coordinates and three-dimensional coordinates of each landing mark;

[0037] obtaining an updated landing area image S" i,n conversion relationship between the pixel coordinates and the three-dimensional coordinates of each landing mark;

[0038] Step S72, constructing an updated landing area image S" i least square problem related to the conversion relationship between the pixel coordinates and the three-dimensional coordinates of each landing mark;

[0039] Step S73, minimizing an updated landing area image S" i,n least square problem related to the conversion relationship between the pixel coordinates and the three-dimensional coordinates of each landing mark, to obtain the pose information T of the i-th position second-updated unmanned aerial vehicle i,n .

[0040] Optionally, the updated landing area image S" i least square problem related to the conversion relationship between the pixel coordinates and the three-dimensional coordinates of each landing mark,

[0041] The expression is:

[0042]

[0043] wherein E is a least square error value, indicates that the corresponding T value is obtained according to the least square error value of the conversion relationship between the pixel coordinates and the three-dimensional coordinates of the landing mark, s i,n is a scale factor of the n-th landing mark of the i-th unmanned aerial vehicle, i = 1, 2, 3,..., I, P i,n is a projection pixel coordinate of the n-th landing mark of the i-th unmanned aerial vehicle, K is an intrinsic matrix of a camera, and T is a Lie group representation of a camera pose R and a translation vector t.

[0044] Optionally, the pose information T of the i-th position second-updated unmanned aerial vehicle in step S7 i,n includes a rotation matrix of the i-th position second-updated unmanned aerial vehicle and a translation vector of the i-th position second-updated unmanned aerial vehicle.

[0045] Compared with the prior art, the present application has at least the following beneficial effects:

[0046] (1) The application uses the DepthMaster model to establish a monocular depth estimation network for depth estimation, the DepthMaster model is based on a pre-trained Stable Diffusion v2 model, and is trained on a large-scale LAION-5B data set, and contains rich image prior knowledge, so the DepthMaster does not need to collect and train large amounts of data, which can effectively reduce the time cost and labor cost;

[0047] (2) The monocular depth estimation network used in the application adopts a two-stage training strategy, the first stage reduces texture overfitting and learns the global structure of the scene through a feature alignment module, and the second stage balances the frequency domain features and improves the detail retention capability through a Fourier enhancement module, which has high efficiency and accuracy, and exhibits excellent generalization performance and depth detail capture capability in monocular depth estimation;

[0048] (3) The application uses the YOLOv8 model to realize the recognition and tracking of the landing mark, uses the DepthMaster to realize the estimation of depth information, and uses the PnP algorithm to realize the measurement of the position information of the landing mark, without the assistance of laser range finders, laser radars and the like, which can effectively reduce the cost;

[0049] (4) The application solves the problem of low precision and poor universality of unmanned aerial vehicle autonomous landing when relying only on monocular vision, and can meet the actual needs of precise landing of unmanned aerial vehicles, and has high practical value;

[0050] (5) The application uses infrared LEDs as landing marks, adopts a chessboard arrangement, and adds a band-pass filter in front of the camera, which can effectively reduce the influence of environmental light and increase the robustness. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not considered to be a limitation of the application.

[0052] Figure 1 is a schematic diagram of the overall process of landing guidance for the unmanned aerial vehicle in the embodiment of the application;

[0053] Figure 2 is a landing trajectory schematic diagram for the landing guidance of the unmanned aerial vehicle in the embodiment of the application;

[0054] Figure 3 is a schematic diagram of the overall framework of the DepthMaster model in the embodiment of the application;

[0055] Figure 4 is a schematic diagram of obtaining the pose information of the unmanned aerial vehicle using the PnP algorithm in the embodiment of the application. DETAILED DESCRIPTION

[0056] In order to enable more clear understanding of the above-mentioned objects, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. In addition, the present application can also be implemented in other ways different from those described herein, and therefore the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0057] One specific embodiment of the present application, as Figures 1-4 , discloses a UAV landing guidance method based on monocular depth estimation network, and the specific implementation steps are as follows:

[0058] Step S1, let i = 1, when i = 1, it means the first UAV;

[0059] Step S2, globally positioning the i-th UAV, and controlling the i-th UAV to be located above the landing mark in the landing area; obtaining the i-th UAV after global positioning;

[0060] Optionally, the specific steps of controlling the i-th UAV to be located above the landing mark in the landing area in step S2 include:

[0061] obtaining the global position information of the i-th UAV based on satellite map GPS; the global position information includes longitude and latitude and altitude;

[0062] obtaining the angular velocity and acceleration of the i-th UAV based on inertial measurement unit IMU;

[0063] based on the global position information of the i-th UAV, the angular velocity of the i-th UAV and the acceleration of the i-th UAV, the i-th UAV is navigated and positioned at a long distance, and the i-th UAV is controlled to be located above the landing mark in the landing area, so as to realize the global positioning of the i-th UAV;

[0064] Optionally, the global positioning is that the UAV is located at a position 60m above the landing mark;

[0065] Optionally, the landing mark is designed in the arrangement mode of a chessboard grid;

[0066] The landing mark is an infrared LED of a specific wavelength, and the wavelength is 940nm, which can effectively penetrate obstacles such as smoke and dust, and is suitable for monitoring in complex environments;

[0067] Optionally, the number of landing marks in the landing area is not less than 3.

[0068] Step S3, based on the monocular camera carried by the i-th UAV after global positioning, the corresponding original landing area image S i;

[0069] Step S4, let n = 1, when n = 1, it means the first landing mark;

[0070] Step S5, input the original landing area image S i Input YOLOv8 model A i , identify the n th landing mark Y i,n , adjust the position of the i th unmanned aerial vehicle after global positioning so that the landing mark Y i,n is in the center position of the landing area image, and obtain the i th position updated unmanned aerial vehicle Z i,n ;

[0071] Based on the i th position updated unmanned aerial vehicle Z i,n , obtain the landing area image S' i,n ;

[0072] Optionally, the monocular camera is installed in the form of an optoelectronic pod, which increases the field of view of the monocular camera and improves the stability of the monocular camera system.

[0073] A band-pass filter with a wavelength close to that of the landing mark is installed in front of the lens of the monocular camera, which reduces the influence of ambient light and makes it easier for the monocular camera to obtain information about the landing mark.

[0074] Step S6, input the landing area image S' i,n to the monocular depth estimation network B i,n , and obtain the depth estimation features of the i th position updated unmanned aerial vehicle Z i,n ;

[0075] Adjust the distance between the i th position updated unmanned aerial vehicle Z i,n and the landing area, complete the identification and positioning of the i th position updated unmanned aerial vehicle Z i,n and the landing area, and obtain the i th position second updated unmanned aerial vehicle Z' i,n ;

[0076] Optionally, the distance between the i th position updated unmanned aerial vehicle Z i,n and the landing area is adjusted, and the identification and positioning of the i th position updated unmanned aerial vehicle Z i,n and the landing area are completed when the distance between the unmanned aerial vehicle and the landing mark is within the range of 5-10 m.

[0077] Optionally, the specific steps of inputting the landing area image S' i,n to the monocular depth estimation network to obtain the depth estimation features of the i th position updated unmanned aerial vehicle Z i,n include:

[0078] Input the landing area image S' i,nInput monocular depth estimation network B i,n , using I2L encoder C i,n , the landing area image S' i,n is encoded into the latent space to obtain the latent state of the landing area image S' i,n ;

[0079] The latent state of the landing area image S' i,n is input into the U-Net model D i,n , and the corresponding image local feature F unet,i,n is extracted based on the intermediate layer;

[0080] The landing area image S' i,n is input into the external encoder DINOv2 E i,n , and the corresponding image external feature F ext,i,n is obtained;

[0081] The image local feature F unet,i,n is projected into the feature space of the image external feature F ext,i,n using a multi-layer perceptron, and the image local feature F unet,i,n is aligned with the image external feature F ext,i,n based on the feature alignment loss, to obtain the alignment feature of the landing area image S' i,n ;

[0082] The alignment feature of the landing area image S' i,n is input into the Fourier transform module F i,n , and the frequency domain feature of the landing area image S' i,n is obtained by fast Fourier transform to the frequency domain;

[0083] The frequency domain feature of the landing area image S' i,n is input into the modulator G i,n , and the balanced frequency domain feature of the landing area image S' i,n is obtained by balancing the information of different frequency bands;

[0084] The balanced frequency domain feature of the landing area image S' i,n is transformed back to the spatial domain using inverse fast Fourier transform to obtain the optimized landing area image F f,i,n ;

[0085] The latent state F i,n of the landing area image S' mid,i,n is input into the spatial channel H i,n , and the spatial feature F s,i,n of the landing area image is obtained by twice convolution transformation;

[0086] The optimized landing area image F f,i,nspatial features F of the landing area image s,i,n performing a cascading operation, performing feature complementation, obtaining the landing area image S' i,n complemented features

[0087] using an I2L decoder J i,n the landing area image S' i,n complementing the features, and decoding to obtain the landing area image S' i,n depth estimation features of the i-th position updating UAV Z i,n depth estimation features of the i-th position updating UAV Z

[0088] Optionally, the monocular depth estimation network in step S6 is a DepthMaster model.

[0089] Further, the monocular depth estimation network comprises an I2L encoder, a U-Net model, a feature alignment module, a Fourier enhancement module and an I2L decoder.

[0090] The Fourier enhancement module comprises a spatial channel and a frequency channel.

[0091] Optionally, the image local features F unet,i,n are projected to the feature space of the image external features F ext,i,n The expression of the transformation relationship is as follows:

[0092]

[0093] wherein, is the feature of the i-th position updating UAV after the n-th landing mark image local feature is projected, h φ is the expression of the projection process.

[0094] Optionally, the expression of the feature alignment loss is as follows:

[0095]

[0096] wherein, L fa is the feature alignment loss, and dist(·) represents the distance between two feature distributions.

[0097] The optimized landing area image F f,i,n spatial features F of the landing area image s,i,n performing a cascading operation, so that the DepthMaster model adaptively balances the low-frequency structural features and the high-frequency detailed features in a single forward channel, and effectively improves the visual quality of depth prediction.

[0098] The monocular depth estimation network DepthMaster model in the application is a single-step diffusion model, which reduces texture overfitting through a feature alignment module, balances frequency domain features through a Fourier enhancement module, and realizes depth estimation of each pixel point in an image in combination with a two-stage training strategy, as shown in Figure 3 .

[0099] Step S7, updating the pose information T i,n of the i-th unmanned aerial vehicle based on the landing area image S i,n ; i,n Step S7, updating the pose information T i,n of the i-th unmanned aerial vehicle based on the landing area image S i,n ;

[0100] Optionally, the specific steps of obtaining the pose information T i,n of the i-th unmanned aerial vehicle in step S7 include:

[0101] Step S71, obtaining the pixel coordinates and three-dimensional coordinates of each landing marker in the updated landing area image S i,n ;

[0102] Step S71, obtaining the conversion relationship between the pixel coordinates and three-dimensional coordinates of each landing marker in the updated landing area image S i,n ;

[0103] s i,n p i,n

[0104] wherein s i,m is a scale factor of the n-th landing marker of the i-th unmanned aerial vehicle, i = 1, 2, 3,..., I, P i,n is the projection pixel coordinates of the n-th landing marker of the i-th unmanned aerial vehicle, P i,n = [u i,n , v i,n ] T , u i,n is the pixel horizontal coordinate of the n-th landing marker of the i-th unmanned aerial vehicle, v i,n is the pixel vertical coordinate of the n-th landing marker of the i-th unmanned aerial vehicle, K is the intrinsic matrix of the camera, T is the Lie group representation of the camera pose R and the translation vector t, i.e. the conversion relationship between the pixel coordinates and three-dimensional coordinates of the landing marker, P i,n is the three-dimensional coordinates of the n-th landing marker of the i-th unmanned aerial vehicle, P i,n = [X i,n , Y i,n , Z i,n ] T , X i,n is the horizontal coordinate of the n-th landing marker of the i-th unmanned aerial vehicle, Y is the vertical coordinate of the n-th landing marker of the i-th unmanned aerial vehicle, and Z is the depth coordinate of the n-th landing marker of the i-th unmanned aerial vehicle.X-axis coordinate of the n th landing mark of the i th UAV, Y i,n Y-axis coordinate of the n th landing mark of the i th UAV, Z i,n Z-axis coordinate of the n th landing mark of the i th UAV.

[0105] Step S72, constructing an updated landing area image S" i The least square problem related to the conversion relationship between the pixel coordinates and the three-dimensional coordinates of each landing mark is expressed as:

[0106]

[0107] wherein E is the least square error value, The corresponding T value is obtained according to the least square error value of the conversion relationship between the pixel coordinates and the three-dimensional coordinates of the landing mark.

[0108] Step S73, minimizing the updated landing area image S" i,n The least square problem related to the conversion relationship between the pixel coordinates and the three-dimensional coordinates of each landing mark is expressed as: i,n

[0109] Optionally, the position and posture information T i,n of the i th position second-updated UAV includes a rotation matrix R i,n and a translation vector t i,n of the i th position second-updated UAV.

[0110] It can be understood that the rotation matrix R i,n and the translation vector t i,n represent the attitude and position of the i th position second-updated UAV relative to the landing mark; wherein the rotation matrix R i,n provides the attitude information of the i th position second-updated UAV, including the pitch angle, the yaw angle and the roll angle; the translation vector t i,n provides the position information (Tx, Ty, Tz) of the i th position second-updated UAV relative to the landing mark. As Figure 4 shown;

[0111] Step S8, based on the position and posture information T i,n of the i th position second-updated UAV and the depth estimation feature of the i th position-updated UAV Z i,n , the position of the i th position-updated UAV Z i,n is adjusted to complete the accurate positioning of the i th position-updated UAV Z i,n to the n th landing mark, that is, the landing guidance of the i th UAV to the n th landing mark is completed.​

[0112] Step S9, judging whether n is greater than or equal to N, N represents the total number of landing marks, if yes, completing the landing guidance of the ith unmanned aerial vehicle to each landing mark, if no, n=n+1, returning to step S4;

[0113] Step S10, judging whether i is greater than or equal to I, I represents the total number of unmanned aerial vehicles in the training set, if yes, completing the accurate positioning of the ith unmanned aerial vehicle to each landing mark, obtaining the final monocular depth estimation network; if no, i=i+1, returning to step S2;

[0114] Step S11, landing guidance is performed on the unmanned aerial vehicle based on the final monocular depth estimation network.

[0115] The above merely describes the preferred embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A landing guidance method for unmanned aerial vehicles (UAVs) based on a monocular depth estimation network, characterized in that, include: Step S1: Let i = 1. When i = 1, it represents the first drone. Step S2: Perform global positioning on the i-th UAV to obtain the i-th UAV after global positioning; Step S3: Identify the original landing area image S corresponding to the i-th UAV after global positioning. i ; Step S4: Let n = 1. When n = 1, it represents the first landing mark. Step S5: Transfer the original landing area image S i Input YOLOv8 model A i Identify the nth landing marker Y i,n The position of the i-th UAV after global localization is adjusted to obtain the position-updated position of the i-th UAV Z. i,n ; Based on the position update of the i-th drone Z i,n Obtain landing area image S′ i,n ; Step S6: Transfer the landing area image S′ i,n Input monocular depth estimation network B i,n Obtain the location update of the i-th drone Z i,n Depth estimation features; Adjust the position of the i-th drone and update drone Z. i,n The distance from the landing area is used to obtain the second update of the position of the i-th UAV Z′. i,n ; Step S7: Update UAV Z′ based on the position of the i-th UAV for the second time. i,n The pose information T of the i-th UAV with its position updated twice is obtained. i,n ; Step S8: Use the pose information T of the UAV updated twice using the position of the i-th UAV. i,n And the location update drone Z of the i-th unit i,n The depth estimation features are used to update the position of the i-th UAV Z. i,n Perform position adjustments and complete the position update for the i-th drone Z. i,n Location of the nth landing marker; Step S9: Determine whether n is greater than or equal to N, where N represents the total number of landing markers. If yes, complete the positioning of each landing marker by the i-th UAV. If no, let n = n + 1 and return to step S5. Step S10: Determine whether i is greater than or equal to I, where I represents the total number of UAVs in the training set. If yes, complete the localization of each UAV to each landing mark and obtain the final monocular depth estimation network; if no, let i = i + 1 and return to step S2. Step S11: Guide the UAV to land based on the final monocular depth estimation network.

2. The UAV landing guidance method based on a monocular depth estimation network according to claim 1, characterized in that, The landing marker is an infrared LED.

3. The UAV landing guidance method based on a monocular depth estimation network according to claim 1, characterized in that, Step S6 describes obtaining the position update of the i-th UAV Z. i,n The specific steps for depth estimation features include: Landing area image S′ i,n Input monocular depth estimation network B i,n For the landing area image S′ i,n Perform latent space encoding, then input it into the U-Net model D. i,n Extract the corresponding local image features F unet,i,n ; Extracting the landing area image S′ i,n Image external features F ext,i,n ; Based on feature alignment loss, local image features F unet,i,n With image external features F ext,i,n Feature alignment is performed to obtain the landing area image S′ i,n Alignment features; Landing area image S′ i,n Alignment feature input Fourier transform module F i,n In the process, the landing area image S′ is obtained. i,n Frequency domain characteristics; Landing area image S′ i,n Frequency domain characteristics of input modulator G i,n The landing area image S′ is obtained. i,n Frequency domain characteristics after balance; The landing area image S′ is obtained using the inverse fast Fourier transform. i,n The balanced frequency domain features are transformed back into the spatial domain to obtain the optimized landing area image F. f,i,n ; Acquire spatial features F of the landing area image s,i,n ; The optimized landing area image F f,i,n Spatial features F of the landing area image s,i,n Perform cascaded operations and feature complementation to obtain the landing area image S′. i,n Features after complementarity; Landing area image S′ i,n The complementary features are decoded to obtain the position update UAV Z of the i-th UAV. i,n Depth estimation features.

4. The UAV landing guidance method based on a monocular depth estimation network according to claim 1, characterized in that, The expression for the feature alignment loss is: Among them, L fa For feature alignment loss, dist(.) represents the distance between two feature distributions. To update the projected features of the local features of the nth landing marker image for the i-th UAV position, F ext,i,n Update the external features of the image of the nth landing marker for the i-th drone position.

5. The UAV landing guidance method based on a monocular depth estimation network according to claim 1, characterized in that, The specific steps for obtaining the pose information of the i-th UAV with its position updated twice, as described in step S7, include: Step S71: Update UAV Z′ based on the position of the i-th UAV for the second time. i,n Obtain updated landing area image S″ i,n ; Obtain updated landing area image S″ i,n The pixel coordinates and 3D coordinates of each landing marker; Obtain updated landing area image S″ i,n The transformation relationship between the pixel coordinates and 3D coordinates of each landing marker; Step S72: Construct and update the landing area image S″ i The least squares problem related to the transformation relationship between the pixel coordinates and 3D coordinates of each landing marker; Step S73: Minimize and update the landing area image S″ i,n The least squares problem related to the transformation relationship between the pixel coordinates and 3D coordinates of each landing marker is used to obtain the pose information T of the i-th UAV with a second position update. i,n .

6. The UAV landing guidance method based on a monocular depth estimation network according to claim 5, characterized in that, The updated landing area image S″ i The least squares problem relating the transformation relationship between the pixel coordinates and 3D coordinates of each landing marker is expressed as follows: Where E is the least squares error value. This indicates that the T value is obtained by using the least squares error value based on the transformation relationship between the pixel coordinates and 3D coordinates of the landing mark. i,n Let P be the scale factor of the nth landing mark of the i-th UAV, where i = 1, 2, 3, ..., I. i,n Let be the projected pixel coordinates of the nth landing mark of the i-th UAV, K be the intrinsic parameter matrix of the camera, and T be the Lie group representation of the camera pose R and the translation vector t.

7. The UAV landing guidance method based on a monocular depth estimation network according to claim 1, characterized in that, The pose information T of the UAV whose position is updated for the second time in step S7. i,n This includes the rotation matrix of the UAV with the second-order position update and the translation vector of the UAV with the second-order position update of the i-th UAV.

Images