Unmanned aerial vehicle fixed-point homeward voyage method, device and equipment facing mobile platform and medium

Abstract

The invention discloses an unmanned aerial vehicle fixed-point homeward voyage method, device and equipment for a mobile platform and a medium, and the method comprises the steps: obtaining the position of an unmanned aerial vehicle and the position of the mobile platform at the current time through responding to a fixed-point homeward voyage instruction; controlling the unmanned aerial vehicle to arrive at a first middle position based on the unmanned aerial vehicle position and the mobile platform position, and determining the time when the unmanned aerial vehicle arrives at the first middle position as a first moment; determining the position of the first identification image based on the platform image data at the first moment; controlling the unmanned aerial vehicle to descend to a second middle position based on the first middle position and the position of the first identification image, and determining the time when the unmanned aerial vehicle descends to the second middle position as a second moment; obtaining platform image data at a second moment, and determining the position of a second identification image based on the platform image data at the second moment; and controlling the unmanned aerial vehicle to land at the target landing position based on the second middle position and the position of the second identification image. The problem of homeward voyage deviation caused by inaccurate homeward voyage path in homeward voyage of the unmanned aerial vehicle is solved.

Description

Technical Field

[0001] The present application relates to the field of UAV technology, and in particular to a method, device, equipment and medium for a UAV fixed-point return for a mobile platform. Background Art

[0002] In the field of drones, most drones are equipped with a return function. After the drone leaves the take-off point, the return mission is triggered manually or automatically, so that the drone returns to the user's desired landing point. In actual applications, the landing point of a drone includes not only a stationary ground or platform, but also a mobile platform.

[0003] The use of a mobile platform can effectively improve the maneuverability of the drone during operation. Since the mobile platform is in motion, the drone needs to be guided to track and finally land on the mobile platform. In the prior art, the location information of the mobile platform and the drone is usually used to plan the return path, and the drone is controlled to return according to the return path. However, the accuracy of the observation of the surrounding environment of this solution is not high. In the case of weak positioning signals, it is easy to cause the drone to be unable to accurately locate, and there is a problem of return deviation caused by inaccurate return path. Summary of the invention

[0004] In order to solve the problems of the prior art, the present application provides a fixed-point return method for a UAV on a mobile platform. The target landing position is determined based on computer vision technology, and the UAV is controlled to return according to the positional relationship between the UAV and the landed mobile platform, so that the UAV lands at the target landing position. By improving the accuracy of positioning the target landing position and coordinating corresponding control methods, the accuracy of the UAV's return is improved, thereby solving the problem of return deviation caused by inaccurate return path during the return of the UAV.

[0005] According to one aspect of the present application, a method for returning to a fixed point by a drone facing a mobile platform is provided, wherein a target landing position of the drone is set on the mobile platform, a first identification image and a second identification image are set at the target landing position, and a size of the second identification image is smaller than a size of the first identification image, and the method comprises:

[0006] In response to a fixed-point return command to the drone, obtain the drone position and the mobile platform position at the current time;

[0007] Based on the position of the drone and the position of the mobile platform, the drone is controlled to arrive at a first intermediate position, and the time when the drone arrives at the first intermediate position is determined as a first moment; the first intermediate position is a position where the horizontal distance from the mobile platform meets a first preset distance condition;

[0008] Acquire platform image data at the first moment, and determine the position of the first identification image based on the platform image data at the first moment;

[0009] Based on the first intermediate position and the position of the first identification image, the drone is controlled to descend to a second intermediate position, and the time when the drone descends to the second intermediate position is determined as a second moment; the second intermediate position is a position where the vertical distance with the mobile platform meets a second preset distance condition;

[0010] Acquire platform image data at the second moment, and determine the position of the second identification image based on the platform image data at the second moment;

[0011] The drone is controlled to land at the target landing position based on the second intermediate position and the position of the second identification image.

[0012] In a possible implementation, controlling the drone to arrive at the first intermediate position based on the drone position and the mobile platform position includes:

[0013] Acquire platform movement information of the mobile platform at a current moment; the platform movement information represents movement parameters of the mobile platform;

[0014] Determining a first operating parameter of the drone based on the drone position, the mobile platform position, and the platform movement information;

[0015] Control the UAV to move based on the first operating parameter to obtain a first movement result;

[0016] When the first movement result indicates that the horizontal distance between the drone and the mobile platform satisfies the first preset distance condition, it is determined that the drone has arrived at the first intermediate position.

[0017] In a possible implementation, the first movement result includes a first position of a drone and a first position of a mobile platform, the first position of the drone represents position information of the drone corresponding to the first movement result, and the first position of the mobile platform represents position information of the mobile platform corresponding to the first movement result.

[0018] The method further comprises:

[0019] Calculating the distance between the first position of the UAV and the first position of the mobile platform in the horizontal direction to obtain a horizontal distance;

[0020] When the horizontal distance is less than a preset horizontal distance threshold, it is determined that the distance between the UAV and the mobile platform meets the first preset distance condition.

[0021] In a possible implementation manner, determining the position of the first identification image based on the platform image data at the first moment includes:

[0022] Obtaining a target identification network; the target identification network is constructed based on a self-attention mechanism;

[0023] Inputting the platform image data at the first moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the first moment to obtain a first identification recognition result;

[0024] Coordinate transformation is performed based on the first marker recognition result to obtain the position of the first marker image.

[0025] In a possible implementation, the method for training the target identification network includes:

[0026] Obtaining an initial identification network; the initial identification network is constructed based on an attention mechanism and a U-shaped network;

[0027] Acquire sample platform image data annotated with a true value label; the sample platform image data is obtained by collecting the first identification image and the second identification image by drones at different locations in different scenes, and the true value label represents the position of at least one of the first identification image and the second identification image;

[0028] Inputting the sample platform image data into the initial identification network to obtain a prediction result;

[0029] Determining a network loss value based on difference information between the true value label of the sample platform image data and the prediction result;

[0030] When the network loss value does not satisfy the preset loss condition, the network parameters of the initial identification network are updated based on the network loss value until the network loss value satisfies the preset loss condition, and the target identification network is determined to be obtained.

[0031] In a possible implementation, the controlling the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image includes:

[0032] Determining a second operating parameter of the drone based on the first intermediate position and the position of the first identification image;

[0033] Control the UAV to move based on the second operating parameter to obtain a second movement result;

[0034] When the second movement result indicates that the vertical distance between the UAV and the mobile platform satisfies the second preset distance condition, it is determined that the UAV descends to the second intermediate position.

[0035] In a possible implementation, the second movement result includes a second position of the drone and a second position of the mobile platform, the second position of the drone represents position information of the drone corresponding to the second movement result, and the second position of the mobile platform represents position information of the mobile platform corresponding to the second movement result.

[0036] The method further comprises:

[0037] Calculating the distance between the second position of the UAV and the second position of the mobile platform in the vertical direction to obtain a vertical distance;

[0038] When the vertical distance is less than a preset vertical distance threshold, it is determined that the distance between the UAV and the mobile platform meets the second preset distance condition.

[0039] In a possible implementation manner, determining the position of the second identification image based on the platform image data at the second moment includes:

[0040] Acquire the target identification network;

[0041] Inputting the platform image data at the second moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the second moment to obtain a second identification recognition result;

[0042] Coordinate transformation is performed based on the second marker recognition result to obtain the position of the second marker image.

[0043] In a possible implementation, the controlling the drone to land at the target landing position based on the second intermediate position and the position of the second identification image includes:

[0044] determining a third operating parameter of the UAV based on the second intermediate position and the position of the second identification image;

[0045] Controlling the UAV to move based on the third operating parameter to obtain the third movement result;

[0046] When the third movement result indicates that the distance between the UAV and the target landing position satisfies a third preset distance condition, the UAV is controlled to land at the target landing position.

[0047] In a possible implementation, the third movement result includes a third position of the drone and a third position of the mobile platform, wherein the third position of the drone represents position information of the drone corresponding to the third movement result, and the third position of the mobile platform represents position information of the target landing position corresponding to the third movement result.

[0048] The method further comprises:

[0049] Calculating the distance between the third position of the UAV and the third position of the mobile platform to obtain a landing distance;

[0050] When the landing distance is less than a preset landing distance threshold, the UAV is controlled to land at the target landing position.

[0051] On the other hand, a fixed-point return device for a UAV facing a mobile platform is provided, wherein a target landing position of the UAV is set on the mobile platform, a first identification image and a second identification image are set at the target landing position, and a size of the second identification image is smaller than a size of the first identification image, and the device comprises:

[0052] An information acquisition module is used to obtain the current position of the drone and the mobile platform in response to a fixed-point return command to the drone;

[0053] A first control module is used to control the drone to arrive at a first intermediate position based on the position of the drone and the position of the mobile platform, and determine the time when the drone arrives at the first intermediate position as a first moment; the first intermediate position is a position where the horizontal distance from the mobile platform meets a first preset distance condition;

[0054] a first identification position determination module, configured to obtain the platform image data at the first moment, and determine the position of the first identification image based on the platform image data at the first moment;

[0055] a second control module, configured to control the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image, and determine the time when the drone descends to the second intermediate position as a second moment; the second intermediate position is a position where the vertical distance from the mobile platform meets a second preset distance condition;

[0056] a second identification position determination module, configured to obtain the platform image data at the second moment, and determine the position of the second identification image based on the platform image data at the second moment;

[0057] A third control module is used to control the UAV to land at the target landing position based on the second intermediate position and the position of the second identification image.

[0058] In a possible implementation, the first control module is configured to:

[0059] Acquire platform movement information of the mobile platform at a current moment; the platform movement information represents movement parameters of the mobile platform;

[0060] Determining a first operating parameter of the drone based on the drone position, the mobile platform position, and the platform movement information;

[0061] Control the UAV to move based on the first operating parameter to obtain a first movement result;

[0062] When the first movement result indicates that the horizontal distance between the drone and the mobile platform satisfies the first preset distance condition, it is determined that the drone has arrived at the first intermediate position.

[0063] In a possible implementation, the first movement result includes a first position of a drone and a first position of a mobile platform, the first position of the drone represents position information of the drone corresponding to the first movement result, and the first position of the mobile platform represents position information of the mobile platform corresponding to the first movement result.

[0064] The first control module is further used for:

[0065] Calculating the distance between the first position of the UAV and the first position of the mobile platform in the horizontal direction to obtain a horizontal distance;

[0066] When the horizontal distance is less than a preset horizontal distance threshold, it is determined that the distance between the UAV and the mobile platform meets the first preset distance condition.

[0067] In a possible implementation manner, the first identification position determination module is configured to:

[0068] Obtaining a target identification network; the target identification network is constructed based on a self-attention mechanism;

[0069] Inputting the platform image data at the first moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the first moment to obtain a first identification recognition result;

[0070] Coordinate transformation is performed based on the first marker recognition result to obtain the position of the first marker image.

[0071] In a possible implementation manner, the device further includes a network training module, and the network training module is used to:

[0072] Obtaining an initial identification network; the initial identification network is constructed based on an attention mechanism and a U-shaped network;

[0073] Acquire sample platform image data annotated with a true value label; the sample platform image data is obtained by collecting the first identification image and the second identification image by drones at different locations in different scenes, and the true value label represents the position of at least one of the first identification image and the second identification image;

[0074] Inputting the sample platform image data into the initial identification network to obtain a prediction result;

[0075] Determining a network loss value based on difference information between the true value label of the sample platform image data and the prediction result;

[0076] When the network loss value does not satisfy the preset loss condition, the network parameters of the initial identification network are updated based on the network loss value until the network loss value satisfies the preset loss condition, and the target identification network is determined to be obtained.

[0077] In a possible implementation, the second control module is configured to:

[0078] Determining a second operating parameter of the drone based on the first intermediate position and the position of the first identification image;

[0079] Control the UAV to move based on the second operating parameter to obtain a second movement result;

[0080] When the second movement result indicates that the vertical distance between the UAV and the mobile platform satisfies the second preset distance condition, it is determined that the UAV descends to the second intermediate position.

[0081] In a possible implementation, the second movement result includes a second position of the drone and a second position of the mobile platform, the second position of the drone represents position information of the drone corresponding to the second movement result, and the second position of the mobile platform represents position information of the mobile platform corresponding to the second movement result.

[0082] The second control module is further used for:

[0083] Calculating the distance between the second position of the UAV and the second position of the mobile platform in the vertical direction to obtain a vertical distance;

[0084] When the vertical distance is less than a preset vertical distance threshold, it is determined that the distance between the UAV and the mobile platform meets the second preset distance condition.

[0085] In a possible implementation manner, the second identification position determination module is configured to:

[0086] Acquire the target identification network;

[0087] Inputting the platform image data at the second moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the second moment to obtain a second identification recognition result;

[0088] Coordinate transformation is performed based on the second identification recognition result to obtain the position of the second identification image.

[0089] In a possible implementation, the third control module is used to:

[0090] determining a third operating parameter of the UAV based on the second intermediate position and the position of the second identification image;

[0091] Controlling the UAV to move based on the third operating parameter to obtain the third movement result;

[0092] When the third movement result indicates that the distance between the UAV and the target landing position satisfies a third preset distance condition, the UAV is controlled to land at the target landing position.

[0093] In a possible implementation, the third movement result includes a third position of the drone and a third position of the mobile platform, wherein the third position of the drone represents position information of the drone corresponding to the third movement result, and the third position of the mobile platform represents position information of the target landing position corresponding to the third movement result.

[0094] The third control module is further used for:

[0095] Calculating the distance between the third position of the UAV and the third position of the mobile platform to obtain a landing distance;

[0096] When the landing distance is less than a preset landing distance threshold, the UAV is controlled to land at the target landing position.

[0097] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the mobile platform-oriented drone fixed-point return method of any of the above aspects.

[0098] On the other hand, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the fixed-point return method of a drone for a mobile platform as described in any of the above aspects.

[0099] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for returning to a fixed point of a UAV for a mobile platform according to any one of the above aspects.

[0100] The embodiment of the present application obtains the current drone position and mobile platform position in response to a fixed-point return command to the drone; controls the drone to arrive at a first intermediate position based on the drone position and the mobile platform position, and determines the time when the drone arrives at the first intermediate position as a first moment; the first intermediate position is a position where the horizontal distance between the drone and the mobile platform meets a first preset distance condition; obtains platform image data at the first moment, and determines the position of the first identification image based on the platform image data at the first moment; controls the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image, and determines the time when the drone descends to the second intermediate position as a second moment; the second intermediate position is a position where the vertical distance between the drone and the mobile platform meets a second preset distance condition; obtains platform image data at the second moment, and determines the position of the second identification image based on the platform image data at the second moment; and controls the drone to land at the target landing position based on the second intermediate position and the position of the second identification image. The target landing position is determined based on computer vision technology, and the return of the UAV is controlled according to the positional relationship between the UAV and the landing mobile platform, so that the UAV lands at the target landing position. By improving the accuracy of positioning the target landing position and coordinating corresponding control methods, the accuracy of the UAV's return is improved, solving the problem of return deviation caused by inaccurate return path during the return of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0102] Figure 1It is a flow chart of a method for returning to a fixed point of a UAV on a mobile platform provided in an embodiment of the present application;

[0103] Figure 2 is a schematic diagram of a method for arranging a first identification image and a second identification image provided in an embodiment of the present application;

[0104] Figure 3 is a schematic diagram of a first identification image at a first distance provided in an embodiment of the present application;

[0105] Figure 4 is a schematic diagram of a second identification image at a second distance provided in an embodiment of the present application;

[0106] Figure 5 This is a flowchart of a fixed-point landing process of a drone provided in an embodiment of the present application;

[0107] Figure 6 is a schematic diagram of the relationship between a camera coordinate system and a world coordinate system provided in an embodiment of the present application;

[0108] Figure 7 is a schematic diagram of the relationship between an image coordinate system and a camera coordinate system provided in an embodiment of the present application;

[0109] Figure 8 is a schematic diagram of the relationship between a pixel coordinate system and an image coordinate system provided in an embodiment of the present application;

[0110] Fig. 9 It is a flowchart of a method for obtaining posture information provided in an embodiment of the present application;

[0111] Fig.10 is a structural schematic diagram of an image preprocessing network provided in an embodiment of the present application;

[0112] Fig.11 It is a structural diagram of a self-attention mechanism provided in an embodiment of the present application;

[0113] Fig.12 It is the first labeled sample Apriltag code data provided in the embodiment of the present application;

[0114] Fig.13 It is the second labeled sample Apriltag code data provided in the embodiment of the present application;

[0115] Fig.14 It is the third labeled sample Apriltag code data provided in the embodiment of the present application;

[0116] Fig.15 It is a structural block diagram of a UAV fixed-point return device for a mobile platform provided in an embodiment of the present application. DETAILED DESCRIPTION

[0117] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0118] There are special places where drones cannot communicate well with satellites, such as inside mountains and on the sea. Mobile plane landing missions are less common but also very important, such as landing on moving aircraft carriers, warships, ground armored vehicles, etc.

[0119] This solution uses visual navigation to land the drone on a mobile platform, which can expand the application range of drones and reduce the requirements of the drone's use environment. When the drone is in low light at night or on a cloudy day, the computer vision technology is used to detect the target landing position and assist the drone to autonomously land on a stationary / mobile platform.

[0120] The embodiment of the present application provides a method for returning a drone to a fixed point on a mobile platform. The mobile platform is provided with a target landing position for the drone. A first identification image and a second identification image are provided at the target landing position. The size of the second identification image is smaller than that of the first identification image. The target landing position is determined based on computer vision technology. According to the positional relationship between the drone and the mobile platform on which it lands, the drone is controlled to return and land at the target landing position. Figure 1 The method for returning to a fixed point by a UAV for a mobile platform includes steps S101 to S1011.

[0121] In a possible implementation manner, the mobile platform may be a vehicle or a ship.

[0122] In a possible implementation, the first identification image and the second identification image are preset AprilTag codes of different scales. Specifically, the AprilTag code is a positioning two-dimensional code used for robot visual navigation and target recognition, which is used to mark the target position. Combined with the target detection algorithm, the AprilTag code can be accurately detected and recognized under various lighting conditions.

[0123] In a possible implementation, in order to ensure that the drone can detect the April tag code within the range of each stage of fixed-point landing, and to ensure that it can be applied to more landing scenarios, the first identification image and the second identification image are stacked and arranged, such as Figure 2As shown, the second logo image is stacked at a preset stacking position of the first logo image.

[0124] In a possible implementation manner, the first identification image and the second identification image are arranged in sequence.

[0125] In a possible implementation, identification images of other scales may be provided at the target landing position, and the first identification image, the second representation image and identification images of other scales may be combined and arranged in a stacked arrangement, a sequential arrangement or other arrangement manners.

[0126] In one possible implementation, Figure 3 and Figure 4 As shown, during the landing process of the drone, the distance between the drone and the mobile platform increases from far to near. The first identification image is detected in the long-distance stage, and the second representation image stacked on the first identification image is detected in the close-distance stage, ensuring that the drone can detect the corner point information of the Apriltag code in each distance stage, and the camera of the drone can detect the ID information carried by the Apriltag code.

[0127] In a possible implementation manner, the ID information carried by the Apriltag code also includes the number information of the Apriltag code.

[0128] In a possible implementation manner, the ID information carried by the Apriltag code of the first identification image includes serial number information of the first identification image.

[0129] In a possible implementation manner, the ID information carried by the Apriltag code of the second identification image includes serial number information of the second identification image.

[0130] In a possible implementation, the mobile platform to be landed can be further determined based on the number information of the Apriltag code.

[0131] In step S101, in response to a fixed-point return instruction to the drone, the drone position and the mobile platform position at the current time are obtained.

[0132] In a possible implementation, the fixed-point return instruction represents a return mission, and the return mission is triggered manually or automatically.

[0133] In one possible implementation, the triggering conditions for the return mission may be low battery, requiring return for charging; bad weather, requiring return for avoidance; or returning after completing the mission, etc.

[0134] In a possible implementation, the object sends the fixed-point return instruction to the drone to instruct the drone to perform a return mission.

[0135] In a possible implementation, the fixed-point return instruction further indicates instructing the UAV to land at the target landing position.

[0136] In a possible implementation manner, the drone position at the current time represents the position information of the drone at the current time.

[0137] In a possible implementation, the current position of the drone is determined by a positioning module of the drone. Specifically, the positioning module includes a positioning system, which may be one of the Beidou Navigation Satellite System (BDS), the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), and the Galileo satellite navigation system (Galileo).

[0138] In a possible implementation manner, the position of the mobile platform at the current time represents the position information of the mobile platform at the current time.

[0139] In a possible implementation, the mobile platform is a vehicle, and the position of the mobile platform at the current time is determined by a positioning module of the vehicle. Specifically, the positioning module of the vehicle includes a positioning system.

[0140] In this implementation, in response to a fixed-point return command to the drone, the drone position and mobile platform position at the current time are obtained, which facilitates subsequent position relationship judgment based on the drone position and mobile platform position at the current time, thereby improving calculation efficiency.

[0141] In step S103, the drone is controlled to arrive at a first intermediate position based on the position of the drone and the position of the mobile platform, and the time when the drone arrives at the first intermediate position is determined as a first moment; the first intermediate position is a position where the horizontal distance between the drone and the mobile platform meets a first preset distance condition.

[0142] In a possible implementation, controlling the drone to arrive at the first intermediate position based on the drone position and the mobile platform position includes:

[0143] Acquire platform movement information of the mobile platform at a current moment; the platform movement information represents movement parameters of the mobile platform;

[0144] Determining a first operating parameter of the drone based on the drone position, the mobile platform position, and the platform movement information;

[0145] Control the UAV to move based on the first operating parameter to obtain a first movement result;

[0146] When the first movement result indicates that the horizontal distance between the drone and the mobile platform satisfies the first preset distance condition, it is determined that the drone has arrived at the first intermediate position.

[0147] In a possible implementation, the first movement result includes a first position of a drone and a first position of a mobile platform, the first position of the drone represents position information of the drone corresponding to the first movement result, and the first position of the mobile platform represents position information of the mobile platform corresponding to the first movement result.

[0148] The method further comprises:

[0149] Calculating the distance between the first position of the UAV and the first position of the mobile platform in the horizontal direction to obtain a horizontal distance;

[0150] When the horizontal distance is less than a preset horizontal distance threshold, it is determined that the distance between the UAV and the mobile platform meets the first preset distance condition.

[0151] In a possible implementation, when the mobile platform is a moving vehicle, the platform movement information of the mobile platform includes speed information of the vehicle.

[0152] In one possible implementation, Figure 5 As shown, the fixed-point landing process of the UAV is divided into three stages: the first stage is the initial descent, the second stage is approaching the platform, and the third stage is landing.

[0153] In one possible implementation, Figure 5 As shown, the initial descent process of the first stage needs to achieve rapid descent and maintain a stable height. Based on the position of the drone, the position of the mobile platform and the platform movement information, the control information of the drone moving to the first intermediate position is determined, and the control information of the first stage is executed by combining GPS and inertial navigation to help the drone find the place where it needs to land, and follow the mobile platform to maintain a certain relative speed. In order to achieve rapid descent, until the drone's onboard camera recognizes the first identification image, the first stage is completed.

[0154] In a possible implementation, after a first preset period of movement, the distance in the horizontal direction between the first position of the UAV and the first position of the mobile platform is calculated to obtain a horizontal distance; when the horizontal distance is less than a preset horizontal distance threshold, it is determined that the distance between the UAV and the mobile platform meets the first preset distance condition.

[0155] In a possible implementation, as the drone approaches the first intermediate position, it decelerates and maintains a preset height to follow the mobile platform until the onboard camera recognizes the first identification image.

[0156] In a possible implementation, the preset horizontal distance threshold is set based on the size of the target landing position so that the UAV is located in an area above the target landing position.

[0157] In a possible implementation, the preset horizontal distance threshold is set based on the center position of the target landing position, and the horizontal distance between the center position of the target landing position and the edge is determined to be the preset horizontal distance threshold. When the horizontal distance between the UAV and the center position of the target landing position is less than the preset horizontal distance threshold, it is determined that the UAV is in the first intermediate position.

[0158] In this implementation, the drone is controlled to arrive at a first intermediate position based on the drone position and the mobile platform position, the time when the drone arrives at the first intermediate position is determined as the first moment, and the drone is controlled to fly to directly above the mobile platform based on the drone position and the mobile platform position, completing the first stage of the control task, making it easier for the drone to identify markers on the mobile platform, thereby improving the accuracy of drone control.

[0159] In step S105, the platform image data at the first moment is acquired, and the position of the first identification image is determined based on the platform image data at the first moment.

[0160] In a possible implementation, the platform image data is obtained by the drone collecting images of the mobile platform. Specifically, the image data of the mobile platform is collected by an onboard camera of the drone as the platform image data.

[0161] In a possible implementation manner, determining the position of the first identification image based on the platform image data at the first moment includes:

[0162] Obtaining a target identification network; the target identification network is constructed based on a self-attention mechanism;

[0163] Inputting the platform image data at the first moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the first moment to obtain a first identification recognition result;

[0164] Coordinate transformation is performed based on the first marker recognition result to obtain the position of the first marker image.

[0165] In a possible implementation, the platform image data at the first moment is an image obtained by capturing the first identification image by an onboard camera of the drone.

[0166] In a possible implementation, during the movement of the UAV, platform image data is continuously collected, and the collected platform image data is input into the target identification network in real time, so that the target identification network can perform real-time identification to improve the accuracy of identification.

[0167] In a possible implementation, the input of the target identification network is the platform image data, and the output is the ID of the platform image data, and the position coordinates of the four corner points and the center point.

[0168] In a possible implementation, the first identification result is the ID of the platform image data, and the position coordinates of four corner points and a center point.

[0169] In a possible implementation, coordinate transformation is performed based on the first identification recognition result to obtain the position of the first identification image, which is to transform the position coordinates of the four corner points and the center point of the platform image data from the pixel coordinate system to the world coordinate system.

[0170] In a possible implementation, the conversion from the pixel coordinate system to the world coordinate system requires conversion from the pixel coordinate system to the image coordinate system, from the image coordinate system to the camera coordinate system, and from the camera coordinate system to the world coordinate system.

[0171] In one possible implementation, the world coordinate system (Xw, Yw, Zw) is a global reference system that describes the position of a target object in three-dimensional space. Unlike the camera and image coordinate systems, the world coordinate system can accurately describe the position and posture of an object in real space without being affected by the camera position or viewing angle. Its main function is to map the image information obtained by the camera back to the real three-dimensional space. For example, during the return of a drone, the world coordinate system is used to describe the location information of the drone and the Apriltag code. The image information captured by the camera needs to be converted from the image coordinate system or the camera coordinate system to the world coordinate system through coordinate transformation. The two position information can be compared and converted in the same coordinate system.

[0172] In one possible implementation, the camera coordinate system (Xc, Yc, Zc) takes the optical center of the camera as its origin, the z-axis is along the optical axis of the camera, and the x- and y-axes are in the image plane of the camera. The coordinates of an object in the camera coordinate system describe the position of the object relative to the camera. A key feature of this coordinate system is that it is directly related to the imaging relationship of the camera. The coordinates of the object in the three-dimensional space, namely the world coordinate system, can be mapped to the image coordinate system through the camera's projection model. For example, during the return of a drone, the camera carried by the drone captures the Apriltag code of the location where it is to land, and the Apriltag code is displayed in the camera as two-dimensional image information.

[0173] In one possible implementation, the image coordinate system (x, y) is a two-dimensional coordinate system used to describe the position of a pixel in an image. It is the result of projecting a three-dimensional space point in the camera coordinate system onto the image plane. The image coordinate system is a two-dimensional coordinate system with its origin at the upper left corner of the image.

[0174] In one possible implementation, the pixel coordinate system (u, v) is used to describe the two-dimensional coordinate system of each pixel position in the image, which is similar to the image coordinate system. However, the pixel coordinate system focuses on using integer values ​​to represent discrete pixel positions in the image. The pixel origin is located in the upper left corner of the image, and the column coordinate u increases from left to right along the horizontal direction; the row coordinate v increases from top to bottom along the vertical direction. For example, for an image with a resolution of 640x480, the column coordinate u of the pixel ranges from [0,639] and the row coordinate v ranges from [0,479]. The position of each pixel can be represented by integer coordinates.

[0175] In one possible implementation, the relationship between the camera coordinate system and the world coordinate system: The camera coordinate system is usually defined relative to the world coordinate system. The world coordinate system is a global reference coordinate system used to describe the position of objects in the real world; while the camera coordinate system is local and bound to the camera, so there is a coordinate transformation between the camera coordinate system and the world coordinate system, including translation and rotation, which can be described by a matrix. Specifically, Figure 6 The relationship diagram between the camera coordinate system and the world coordinate system is shown, where p can be regarded as the point coordinate information provided by the Apriltag code.

[0176] In a possible implementation, assuming that the coordinates of the object (Apriltag code) in the world coordinate system are (Xw, Yw, Zw), then through the rotation matrix R and the translation vector t, the position of the object (Apriltag code) in the camera coordinate system (Xc, Yc, Zc) can be expressed as:

[0177]

[0178] In one possible implementation, in homogeneous coordinates, the motion of a rigid body can be represented by 6 parameters (3 rotation vectors, 3 translation vectors), which form the camera's extrinsic parameter matrix. The camera's extrinsic parameters determine the transformation of a space point from the world coordinate system to the camera coordinate system. The formula for its homogeneous coordinates is as follows:

[0179]

[0180] Based on the relationship between the camera coordinate system and the world coordinate system, objects in the world coordinate system can be transferred to the camera coordinate system for description.

[0181] In one possible implementation, Figure 7 The relationship between the image coordinate system and the camera coordinate system is shown below: The transformation between the camera coordinate system and the image coordinate system is usually described by the pinhole camera model. The object in the three-dimensional world is projected onto the two-dimensional image plane through the camera. The formula is as follows:

[0182]

[0183] Where f is the focal length of the camera, which is expressed in homogeneous coordinates as Formula 4:

[0184]

[0185] In one possible implementation, Figure 8 The relationship between the pixel coordinate system and the image coordinate system is shown. The image coordinates can be further converted to the position of the pixel in the image. The formula is as follows:

[0186]

[0187] The homogeneous coordinates are expressed as:

[0188]

[0189] Among them, (u0, v0) is usually the center point of the image, s x ,s y It is the pixel size in the X and Y axis directions of the image, in millimeters / pixel.

[0190] In one possible implementation, Fig. 9 The figure shows a method for obtaining the relative posture information required for the UAV return mission using the Apriltag code.

[0191] In a possible implementation, formulas (1) to (6) are combined to obtain:

[0192]

[0193] From formula (7), we can see that if the pixel coordinates [u, v], f and its coordinates in the world coordinate system [X w Y w Z w ], the camera external parameter matrix can be obtained through the conversion relationship between the coordinate systems The camera external parameter matrix is ​​the rotation matrix and displacement matrix required for the fixed-point landing of the drone. R and t are the relative posture information required for the drone to return home. With this information, the actual hardware of the drone can be further operated to complete the expected action and realize the return mission.

[0194] In this implementation, the platform image data at the first moment is input into the pre-trained target identification recognition network, so that the target identification recognition network identifies the platform image data at the first moment to obtain a first identification recognition result, and coordinate transformation is performed based on the first identification recognition result to obtain the position of the first identification image. Combined with computer vision technology, the accuracy of determining the position of the first identification image is improved, thereby improving the accuracy of fixed-point return.

[0195] In a possible implementation, the method for training the target identification network includes:

[0196] Obtaining an initial identification network; the initial identification network is constructed based on an attention mechanism and a U-shaped network;

[0197] Acquire sample platform image data annotated with a true value label; the sample platform image data is obtained by collecting the first identification image and the second identification image by drones at different locations in different scenes, and the true value label represents the position of at least one of the first identification image and the second identification image;

[0198] Inputting the sample platform image data into the initial identification network to obtain a prediction result;

[0199] Determining a network loss value based on difference information between the true value label of the sample platform image data and the prediction result;

[0200] When the network loss value does not satisfy the preset loss condition, the network parameters of the initial identification network are updated based on the network loss value until the network loss value satisfies the preset loss condition, and the target identification network is determined to be obtained.

[0201] In a possible implementation, the sample platform image data includes image data collected in different application scenarios, including an Apriltag data set under poor lighting conditions such as at night or on cloudy days.

[0202] In one possible implementation, the application scenarios include scenarios where the drone operates in special locations such as mountains or sea surfaces, and the application scenarios also include scenarios where the drone lands on a moving aircraft carrier, warship, ground armored vehicle, etc.

[0203] In a possible implementation, the application scenario also includes a scenario in which the drone operates in dim light conditions such as at night or on a cloudy day.

[0204] In a possible implementation manner, the sample platform image data further includes image data of motion blur defects generated by the mobile camera during shooting.

[0205] In one possible implementation, the initial identification network is a U-shaped neural network combined with a self-attention mechanism, which is trained based on sample platform image data collected in different scenarios and labeled with true value labels, ensuring that the drone can successfully complete the return mission in different scenarios without being constrained by the environment.

[0206] In a possible implementation, the true value label of the sample platform image data is the ID of the sample platform image data, the position coordinates of the four corner points and the center point. Specifically, the position coordinates are coordinates in a pixel coordinate system.

[0207] In one possible implementation, the sample platform image data is input into the initial identification network to obtain a prediction result; based on the difference information between the true value label of the sample platform image data and the prediction result, the network loss value is determined; based on the network loss value, the network parameters of the initial identification network are updated until the network loss value meets the preset loss condition, and it is determined that the target identification network is obtained.

[0208] In this implementation, network training is performed based on samples from different scenarios, so that the pre-trained target identification recognition network has the ability to recognize Apriltag codes in different scenarios, which improves the robustness of the network, expands the application scenarios of UAV fixed-point landing, and ensures that it can work normally in environments with low image signal-to-noise ratio such as night, cloudy days, and dim light.

[0209] In a possible implementation manner, acquiring the platform image data at the first moment, and determining the position of the first identification image based on the platform image data at the first moment, further includes:

[0210] Inputting the platform image data at the first moment into an image preprocessing network, performing image denoising processing based on the image preprocessing network, and obtaining denoised platform data;

[0211] Inputting the noise reduction platform data into the Apriltag detector for image recognition to obtain an image recognition result;

[0212] Coordinate transformation is performed based on the image recognition result to obtain the position of the first identification image.

[0213] In a possible implementation, the Apriltag detector is an open source tool, and the Apriltag detector can detect any Apriltag code information through the compilation library built by the environment. The Apriltag detector is trained based on the open source Apriltag dataset, including 8 datasets such as tag16h5 and tag25h9.

[0214] In one possible implementation, Fig.10 As shown, the image preprocessing network is constructed based on the attention mechanism and the U-shaped network. The input of the image preprocessing network is the platform image data, and the output is the platform image data after denoising, that is, the denoised platform data.

[0215] In one possible implementation, Fig.11 A structural diagram of the self-attention mechanism layer of the image preprocessing network is shown.

[0216] In a possible implementation, the training method of the image preprocessing network is:

[0217] Obtaining an initial image preprocessing network; the initial image preprocessing network is constructed based on an attention mechanism and a U-shaped network;

[0218] Acquire sample Apriltag code data with labels; the labels of the sample Apriltag code data are image data with a signal-to-noise ratio greater than a preset signal-to-noise ratio obtained by performing image acquisition on the Apriltag code, and the sample Apriltag code data are sample Apriltag code data with preset noise added;

[0219] Inputting the sample Apriltag code data into the initial image preprocessing network to obtain a prediction result;

[0220] Determine a network loss value based on the difference information between the label of the sample Apriltag code data and the prediction result;

[0221] When the network loss value does not meet the preset loss condition, the network parameters of the initial image preprocessing network are updated based on the network loss value until the network loss value meets the preset loss condition, and it is determined that the image preprocessing network is obtained.

[0222] In one possible implementation, Fig.12As shown, the label of the sample Apriltag code data includes a clear Apriltag code under normal light; and the sample Apriltag code data includes an Apriltag code under dim light.

[0223] In one possible implementation, Fig.13 As shown, the label of the sample Apriltag code data includes the Apriltag code without the motion blur defect; the sample Apriltag code data includes the Apriltag code with the motion blur defect.

[0224] In one possible implementation, Fig.14 As shown, the label of the sample Apriltag code data includes the Apriltag code under normal light; the sample Apriltag code data includes the Apriltag code with preset noise added.

[0225] In a possible implementation, the preset noise includes salt and pepper noise, etc. Salt and pepper noise is noise generated by random signal interference during image acquisition and transmission, such as thermal noise caused by resistance, photon noise, dark current noise, and light response non-uniformity noise.

[0226] In a possible implementation, the difference information between the label of the sample Apriltag code data and the prediction result is calculated based on a loss function to obtain the network loss value. Specifically, the loss function includes a regression loss function, a mean square error function, an intersection-over-union loss function, and the like.

[0227] In step S107, the UAV is controlled to descend to a second intermediate position based on the position of the first intermediate position and the first identification image, and the time when the UAV descends to the second intermediate position is determined as a second moment; the second intermediate position is a position where the vertical distance between the UAV and the mobile platform meets a second preset distance condition.

[0228] In a possible implementation, the controlling the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image includes:

[0229] Determining a second operating parameter of the drone based on the first intermediate position and the position of the first identification image;

[0230] Control the UAV to move based on the second operating parameter to obtain a second movement result;

[0231] When the second movement result indicates that the vertical distance between the UAV and the mobile platform satisfies the second preset distance condition, it is determined that the UAV descends to the second intermediate position.

[0232] In a possible implementation, the second movement result includes a second position of the drone and a second position of the mobile platform, the second position of the drone represents position information of the drone corresponding to the second movement result, and the second position of the mobile platform represents position information of the mobile platform corresponding to the second movement result.

[0233] The method further comprises:

[0234] Calculating the distance between the second position of the UAV and the second position of the mobile platform in the vertical direction to obtain a vertical distance;

[0235] When the vertical distance is less than a preset vertical distance threshold, it is determined that the distance between the UAV and the mobile platform meets the second preset distance condition.

[0236] In one possible implementation, Figure 5 As shown, in the second stage of approaching the platform, it is necessary to slow down and maintain a certain height and adjust the position in real time.

[0237] In one possible implementation, in the second stage, the onboard camera can completely recognize the QR code for visual navigation and then perform fixed-point landing.

[0238] In one possible implementation, in the second stage of approaching the platform, when the onboard camera recognizes the larger Apriltag code, the second stage of slow descent to the platform begins. The coordinate information of the QR code in the camera coordinate system is obtained by detecting the QR code. The onboard CPU calculates the relative coordinate information of the drone relative to the Apriltag code in the world coordinate system through coordinate conversion, and then obtains the rotation and translation posture changes that the drone needs to make in the world coordinate system to reach the landing position, that is, the matrices R and t. As the drone descends, the distance from the landing point gradually decreases. When the larger Apriltag code, that is, the first identification image, exceeds the field of view of the onboard camera and the smaller Apriltag code, that is, the second identification image, is recognized, the second stage is completed.

[0239] In a possible implementation, after moving for a second preset period, the distance in the vertical direction between the second position of the UAV and the second position of the mobile platform is calculated to obtain the vertical distance; when the vertical distance meets the second preset distance condition, it is determined that the UAV has descended to the second intermediate position.

[0240] In a possible implementation, as the drone approaches the second intermediate position, it decelerates and maintains a preset height to follow the mobile platform until the onboard camera recognizes the second identification image.

[0241] In this implementation, the UAV is controlled to descend to the second intermediate position based on the position of the first intermediate position and the first identification image, and the time when the UAV descends to the second intermediate position is determined as the second moment, so that the UAV completes the task of approaching the mobile platform in the second stage, providing a basis for subsequently meeting the conditions for landing.

[0242] In step S109, the platform image data at the second moment is acquired, and the position of the second identification image is determined based on the platform image data at the second moment.

[0243] In a possible implementation manner, determining the position of the second identification image based on the platform image data at the second moment includes:

[0244] Acquire the target identification network;

[0245] Inputting the platform image data at the second moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the second moment to obtain a second identification recognition result;

[0246] Coordinate transformation is performed based on the second marker recognition result to obtain the position of the second marker image.

[0247] In a possible implementation, the platform image data at the second moment is an image obtained by capturing the second identification image by an onboard camera of the drone.

[0248] In one possible implementation, during the movement of the drone in the second phase, platform image data is continuously collected, and the collected platform image data is input into the target identification network in real time, so that the target identification network can perform real-time identification to improve the accuracy of identification.

[0249] In a possible implementation, the input of the target identification network is the platform image data of the second stage, and the output is the ID of the platform image data and the position coordinates of the four corner points and the center point.

[0250] In a possible implementation, the second identification result is the ID of the platform image data and the position coordinates of four corner points and a center point.

[0251] In a possible implementation, coordinate transformation is performed based on the second identification recognition result to obtain the position of the second identification image, which is to transform the position coordinates of the four corner points and the center point of the platform image data from the pixel coordinate system to the world coordinate system.

[0252] In a possible implementation, the target identification network has the function of detecting Apriltag codes of different scales.

[0253] In this implementation, the platform image data at the second moment is obtained, and the position of the second identification image is determined based on the platform image data at the second moment. Since the target identification recognition network is trained based on samples with Apriltag codes at different scales, one recognition network can be used for recognition in the entire control process, thereby improving recognition efficiency and reducing R&D costs.

[0254] In step S1011, the drone is controlled to land at the target landing position based on the second intermediate position and the position of the second identification image.

[0255] In a possible implementation, the controlling the drone to land at the target landing position based on the second intermediate position and the position of the second identification image includes:

[0256] determining a third operating parameter of the UAV based on the second intermediate position and the position of the second identification image;

[0257] Controlling the UAV to move based on the third operating parameter to obtain the third movement result;

[0258] When the third movement result indicates that the distance between the UAV and the target landing position satisfies a third preset distance condition, the UAV is controlled to land at the target landing position.

[0259] In a possible implementation, the third movement result includes a third position of the drone and a third position of the mobile platform, wherein the third position of the drone represents position information of the drone corresponding to the third movement result, and the third position of the mobile platform represents position information of the target landing position corresponding to the third movement result.

[0260] The method further comprises:

[0261] Calculating the distance between the third position of the UAV and the third position of the mobile platform to obtain a landing distance;

[0262] When the landing distance is less than a preset landing distance threshold, the UAV is controlled to land at the target landing position.

[0263] In one possible implementation, Figure 5As shown, the third stage of landing, when the onboard camera recognizes the smaller Apriltag code, the third stage of landing begins. The process is the same as the second stage, and the attitude control of the drone is performed by obtaining the rotation and translation matrices.

[0264] In a possible implementation, after moving for a third preset period, the distance between the third position of the UAV and the third position of the mobile platform is calculated to obtain a landing distance; when the landing distance is less than a preset landing distance threshold, the UAV is controlled to land at the target landing position.

[0265] In a possible implementation, as the drone approaches the third position of the mobile platform, it decelerates and lands on the mobile platform.

[0266] In one possible implementation, the Proportional-Integral-Differential (PID) control algorithm is used in all three stages. After the April tag code is recognized by the onboard camera, the rotation matrix and the translation matrix, i.e., the matrices R and T, can be obtained through CPU calculation. At this time, the relative posture is set as the target posture, and the posture output by the inertial navigation unit (IMU) carried by the drone is the current posture. By calculating the error between the two, the error is input into the PID controller to calculate the control output. The output of the PID controller will drive the mechanical structures of various parts of the drone to adjust the drone's posture to achieve the target posture.

[0267] In a possible implementation, the preset landing distance threshold is set based on the relative speed of the mobile platform and the drone, so that the drone is at a safe landing height and will not be damaged due to the falling height.

[0268] In a possible implementation, the preset landing distance threshold is set based on the distance between the center position of the target landing position and the drone. Specifically, the preset landing distance threshold is set to 0.5 meters, and when the distance between the drone and the center position of the target landing position is less than 0.5 meters under clear and negative temperatures, the drone is controlled to land at the target landing position.

[0269] In this implementation, the UAV is controlled to land at the target landing position based on the position of the second intermediate position and the second identification image. When the distance between the UAV and the mobile platform is within a safe distance for a smooth landing, the UAV is controlled to land at the target landing position, thereby improving the safety of landing.

[0270] The present solution provides a method for returning to a fixed point by a UAV on a mobile platform, which has the following beneficial effects: by responding to a return to a fixed point command for the UAV, obtaining the current position of the UAV and the position of the mobile platform; controlling the UAV to arrive at a first intermediate position based on the position of the UAV and the position of the mobile platform, and determining the time when the UAV arrives at the first intermediate position as a first moment; the first intermediate position is a position where the horizontal distance from the mobile platform meets a first preset distance condition; obtaining platform image data at the first moment, and determining the position of the first identification image based on the platform image data at the first moment; controlling the UAV to descend to a second intermediate position based on the position of the first intermediate position and the first identification image, and determining the time when the UAV descends to the second intermediate position as a second moment; the second intermediate position is a position where the vertical distance from the mobile platform meets a second preset distance condition; obtaining platform image data at the second moment, and determining the position of the second identification image based on the platform image data at the second moment; and controlling the UAV to land at the target landing position based on the position of the second intermediate position and the second identification image. The target landing position is determined based on computer vision technology, and the return of the UAV is controlled according to the positional relationship between the UAV and the landing mobile platform, so that the UAV lands at the target landing position. By improving the accuracy of positioning the target landing position and coordinating corresponding control methods, the accuracy of the UAV's return is improved, solving the problem of return deviation caused by inaccurate return path during the return of the UAV.

[0271] Fig.15 The schematic diagram of the structure of a fixed-point return device 1500 for a UAV on a mobile platform provided in an embodiment of the present application is shown. The device has the function of implementing the fixed-point return method for a UAV on a mobile platform in the above method embodiment. The function can be implemented by hardware or by hardware executing corresponding software. Fig.15 As shown, the device may include:

[0272] The information acquisition module 1501 is used to obtain the current position of the drone and the mobile platform in response to the fixed-point return command to the drone;

[0273] The first control module 1502 is used to control the drone to arrive at a first intermediate position based on the drone position and the mobile platform position, and determine the time when the drone arrives at the first intermediate position as a first moment; the first intermediate position is a position where the horizontal distance from the mobile platform meets a first preset distance condition;

[0274] The first identification position determination module 1503 is used to obtain the platform image data at the first moment, and determine the position of the first identification image based on the platform image data at the first moment; the platform image data is obtained by the drone collecting images of the mobile platform;

[0275] The second control module 1504 is used to control the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image, and determine the time when the drone descends to the second intermediate position as a second moment; the second intermediate position is a position where the vertical distance from the mobile platform meets a second preset distance condition;

[0276] A second identification position determination module 1505 is used to obtain the platform image data at the second moment, and determine the position of the second identification image based on the platform image data at the second moment;

[0277] The third control module 1506 is used to control the UAV to land at the target landing position based on the second intermediate position and the position of the second identification image.

[0278] In a possible implementation, the first control module 1502 is configured to:

[0279] Acquire platform movement information of the mobile platform at a current moment; the platform movement information represents movement parameters of the mobile platform;

[0280] Determining a first operating parameter of the drone based on the drone position, the mobile platform position, and the platform movement information;

[0281] Control the UAV to move based on the first operating parameter to obtain a first movement result;

[0282] When the first movement result indicates that the horizontal distance between the drone and the mobile platform satisfies the first preset distance condition, it is determined that the drone has arrived at the first intermediate position.

[0283] In a possible implementation, the first movement result includes a first position of a drone and a first position of a mobile platform, the first position of the drone represents position information of the drone corresponding to the first movement result, and the first position of the mobile platform represents position information of the mobile platform corresponding to the first movement result.

[0284] The first control module 1502 is further configured to:

[0285] Calculating the distance between the first position of the UAV and the first position of the mobile platform in the horizontal direction to obtain a horizontal distance;

[0286] When the horizontal distance is less than a preset horizontal distance threshold, it is determined that the distance between the UAV and the mobile platform meets the first preset distance condition.

[0287] In a possible implementation, the first identification position determining module 1503 is configured to:

[0288] Obtaining a target identification network; the target identification network is constructed based on a self-attention mechanism;

[0289] Inputting the platform image data at the first moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the first moment to obtain a first identification recognition result;

[0290] Coordinate transformation is performed based on the first marker recognition result to obtain the position of the first marker image.

[0291] In a possible implementation, the apparatus further includes a network training module 1507, wherein the network training module is configured to:

[0292] Obtaining an initial identification network; the initial identification network is constructed based on an attention mechanism and a U-shaped network;

[0293] Acquire sample platform image data annotated with a true value label; the sample platform image data is obtained by collecting the first identification image and the second identification image by drones at different locations in different scenes, and the true value label represents the position of at least one of the first identification image and the second identification image;

[0294] Inputting the sample platform image data into the initial identification network to obtain a prediction result;

[0295] Determining a network loss value based on difference information between the true value label of the sample platform image data and the prediction result;

[0296] When the network loss value does not satisfy the preset loss condition, the network parameters of the initial identification network are updated based on the network loss value until the network loss value satisfies the preset loss condition, and the target identification network is determined to be obtained.

[0297] In a possible implementation, the second control module 1504 is configured to:

[0298] Determining a second operating parameter of the drone based on the first intermediate position and the position of the first identification image;

[0299] Control the UAV to move based on the second operating parameter to obtain a second movement result;

[0300] When the second movement result indicates that the vertical distance between the UAV and the mobile platform satisfies the second preset distance condition, it is determined that the UAV descends to the second intermediate position.

[0301] In a possible implementation, the second movement result includes a second position of the drone and a second position of the mobile platform, the second position of the drone represents position information of the drone corresponding to the second movement result, and the second position of the mobile platform represents position information of the mobile platform corresponding to the second movement result.

[0302] The second control module 1504 is further configured to:

[0303] Calculating the distance between the second position of the UAV and the second position of the mobile platform in the vertical direction to obtain a vertical distance;

[0304] When the vertical distance is less than a preset vertical distance threshold, it is determined that the distance between the UAV and the mobile platform meets the second preset distance condition.

[0305] In a possible implementation, the second identification position determining module 1505 is configured to:

[0306] Acquire the target identification network;

[0307] Inputting the platform image data at the second moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the second moment to obtain a second identification recognition result;

[0308] Coordinate transformation is performed based on the second marker recognition result to obtain the position of the second marker image.

[0309] In a possible implementation, the third control module 1506 is configured to:

[0310] determining a third operating parameter of the UAV based on the second intermediate position and the position of the second identification image;

[0311] Controlling the UAV to move based on the third operating parameter to obtain the third movement result;

[0312] When the third movement result indicates that the distance between the UAV and the target landing position satisfies a third preset distance condition, the UAV is controlled to land at the target landing position.

[0313] In a possible implementation, the third movement result includes a third position of the drone and a third position of the mobile platform, wherein the third position of the drone represents position information of the drone corresponding to the third movement result, and the third position of the mobile platform represents position information of the target landing position corresponding to the third movement result.

[0314] The third control module 1506 is further configured to:

[0315] Calculating the distance between the third position of the UAV and the third position of the mobile platform to obtain a landing distance;

[0316] When the landing distance is less than a preset landing distance threshold, the UAV is controlled to land at the target landing position.

[0317] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0318] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the methods for returning to a fixed point by a drone for a mobile platform as provided in the above method embodiments.

[0319] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0320] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method for returning to a fixed point for an unmanned aerial vehicle on a mobile platform. The at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the methods for returning to a fixed point for an unmanned aerial vehicle on a mobile platform provided in the above method embodiments.

[0321] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0322] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0323] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0324] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0325] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for returning to a fixed point by a UAV on a mobile platform, characterized in that: The mobile platform is provided with a target landing position of the UAV, a first identification image and a second identification image are provided at the target landing position, and a size of the second identification image is smaller than a size of the first identification image, and the method comprises: In response to a fixed-point return command to the drone, obtain the drone position and the mobile platform position at the current time; Based on the position of the drone and the position of the mobile platform, the drone is controlled to arrive at a first intermediate position, and the time when the drone arrives at the first intermediate position is determined as a first moment; the first intermediate position is a position where the horizontal distance from the mobile platform meets a first preset distance condition; Acquire platform image data at the first moment, and determine the position of the first identification image based on the platform image data at the first moment; Based on the first intermediate position and the position of the first identification image, the drone is controlled to descend to a second intermediate position, and the time when the drone descends to the second intermediate position is determined as a second moment; the second intermediate position is a position where the vertical distance with the mobile platform meets a second preset distance condition; Acquire platform image data at the second moment, and determine the position of the second identification image based on the platform image data at the second moment; The drone is controlled to land at the target landing position based on the second intermediate position and the position of the second identification image.

2. The method for returning to a fixed point by a UAV on a mobile platform according to claim 1, characterized in that: The controlling the drone to arrive at the first intermediate position based on the drone position and the mobile platform position includes: Acquire platform movement information of the mobile platform at a current moment; the platform movement information represents movement parameters of the mobile platform; Determining a first operating parameter of the drone based on the drone position, the mobile platform position, and the platform movement information; Control the UAV to move based on the first operating parameter to obtain a first movement result; When the first movement result indicates that the horizontal distance between the drone and the mobile platform satisfies the first preset distance condition, it is determined that the drone has arrived at the first intermediate position.

3. The method for returning to a fixed point by a UAV for a mobile platform according to claim 2, characterized in that: The first movement result includes a first position of the drone and a first position of the mobile platform, wherein the first position of the drone represents the position information of the drone corresponding to the first movement result, and the first position of the mobile platform represents the position information of the mobile platform corresponding to the first movement result. The method further comprises: Calculating the distance between the first position of the UAV and the first position of the mobile platform in the horizontal direction to obtain a horizontal distance; When the horizontal distance is less than a preset horizontal distance threshold, it is determined that the distance between the UAV and the mobile platform meets the first preset distance condition.

4. The method for returning to a fixed point by a UAV for a mobile platform according to claim 1, characterized in that: The determining the position of the first identification image based on the platform image data at the first moment includes: Obtaining a target identification network; the target identification network is constructed based on a self-attention mechanism; Inputting the platform image data at the first moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the first moment to obtain a first identification recognition result; Coordinate transformation is performed based on the first marker recognition result to obtain the position of the first marker image.

5. The method for returning to a fixed point by a UAV for a mobile platform according to claim 4, characterized in that: The method for training the target identification network includes: Obtaining an initial identification network; the initial identification network is constructed based on an attention mechanism and a U-shaped network; Acquire sample platform image data annotated with a true value label; the sample platform image data is obtained by collecting the first identification image and the second identification image by drones at different locations in different scenes, and the true value label represents the position of at least one of the first identification image and the second identification image; Inputting the sample platform image data into the initial identification network to obtain a prediction result; Determining a network loss value based on difference information between the true value label of the sample platform image data and the prediction result; When the network loss value does not satisfy the preset loss condition, the network parameters of the initial identification network are updated based on the network loss value until the network loss value satisfies the preset loss condition, and the target identification network is determined to be obtained.

6. The method for returning to a fixed point by a UAV for a mobile platform according to claim 1, characterized in that: The controlling the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image comprises: Determining a second operating parameter of the drone based on the first intermediate position and the position of the first identification image; Control the UAV to move based on the second operating parameter to obtain a second movement result; When the second movement result indicates that the vertical distance between the UAV and the mobile platform satisfies the second preset distance condition, it is determined that the UAV descends to the second intermediate position.

7. The method for returning to a fixed point by a UAV for a mobile platform according to claim 6, characterized in that: The second movement result includes a second position of the drone and a second position of the mobile platform, wherein the second position of the drone represents the position information of the drone corresponding to the second movement result, and the second position of the mobile platform represents the position information of the mobile platform corresponding to the second movement result. The method further comprises: Calculating the distance between the second position of the UAV and the second position of the mobile platform in the vertical direction to obtain a vertical distance; When the vertical distance is less than a preset vertical distance threshold, it is determined that the distance between the UAV and the mobile platform meets the second preset distance condition.

8. The method for returning to a fixed point by a UAV for a mobile platform according to claim 5, characterized in that: The determining the position of the second identification image based on the platform image data at the second moment includes: Acquire the target identification network; Inputting the platform image data at the second moment into the target identification recognition network, so that the target identification recognition network performs identification recognition on the platform image data at the second moment to obtain a second identification recognition result; Coordinate transformation is performed based on the second identification recognition result to obtain the position of the second identification image.

9. The method for returning to a fixed point by a UAV for a mobile platform according to claim 1, characterized in that: The controlling the UAV to land at the target landing position based on the second intermediate position and the position of the second identification image includes: determining a third operating parameter of the UAV based on the second intermediate position and the position of the second identification image; Controlling the UAV to move based on the third operating parameter to obtain the third movement result; When the third movement result indicates that the distance between the UAV and the target landing position satisfies a third preset distance condition, the UAV is controlled to land at the target landing position.

10. The method for returning to a fixed point by a UAV for a mobile platform according to claim 9, characterized in that: The third movement result includes a third position of the drone and a third position of the mobile platform, wherein the third position of the drone represents the position information of the drone corresponding to the third movement result, and the third position of the mobile platform represents the position information of the target landing position corresponding to the third movement result. The method further comprises: Calculating the distance between the third position of the UAV and the third position of the mobile platform to obtain a landing distance; When the landing distance is less than a preset landing distance threshold, the UAV is controlled to land at the target landing position.

11. A fixed-point return device for a UAV facing a mobile platform, characterized in that: The mobile platform is provided with a target landing position of the UAV, a first identification image and a second identification image are provided at the target landing position, and a size of the second identification image is smaller than a size of the first identification image, and the device comprises: An information acquisition module is used to obtain the current position of the drone and the mobile platform in response to a fixed-point return command to the drone; A first control module is used to control the drone to arrive at a first intermediate position based on the position of the drone and the position of the mobile platform, and determine the time when the drone arrives at the first intermediate position as a first moment; the first intermediate position is a position where the horizontal distance from the mobile platform meets a first preset distance condition; a first identification position determination module, configured to obtain the platform image data at the first moment, and determine the position of the first identification image based on the platform image data at the first moment; a second control module, configured to control the drone to descend to a second intermediate position based on the first intermediate position and the position of the first identification image, and determine the time when the drone descends to the second intermediate position as a second moment; the second intermediate position is a position where the vertical distance from the mobile platform meets a second preset distance condition; a second identification position determination module, configured to obtain the platform image data at the second moment, and determine the position of the second identification image based on the platform image data at the second moment; A third control module is used to control the UAV to land at the target landing position based on the second intermediate position and the position of the second identification image.

12. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the fixed-point return method for a mobile platform-oriented unmanned aerial vehicle as described in any one of claims 1 to 10.

13. A computer-readable storage medium, wherein at least one instruction or at least one program is stored in the computer-readable storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the fixed-point return method for a mobile platform-oriented drone as described in any one of claims 1 to 10.

Images