A Method and System for Autonomous Recovery of a Hybrid-Wing Unmanned Aerial Vehicle on the Sea Surface Based on Vision Algorithms

Abstract

The present invention specifically relates to a method and system for autonomous recovery of a blended-wing unmanned aerial vehicle on the sea surface based on a vision algorithm. The method includes: when a blended-wing unmanned aerial vehicle to be recovered is identified within a preset recovery airspace, performing image acquisition to obtain at least one frame of unmanned aerial vehicle image containing a cooperation identifier; identifying at least one frame of unmanned aerial vehicle image containing a cooperation identifier, and screening for a preferred cooperation identifier based on the distortion degree of the cooperation identifier in the unmanned aerial vehicle image; collecting a current unmanned aerial vehicle image containing the preferred cooperation identifier for the blended-wing unmanned aerial vehicle, and determining the current position information of the blended-wing unmanned aerial vehicle according to the current unmanned aerial vehicle image; predicting the motion trajectory of the blended-wing unmanned aerial vehicle based on the current position information to obtain a position estimate of the blended-wing unmanned aerial vehicle at the next moment; configuring an autonomous guidance control instruction for the blended-wing unmanned aerial vehicle according to the position estimate to be used for realizing closed-loop control of the blended-wing unmanned aerial vehicle until the blended-wing unmanned aerial vehicle lands at a predetermined position.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous recovery of sea drones, and particularly to a method and system for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm. Background Art

[0002] In the related art, the high flight speed, high flexibility, and long endurance of drones are key factors for performing sea tasks. Hybrid-wing drones combine the advantages of fixed-wing drones and rotor drones in terms of speed and flexibility. However, limited by the sea recovery conditions, when the existing skyhook recovery methods are used to recover small drones, there are problems of insufficient reliability; moreover, the volume of skyhook recovery equipment is relatively large, making it difficult to implement on small unmanned platforms. In addition, the existing net-collision recovery methods are likely to cause damage to the recovery device and the structure of the drone itself; at the same time, the preparation period of net-collision recovery equipment is relatively long. Therefore, there is a lack of relevant technical solutions for the sea recovery of hybrid-wing drones in the prior art, which also restricts the marine application of hybrid-wing drones.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present invention, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] The present invention provides a method for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm, a system for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm, a computer program product, and a storage medium, thereby being able to overcome the defects existing in the prior art to a certain extent.

[0005] Other features and advantages of the present invention will become apparent through the following detailed description, or will be partially learned through the practice of the present invention.

[0006] According to a first aspect of the present invention, there is provided a method for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm, the method comprising:

[0007] When a hybrid-wing drone to be recovered is identified within a preset recovery airspace of an autonomous recovery platform, image acquisition is performed on the hybrid-wing drone to obtain at least one frame of drone image containing a cooperation mark; wherein, the drone image includes an optical image and a corresponding depth image; cooperation marks are respectively arranged at different positions on the bottom and side of the hybrid-wing drone;

[0008] Identify at least one frame of drone image containing a cooperation mark, and screen out preferred cooperation marks based on the distortion degree of the cooperation marks in the drone image;

[0009] The hybrid-wing unmanned aerial vehicle (UAV) captures the current UAV image containing the preferred cooperation identifier, and determines the current position information of the hybrid-wing UAV according to the current UAV image;

[0010] Based on the current position information, predict the motion trajectory of the hybrid-wing UAV to obtain the position estimate of the hybrid-wing UAV at the next moment;

[0011] Configure the autonomous guidance control instruction of the hybrid-wing UAV according to the position estimate to achieve the closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at the predetermined position; wherein, the autonomous guidance control instruction includes: position adjustment instructions for at least one of the x-axis direction, y-axis direction, and z-axis direction.

[0012] In some exemplary embodiments, the recognition of at least one frame of UAV image containing a cooperation identifier and the screening of the preferred cooperation identifier based on the distortion degree of the cooperation identifier in the UAV image include:

[0013] When a frame of optical image contains multiple cooperation identifiers, evaluate the distortion degree of the cooperation identifiers based on the area and perimeter of the cooperation identifiers in the optical image;

[0014] According to the evaluation result of the distortion degree of the cooperation identifier, screen the preferred cooperation identifier.

[0015] In some exemplary embodiments, the recognition of at least one frame of UAV image containing a cooperation identifier and the screening of the preferred cooperation identifier based on the distortion degree of the cooperation identifier in the UAV image include:

[0016] When the same cooperation identifier is included in multiple frames of UAV images, evaluate the distortion degree of the cooperation identifier according to the resolution of the optical image and the position information of the center of the cooperation identifier in the image;

[0017] According to the evaluation result of the distortion degree of the cooperation identifier, determine the preferred cooperation identifier and configure the corresponding optical image for calculating the current position information of the hybrid-wing UAV.

[0018] In some exemplary embodiments, the determination of the current position information of the hybrid-wing UAV according to the current UAV image includes:

[0019] Determine the conversion relationship between the pixel coordinate system of the optical image and the camera coordinate system according to the camera internal parameters;

[0020] Based on the conversion relationship, convert the pixel coordinates of each end angle of the preferred cooperation identifier in the pixel coordinate system into the corresponding three-dimensional point cloud coordinates in the camera coordinate system;

[0021] When the preferred cooperation identifier is the bottom cooperation identifier, configure the positioning coordinates of the preferred cooperation identifier in the camera coordinate system according to the mean value of the three-dimensional point cloud coordinates of each end corner; or,

[0022] When the preferred cooperation identifier is the side cooperation identifier, convert the mean value of the three-dimensional point cloud coordinates of each end corner of the preferred cooperation identifier according to the preset transformation matrix between the side cooperation identifier and the bottom cooperation identifier to determine the positioning coordinates of the preferred cooperation identifier; wherein, the positioning coordinates are the central coordinates of the preferred cooperation identifier;

[0023] Based on the rotation matrix between the camera coordinate system and the platform coordinate system, convert the positioning coordinates in the camera coordinate system into the relative position coordinates in the platform coordinate system and configure them as the current position information of the hybrid-wing unmanned aerial vehicle.

[0024] In some exemplary embodiments, the rotation matrix between the camera coordinate system and the platform coordinate system is determined based on the product of the motion rotation matrix between two consecutive frames of the image and the platform coordinate system rotation matrix.

[0025] In some exemplary embodiments, the motion trajectory prediction of the hybrid-wing unmanned aerial vehicle based on the current position information to obtain the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment includes:

[0026] Define the system state vector based on the physical quantities predicted for the hybrid-wing unmanned aerial vehicle and initialize it; wherein, the physical quantities include at least one of position coordinates, velocity, and acceleration; the initialization includes: initializing the state estimate and initializing the error covariance matrix;

[0027] At each time , based on the posterior state estimate value at the moment predict the prior state estimate value at the current moment , expressed as:

[0028]

[0029] wherein, represents the state transition matrix, represents the control matrix, represents the control input at the moment; the current prediction error covariance matrix at the moment represents the estimate of the uncertainty of the predicted state;

[0030] Use the measurement value at the current moment , the measurement matrix and the measurement error covariance matrix to calculate the Kalman gain coefficient , and update the posterior estimate value at the moment , expressed as:

[0031]

[0032]

[0033] wherein, represents the transpose of the measurement matrix, represents the estimation of the state estimation uncertainty, represents the inverse matrix of the measurement error covariance matrix and the prediction error covariance matrix;

[0034] And, update the covariance matrix of the error, expressed as:

[0035]

[0036] wherein, represents the estimation of the state estimation uncertainty;

[0037] Repeat the above steps iteratively using the currently obtained measurement values to update the state estimation of the target, and configure it as the position estimation of the hybrid-wing unmanned aerial vehicle at the next moment.

[0038] In some exemplary embodiments, configuring the autonomous guidance control instruction of the hybrid-wing unmanned aerial vehicle according to the position estimation for implementing closed-loop control of the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands at a predetermined position includes:

[0039] Using the PID closed-loop control method to decouple the adjustment of the hybrid-wing unmanned aerial vehicle in the x-axis direction, y-axis direction, and z-axis direction, expressed as:

[0040]

[0041] wherein, is the control output, is the deviation between the measurement value and the control value, is the proportionality coefficient, is the integral time coefficient, is the differential time coefficient.

[0042] In some exemplary embodiments, the method further includes:

[0043] After recognizing that the blended-wing unmanned aerial vehicle (UAV) touches the autonomous recovery platform, the autonomous recovery platform controls the mechanical clamping rod to directly contact the landing gear of the blended-wing UAV, so as to finely adjust the position of the blended-wing UAV to a predetermined landing point and lock the landing gear of the blended-wing UAV by using the mechanical clamping rod.

[0044] According to a second aspect of the present invention, there is provided a sea surface autonomous recovery system for a blended-wing UAV based on a vision algorithm, which is applied to the method in the above embodiment; the system includes: a remotely operated underwater vehicle (ROV) subsystem, and an autonomous recovery system disposed on the ROV subsystem;

[0045] The autonomous recovery system includes:

[0046] A landing platform for carrying the blended-wing UAV;

[0047] An optical detection module disposed at the edge of the landing platform for collecting optical and depth images at different angles and heights;

[0048] A correction and locking module disposed on the landing platform for locking the blended-wing UAV;

[0049] An underwater wireless charging module disposed at the bottom of the landing platform for performing close-range fast contactless charging on the blended-wing UAV;

[0050] An integrated control module disposed at the bottom of the landing platform for performing real-time processing on data and controlling the working states of each module.

[0051] In some exemplary embodiments, the landing platform includes:

[0052] A platform bracket;

[0053] A V-shaped passive guiding structure symmetrically disposed on the platform bracket; wherein, the center distance between the V-shaped grooves of the passive guiding structure matches the distance between the skid-type landing gears of the blended-wing UAV;

[0054] A platform support disposed within the frame structure of the platform bracket for providing support for the V-shaped passive guiding structure.

[0055] According to a third aspect of the present invention, there is provided a computer program product having a computer program stored thereon, and when the computer program is executed by a processor, it implements the above-mentioned sea surface autonomous recovery method for a blended-wing UAV based on a vision algorithm.

[0056] According to a fourth aspect of the present invention, there is provided a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the above-mentioned sea surface autonomous recovery method for a blended-wing UAV based on a vision algorithm.

[0057] According to a fifth aspect of the present invention, there is provided an electronic device, comprising:

[0058] a processor and a memory;

[0059] wherein, the memory is used to store executable instructions of the processor; the processor is configured to implement the above-mentioned method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm when executing the executable instructions.

[0060] The method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm provided by the embodiments of the present invention collects corresponding optical images and depth images after the hybrid-wing unmanned aerial vehicle enters a preset airspace of an autonomous recovery platform, screens out preferred cooperation identifiers for subsequent calculations by identifying the optical images, then calculates the current position information of the hybrid-wing unmanned aerial vehicle by using the depth images containing the preferred cooperation identifiers, predicts the motion trajectory of the hybrid-wing unmanned aerial vehicle based on the current position information, obtains the position estimate at the next moment, and further performs closed-loop control on the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands at a predetermined position. This solution realizes the recovery control strategy for the hybrid-wing unmanned aerial vehicle based on a vision algorithm, realizes the autonomous guided recovery of the hybrid-wing unmanned aerial vehicle under sea surface conditions, reduces the intervention of humans, and improves the autonomy of the recovery process. Using image data to position the hybrid-wing unmanned aerial vehicle is fast and accurate, and can quickly and reliably respond to the motion of the hybrid-wing unmanned aerial vehicle. Through trajectory prediction and control strategies, the error between the actual landing point and the predetermined landing point of the hybrid-wing unmanned aerial vehicle can be reduced, and the success rate of recovering the hybrid-wing unmanned aerial vehicle on the sea surface can be improved. The autonomous recovery process ensures its recovery efficiency.

[0061] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0063] Figure 1 A schematic diagram schematically showing an exemplary embodiment of a method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm of the present invention;

[0064] Figure 2 A schematic diagram schematically showing the flow of an exemplary embodiment of a method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm of the present invention;

[0065] Figure 3 Schematically show the overall architecture diagram of a hybrid-wing UAV sea surface autonomous recovery system according to an exemplary embodiment of the present invention;

[0066] Figure 4 Schematically show the explosion diagram of a hybrid-wing UAV sea surface autonomous recovery system according to an exemplary embodiment of the present invention;

[0067] Figure 5 Schematically show the structural diagram of a positive and negative thread screw rod slider module of a correction and locking module according to an exemplary embodiment of the present invention;

[0068] Figure 6 Schematically show the composition diagram of an integrated control module according to an exemplary embodiment of the present invention.

[0069] Among them, the reference numerals include: optical detection module 1, distal binocular camera 11, proximal binocular camera 12, landing platform 2, first V-shaped passive guiding structure 21, second V-shaped passive guiding structure 22, platform bracket 23, platform support 24, correction and locking module 3, first positive and negative thread screw rod slider module 31, second positive and negative thread screw rod slider module 32, drive motor 321, coupling 322, positive and negative thread screw rod 323, screw rod base 324, first slider 3251, second slider 3252, first mechanical clamp rod connecting piece 3261, second mechanical clamp rod connecting piece 3262, mechanical clamp rod 33, first limit switch 341, second limit switch 342, first electromagnet 351, second electromagnet 352, underwater wireless charging module 4, integrated control module 5, vision computing unit 51, sensor unit 52, wireless communication unit 53, execution controller unit 54, STM32 data processing unit 55, power supply unit 56, Jetson board 511, installation housing 512, four-way relay 541, encoder 542 includes: first encoder 5421 and second encoder 5422, motor controller 543 includes: first motor controller 5431 and second motor controller 5432, 24V power supply 561, 24V to 19V power module 562; hybrid-wing UAV 301, sea surface autonomous recovery system 302, ROV subsystem 303. Detailed implementation manners

[0070] Now, the exemplary embodiments will be described more comprehensively with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The described features, structures or characteristics can be combined in any suitable manner in one or more embodiments.

[0071] In addition, the attached drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0072] In the related art, compared with land recovery, the sea surface recovery of a hybrid-wing unmanned aerial vehicle (UAV) faces various difficulties. The UAV recovery system that requires manual intervention not only limits the usage scenarios of the system, but also reduces the recovery success rate and efficiency in complex environments such as sea surface wind and wave interference, making it difficult to meet the requirements of such cross-domain unmanned system cooperation paradigms.

[0073] In view of the disadvantages and deficiencies of the prior art, in this exemplary embodiment, a method for autonomous sea surface recovery of a hybrid-wing UAV based on a vision algorithm is provided. It solves the technical problem that the existing sea surface recovery method of a hybrid-wing UAV requires manual intervention, resulting in low efficiency and safety. Refer to Figure 1 As shown, the method may include the following steps:

[0074] Step S11, when a hybrid-wing UAV to be recovered is identified within a preset recovery airspace of the autonomous recovery platform, image acquisition is performed on the hybrid-wing UAV to obtain at least one frame of UAV image containing a cooperation identifier; wherein, the UAV image includes an optical image and a corresponding depth image; cooperation identifiers are respectively arranged at different positions on the bottom and side of the hybrid-wing UAV;

[0075] Step S12, identify at least one frame of UAV image containing a cooperation identifier, and screen for preferred cooperation identifiers based on the distortion degree of the cooperation identifiers in the UAV image;

[0076] Step S13, acquire a current UAV image of the hybrid-wing UAV containing the preferred cooperation identifier, and determine the current position information of the hybrid-wing UAV according to the current UAV image;

[0077] Step S14, perform motion trajectory prediction on the hybrid-wing UAV based on the current position information to obtain a position estimate of the hybrid-wing UAV at the next moment;

[0078] Step S15, configure an autonomous guidance control instruction for the hybrid-wing UAV according to the position estimate to be used for realizing closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position; wherein, the autonomous guidance control instruction includes: position adjustment instructions for at least one of the x-axis direction, y-axis direction, and z-axis direction.

[0079] Next, each step of the autonomous recovery method for a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm in this exemplary embodiment will be described in more detail in conjunction with the accompanying drawings and embodiments.

[0080] In step S11, when a hybrid-wing unmanned aerial vehicle to be recovered is identified within a preset recovery airspace of the autonomous recovery platform, image acquisition is performed on the hybrid-wing unmanned aerial vehicle to obtain at least one frame of unmanned aerial vehicle image containing a cooperation mark; wherein, the unmanned aerial vehicle image includes an optical image and a corresponding depth image; cooperation marks are respectively arranged at different positions on the bottom and side of the hybrid-wing unmanned aerial vehicle.

[0081] Exemplarily, for a hybrid-wing unmanned aerial vehicle, cooperation marks can be respectively arranged at different positions such as the bottom and side of the unmanned aerial vehicle body. For example, a cooperation mark can be arranged at the bottom of the unmanned aerial vehicle body, and a cooperation mark can be arranged on each of the four side walls of the unmanned aerial vehicle body. Among them, the cooperation mark can be an Aruco code. Each Aruco code set on the unmanned aerial vehicle body has a unique ID, and the IDs between the Aruco codes are different. By installing Aruco codes at different positions of the hybrid-wing unmanned aerial vehicle and using them for detection and positioning, it is ensured that at least one cooperation mark can be collected and recognized at each moment during the whole process of unmanned aerial vehicle recovery.

[0082] For example, at least one set of binocular cameras can be arranged on the autonomous recovery platform, and the binocular cameras are used to collect an optical image and a corresponding depth image of the hybrid-wing unmanned aerial vehicle.

[0083] Among them, the optical image can be an RGB image.

[0084] Specifically, the hybrid-wing unmanned aerial vehicle can be controlled to fly to the preset recovery airspace of the autonomous recovery platform by using Beidou positioning information and / or GPS positioning information, and start to reduce its altitude. For example, after the hybrid-wing unmanned aerial vehicle enters the preset recovery airspace, it can send a recovery instruction message to the autonomous recovery platform and establish a communication link between the autonomous recovery platform and the hybrid-wing unmanned aerial vehicle; after receiving the recovery instruction message sent by the hybrid-wing unmanned aerial vehicle, the autonomous recovery platform can start to perform image acquisition on the hybrid-wing unmanned aerial vehicle. Or, it can also be that after the autonomous recovery platform identifies a drone object in the preset recovery airspace, it first establishes a communication link with the drone object, performs drone identity authentication, and after the identity authentication is successful, triggers the start of image acquisition on the hybrid-wing unmanned aerial vehicle.

[0085] In step S12, at least one frame of unmanned aerial vehicle image containing a cooperation mark is identified, and preferred cooperation marks are selected based on the distortion degree of the cooperation marks in the unmanned aerial vehicle image.

[0086] Exemplarily, the autonomous recovery platform can control a binocular camera to collect multiple frames of drone images of the hybrid-wing drone to be recovered, and perform image recognition processing on each drone image respectively to determine whether there is a cooperation mark in the current drone image. If it is determined that there is a cooperation mark in the image, it can be used for subsequent screening of preferred cooperation marks.

[0087] Exemplarily, the recognition of at least one frame of drone image containing a cooperation mark and the screening of preferred cooperation marks based on the distortion degree of the cooperation mark in the drone image include:

[0088] When there are multiple cooperation marks in one frame of optical image, evaluate the distortion degree of the cooperation marks based on the area and perimeter of the cooperation marks in the optical image; according to the evaluation result of the distortion degree of the cooperation marks, screen the preferred cooperation marks.

[0089] Specifically, image recognition can be performed on the optical image and preferred cooperation marks can be screened. Among them, if two or more cooperation marks are recognized in one frame of optical image, for example, there is a bottom cooperation mark and at least one side cooperation mark in one frame of optical image at the same time, the distortion of each cooperation mark can be evaluated through a preset evaluation function. Among them, the first evaluation function can be expressed as:

[0090]

[0091] Among them, is the area of the cooperation mark detected by the camera in the image, is the perimeter of the cooperation mark detected by the camera in the image, is the area of the standard square under this perimeter.

[0092] Based on this formula, the closer the value of is to 1, the smaller the degree of tangential distortion of the corresponding cooperation mark. Therefore, for each cooperation mark, the one with the largest value of can be selected and configured as the preferred cooperation mark. the largest value of and configured as the preferred cooperation mark.

[0093] Exemplarily, the recognition of at least one frame of drone image containing a cooperation mark and the screening of preferred cooperation marks based on the distortion degree of the cooperation mark in the drone image include:

[0094] When the same cooperation mark is included in multiple frames of drone images, evaluate the distortion degree of the cooperation mark according to the resolution of the optical image and the position information of the center of the cooperation mark in the image; according to the evaluation result of the distortion degree of the cooperation mark, determine the preferred cooperation mark and configure the corresponding optical image for calculating the current position information of the hybrid-wing drone.

[0095] Specifically, if the same cooperation identifier is detected in multiple frames of images, the images can be selected through an evaluation function. Specifically, the second evaluation function can be expressed as:

[0096]

[0097] Wherein, and are the resolutions of the acquired images respectively, and are the centers of the cooperation identifier.

[0098] Based on this evaluation function, the smaller the value, the closer the center of the detected cooperation identifier is to the center of the image, indicating a smaller possibility of radial distortion. Therefore, the image with the smallest value can be selected as the initial image for subsequent calculation of the current position information of the hybrid-wing unmanned aerial vehicle.

[0099] For example, if multiple frames of optical images are acquired simultaneously and each frame of optical image contains multiple cooperation identifiers, the above two methods can be used simultaneously to screen the preferred cooperation identifiers and the subsequent images for calculating the current position information.

[0100] In step S13, the hybrid-wing unmanned aerial vehicle acquires the current unmanned aerial vehicle image containing the preferred cooperation identifier, and determines the current position information of the hybrid-wing unmanned aerial vehicle according to the current unmanned aerial vehicle image.

[0101] Exemplarily, after determining the preferred cooperation identifier, the unmanned aerial vehicle image can be re-acquired for the preferred cooperation identifier and used for calculating the current position information. Or, the already acquired unmanned aerial vehicle image containing the preferred cooperation identifier can also be used for calculating the current position information.

[0102] Exemplarily, the determining the current position information of the hybrid-wing unmanned aerial vehicle according to the current unmanned aerial vehicle image includes:

[0103] Step S21, determining the conversion relationship between the pixel coordinate system and the camera coordinate system of the optical image according to the camera internal parameters;

[0104] Step S22, based on the conversion relationship, converting the pixel coordinates of each corner of the preferred cooperation identifier in the pixel coordinate system into the corresponding three-dimensional point cloud coordinates in the camera coordinate system;

[0105] Step S23, when the preferred cooperation identifier is the bottom cooperation identifier, configuring the positioning coordinates of the preferred cooperation identifier in the camera coordinate system according to the mean value of the three-dimensional point cloud coordinates of each corner; or,

[0106] Step S24, when the preferred cooperation identifier is the side cooperation identifier, convert the mean value of the three-dimensional point cloud coordinates of each corner of the preferred cooperation identifier according to the preset conversion matrix between the side cooperation identifier and the bottom cooperation identifier to determine the positioning coordinates of the preferred cooperation identifier; wherein, the positioning coordinates are the central coordinates of the preferred cooperation identifier.

[0107] Step S25, based on the rotation matrix between the camera coordinate system and the platform coordinate system, convert the positioning coordinates in the camera coordinate system into the relative position coordinates in the platform coordinate system and configure them as the current position information of the hybrid-wing unmanned aerial vehicle.

[0108] Exemplarily, the rotation matrix between the camera coordinate system and the platform coordinate system is determined based on the product of the motion rotation matrix between two consecutive frames of the image and the platform coordinate system rotation matrix.

[0109] Specifically, it can be to calculate the positioning coordinates by using the depth image containing the preferred cooperation identifier; obtain the relative position coordinates of the hybrid-wing unmanned aerial vehicle through information fusion; and continuously predict its relative position at the next moment according to the motion trajectory of the hybrid-wing unmanned aerial vehicle.

[0110] Specifically, first, taking the coordinate system of the binocular camera as a reference, the pixels in the known depth image are , and the corresponding depth map projection coordinates are , and the overall projection coordinates are , and using the camera internal parameters, conversion can be performed, and the formula is expressed as:

[0111]

[0112]

[0113] Among them, , represent the pixel coordinates of the optical center in the RGB optical image; , are the focal lengths of the camera in the axis and the axis directions respectively.

[0114] Then, use the three-dimensional point cloud coordinates of the cooperation identifier in the camera coordinate system to calculate the positioning coordinates in the camera coordinate system. Specifically, convert the pixel coordinates , , , of the four corners of the detected cooperation identifier into three-dimensional point cloud coordinates , , , , and the formula can be expressed as:

[0115]

[0116] That is, the position coordinates of the center point of the preferred cooperation identifier can be configured as the position coordinates of the hybrid-wing unmanned aerial vehicle.

[0117] In addition, for the side cooperation identifier, the positioning coordinates in the camera coordinate system can be obtained through conversion , and the formula can be expressed as:

[0118]

[0119] where is the transformation matrix between the side cooperation identifier and the bottom cooperation identifier, is the center coordinate of the cooperation identifier.

[0120] Specifically, the relative position coordinates of the hybrid-wing unmanned aerial vehicle are obtained through information fusion, and the rotation matrix is used to convert the positioning coordinates in the camera coordinate system into the relative position coordinates in the platform coordinate system, and the formula can be expressed as:

[0121]

[0122] where the rotation matrix includes the motion rotation matrix between two consecutive frames of the image and the rotation matrix of the platform coordinate system, and the formula includes:

[0123]

[0124] In addition, in some exemplary embodiments, an attitude sensor may also be provided on the autonomous recovery platform for collecting the attitude information of the autonomous recovery platform in real time. According to the collected attitude information, motion compensation can be performed on the positioning information at the corresponding moment to reduce the interference caused by the disturbance of the sea surface water body to the positioning.

[0125] In step S14, based on the current position information, the motion trajectory of the hybrid-wing unmanned aerial vehicle is predicted to obtain the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment.

[0126] Exemplarily, according to the motion trajectory of the hybrid-wing unmanned aerial vehicle, its relative position at the next moment is continuously predicted, and a target tracking algorithm based on motion prediction, such as the Kalman filter algorithm, can be used.

[0127] Specifically, the predicting the motion trajectory of the hybrid-wing unmanned aerial vehicle based on the current position information to obtain the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment includes:

[0128] Define the system state vector based on the predicted physical quantities of the hybrid-wing unmanned aircraft and initialize it; wherein, the physical quantities include at least one of position coordinates, velocity, and acceleration; the initialization includes: initializing the state estimation and initializing the error covariance matrix.

[0129] At each time , based on the posterior state estimation value at the time, predict the prior state estimation value at the current time , expressed as:

[0130]

[0131] wherein, represents the state transition matrix, represents the control matrix, represents the control input at the time; the predicted error covariance matrix at the current represents the estimation of the uncertainty of the predicted state;

[0132] Use the measurement value at the current time , the measurement matrix and the measurement error covariance matrix to calculate the Kalman gain coefficient , and update the posterior estimation value at the time, expressed as:

[0133]

[0134]

[0135] wherein, represents the transpose of the measurement matrix, represents the estimation of the uncertainty of the state estimation , represents the inverse matrix of the measurement error covariance matrix and the predicted error covariance matrix;

[0136] And, update the error covariance matrix, expressed as:

[0137]

[0138] wherein, represents the estimation of the uncertainty of the state estimation ;

[0139] Repeat the above steps iteratively using the currently obtained measurement values to update the state estimate of the target and configure the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment.

[0140] For example, the system state vector can be defined and initialized first. Among them, the state vector of the system can be defined according to the predicted physical quantities of the target (i.e., the hybrid-wing unmanned aerial vehicle), which can include position coordinates , velocity , acceleration and other physical parameters.

[0141] For initialization, it includes:

[0142] 1) Initialize the state estimate: Assign an initial value to the initial state estimate .

[0143] 2) Initialize the error covariance: Assign an initial value to the initial error covariance .

[0144] In the prediction stage, at each time , based on the posterior state estimate value at time , predict the prior state estimate value at the current time. The prediction process can be expressed as:

[0145]

[0146] where represents the state transition matrix, represents the control matrix, represents the control input at time

[0147] The predicted error covariance matrix at the current time represents the estimate of the uncertainty of the predicted state, and the formula can be expressed as:

[0148]

[0149] where represents the predicted error covariance matrix at time , represents the transpose of the state transition matrix,

[0150] At the update time, use the measurement value at the current , the measurement matrix and the measurement error covariance matrix to calculate the Kalman gain coefficient , and update the posteriori estimate value at the moment , which is expressed as:

[0151]

[0152]

[0153] Among them, represents the transpose of the measurement matrix, represents the estimation of the state estimation uncertainty, represents the inverse matrix of the measurement error covariance matrix and the prediction error covariance matrix.

[0154] At the same time, for the iterative calculation of the next moment, the covariance matrix of the error is updated, and the formula is expressed as:

[0155]

[0156] Among them, represents the estimation of the state estimation uncertainty.

[0157] During the iterative prediction update process, the above steps are continuously repeated using the currently obtained measurement values (the position information of the hybrid-wing unmanned aerial vehicle), and the state estimation of the target can be continuously updated, that is, the position estimation of the hybrid-wing unmanned aerial vehicle at the next moment.

[0158] In step S15, according to the position estimation, an autonomous guidance control instruction for the hybrid-wing unmanned aerial vehicle is configured to implement closed-loop control of the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands at a predetermined position; wherein, the autonomous guidance control instruction includes: a position adjustment instruction for at least one of the x-axis direction, y-axis direction, and z-axis direction.

[0159] Exemplarily, according to the currently predicted relative position coordinates (i.e., the position estimation), closed-loop control is performed on the hybrid-wing unmanned aerial vehicle, and it is autonomously guided to perform horizontal and vertical position adjustments and altitude descent (i.e., the x-axis direction, y-axis direction, and z-axis direction) until the hybrid-wing unmanned aerial vehicle lands at a predetermined position.

[0160] Exemplarily, the configuring of the autonomous guidance control instruction for the hybrid-wing unmanned aerial vehicle according to the position estimation to implement closed-loop control of the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands at a predetermined position includes:

[0161] Using the PID closed-loop control method to decouple the adjustment of the x-axis direction, y-axis direction, and z-axis direction of the hybrid-wing unmanned aerial vehicle, which is expressed as:

[0162]

[0163] Among them, is the control output, is the deviation between the measured value and the control value, is the proportionality coefficient, is the integral time coefficient, is the derivative time coefficient.

[0164] Specifically, the PID closed-loop control algorithm is used to decouple the control of the lateral position, longitudinal position, and altitude of the hybrid-wing unmanned aerial vehicle. The closed-loop control has a feedback mechanism, and the output of the system can participate in the control, avoiding the potential risk of unstable guidance in open-loop control and ensuring the reliability of the system.

[0165] Exemplarily, the method further includes: after identifying that the hybrid-wing unmanned aerial vehicle touches the autonomous recovery platform, the autonomous recovery platform controls the mechanical clamping rod to directly contact the landing gear of the hybrid-wing unmanned aerial vehicle, so as to finely adjust the position of the hybrid-wing unmanned aerial vehicle to a predetermined landing point, and lock the landing gear of the hybrid-wing unmanned aerial vehicle by using the mechanical clamping rod.

[0166] Specifically, after the hybrid-wing unmanned aerial vehicle touches the recovery platform, the platform will further finely adjust its position by using the mechanical clamping rod and lock it. Specifically, after the hybrid-wing unmanned aerial vehicle touches the platform, the recovery platform will control the mechanical clamping rod to directly contact the landing gear of the hybrid-wing unmanned aerial vehicle, finely adjust its position to a predetermined landing point, and lock the landing gear of the hybrid-wing unmanned aerial vehicle by using the mechanical clamping rod.

[0167] Exemplarily, during the process of guiding the hybrid-wing unmanned aerial vehicle to land on the autonomous recovery platform, it is also possible to judge the distortion degree of the cooperation mark in the collected optical image, that is, to judge the distortion degree of the preferred cooperation mark in the current image containing the preferred cooperation mark; if the distortion degree is greater than the preset threshold, the update of the preferred cooperation mark can be triggered. Or, if the preferred cooperation mark cannot be recognized in the real-time collected unmanned aerial vehicle image during the closed-loop control process, the update of the preferred cooperation mark can be triggered, and the updated preferred cooperation mark is used for image collection and position estimation.

[0168] Exemplarily, referring to Figure 2As shown, the blended-wing unmanned aerial vehicle returns to the recovery airspace of the recovery platform based on Beidou / GPS positioning information. The recovery platform uses binocular cameras to continuously acquire the cooperative markers installed at the bottom and sides of the blended-wing unmanned aerial vehicle in real time, selects the most suitable cooperative marker for detection, and continuously collects its optical image and depth image. Calculate the positioning coordinates for the optical image and depth image of the preferred cooperative marker; obtain the relative position coordinates of the blended-wing unmanned aerial vehicle through information fusion; continuously predict its relative position at the next moment based on the movement trajectory of the blended-wing unmanned aerial vehicle; the recovery platform performs closed-loop control on the blended-wing unmanned aerial vehicle based on the predicted relative position coordinates, autonomously guides it to adjust its horizontal and vertical positions, and descend in altitude until the blended-wing unmanned aerial vehicle lands at a predetermined position. After the blended-wing unmanned aerial vehicle contacts the recovery platform, the platform will use mechanical clamping rods to further finely adjust its position and lock it. Among them, the cooperative markers collected in the image can be evaluated, and the optimal marker can be dynamically selected for continuous collection. When calculating the position coordinates of the blended-wing unmanned aerial vehicle using the preferred cooperative marker, the depth coordinates can be solved by the parallax geometry method according to the position of the center of the cooperative marker corresponding to the depth image; the positioning coordinates in the camera coordinate system can be solved using pixel relationships; the positioning coordinates in the camera coordinate system are converted into relative positioning coordinates using coordinate transformation; the motion compensation of the positioning information is combined with the angle data of the attitude sensor of the recovery platform to reduce the interference caused by the disturbance of the sea surface water body to the positioning.

[0169] The method for recovering a blended-wing unmanned aerial vehicle (UAV) on the sea surface provided by the present invention uses far-end and near-end binocular cameras with different elevation angles to dynamically select the optimal identification mark from the cooperative identification marks installed at different positions on the fuselage of the blended-wing UAV, and continuously collect its optical images and depth images; uses the collected image data to calculate the relative positioning information of the blended-wing UAV, compensates the positioning information for motion based on the angle data of the attitude sensor of the recovery platform, and continuously predicts its relative position at the next moment according to the motion trajectory of the blended-wing UAV; autonomously guides and controls the blended-wing UAV to perform horizontal and vertical position adjustments and altitude descent based on the predicted relative position information until the predetermined landing position. This solution, based on vision algorithms and control strategies, realizes the autonomous guidance and recovery of the blended-wing UAV under sea surface conditions, reduces human intervention, and improves the autonomy of the recovery process. Using image data for positioning the blended-wing UAV is fast and accurate, and can quickly and reliably respond to the motion of the blended-wing UAV. Compared with the existing recovery method that installs cooperative identification marks on the recovery platform and uses the UAV to actively identify and position, the recovery method provided by the present invention installs cameras on the recovery platform, further reducing the load requirement on the UAV, enabling it to carry more devices to improve its working ability. At the same time, the recovery method provided by the present invention avoids the risk of the cooperative identification marks being covered by sea water splashes. In addition, through trajectory prediction and decoupled control strategies, the error between the actual landing point and the predetermined landing point of the blended-wing UAV can be reduced, the complexity of the system can be reduced, the response performance of the system can be optimized, and the success rate of recovering the blended-wing UAV on the sea surface can be improved. The autonomous recovery process ensures its recovery efficiency.

[0170] Exemplarily, the method of the present invention utilizes the cooperative identification marks carried by the blended-wing UAV, uses the cameras installed on the recovery platform, and based on vision algorithms, real-time positions the blended-wing UAV, and can autonomously complete the recovery of the blended-wing UAV on the sea surface, reducing the load requirement of the blended-wing UAV and improving the success rate and efficiency of recovery while reducing the need for manual intervention. During the recovery process, the optical information and depth information of the cooperative identification marks carried by the blended-wing UAV are obtained in real time, and the relative coordinates of the blended-wing UAV are calculated through vision algorithms and information fusion, ensuring the accuracy of the recovery; by predicting the motion trajectory of the blended-wing UAV in real time and performing closed-loop control based on this, it can respond more accurately to the dynamic changes of the blended-wing UAV, avoiding potential risks such as inaccurate landing points and too fast landing speeds during the recovery process, and improving the reliability of the recovery.

[0171] In this exemplary embodiment, a hybrid-wing unmanned aerial vehicle (UAV) sea surface autonomous recovery system based on a vision algorithm is provided. Depending on a remotely operated vehicle (ROV), it autonomously recovers, replenishes energy, and carries the hybrid-wing UAV on the sea surface to solve the technical problem that the existing sea surface recovery method for hybrid-wing UAVs requires manual intervention, resulting in low efficiency and safety. Refer to Figure 3 As shown, the system includes: an ROV subsystem 303 of the remotely operated vehicle, and an autonomous recovery system 302 provided on the ROV subsystem 303; the autonomous recovery system 302 is used to receive the hybrid-wing UAV 301.

[0172] Refer to Figure 4 As shown, the autonomous recovery system includes: a landing platform 2 for carrying the hybrid-wing UAV; an optical detection module 1 provided at the edge of the landing platform 2 for collecting optical and depth images at different angles and heights; a correction and locking module 3 provided on the landing platform 2 for locking the hybrid-wing UAV; an underwater wireless charging module 4 provided at the bottom of the landing platform 2 for performing fast contactless charging on the hybrid-wing UAV at close range; and an integrated control module 5 provided at the bottom of the landing platform 2 for performing real-time processing on data and controlling the working states of each module.

[0173] Exemplarily, refer to Figure 4 As shown, the landing platform 2 includes: a platform bracket 23; a V-shaped passive guiding structure symmetrically provided on the platform bracket 23; wherein, the center distance between the V-shaped grooves of the passive guiding structure matches the distance between the skid-type landing gears of the hybrid-wing UAV; a platform support 24 provided within the frame structure of the platform bracket 23 for providing support for the V-shaped passive guiding structure.

[0174] Specifically, the optical detection module 1 is composed of a distal binocular camera 11 and a proximal binocular camera 12 for collecting optical and depth images at different angles and heights. The distal binocular camera 11 and the proximal binocular camera 12 can be symmetrically provided on both sides of the landing platform 2.

[0175] The landing platform 2 is composed of a first V-shaped passive guiding structure 21, a second V-shaped passive guiding structure 22, a platform bracket 23, and a platform support 24. Among them, the center distance between the V-shaped grooves of the passive guiding structures 21 and 22 matches the distance between the skid-type landing gears of the hybrid-wing UAV. Therefore, during the landing process of the hybrid-wing UAV, when the landing gear touches this structure, under the action of gravity, this structure can guide the hybrid-wing UAV to slowly slide towards the plane where the recovery position is located, completing lateral recovery. The platform bracket 23 is the installation frame of the landing platform 2. The platform support 24 provides support for the first V-shaped passive guiding structure 21 and the second V-shaped passive guiding structure 22.

[0176] Exemplarily, referring to Figure 4 and Figure 5 as shown, the correction and locking module 3 is composed of a first right - hand and left - hand screw slider module 31 and a second right - hand and left - hand screw slider module 32 symmetrically distributed on both sides of the landing platform 2, parallel mechanical clamping rods 33, two limit switches (the first limit switch 341 and the second limit switch 342), and a first electromagnet 351 and a second electromagnet 352, which are used to achieve the precise recovery and locking of the hybrid - wing UAV, and establish conditions for subsequent carrying and energy replenishment. As Figure 5 shown in the structural schematic diagram of the second right - hand and left - hand screw slider module 32, it includes a driving motor 321, a coupling 322, a right - hand and left - hand screw 323, a screw base 324, a first slider 3251 and a second slider 3252, and a first mechanical clamping rod connecting piece 3261 and a second mechanical clamping rod connecting piece 3262. When the driving motor 321 rotates forward, it drives the first slider 3251 and the second slider 3252 to move inward, thereby driving the mechanical clamping rods 33 to close inward through the first mechanical clamping rod connecting piece 3261 and the second mechanical clamping rod connecting piece 3262. The first right - hand and left - hand screw slider module 31 works in the same way, so as to correct the recovery position of the hybrid - wing UAV and achieve the longitudinal recovery of the hybrid - wing UAV. The first limit switch 341 is used for protection when the slider moves outward, and the second limit switch 342 is used for protection when the slider moves inward. The first electromagnet 351 and the second electromagnet 352 are used for magnetic attraction locking of the hybrid - wing UAV, so as to meet the conditions for underwater carrying with the ROV.

[0177] Exemplarily, the underwater wireless charging module 4 is mainly composed of a wireless charging transmitter. When the hybrid - wing UAV is recovered to a specified position, the wireless charging receiver under its belly is aligned with the wireless charging transmitter, and energy is transmitted through electromagnetic waves to achieve fast non - contact charging of the hybrid - wing UAV at a short distance.

[0178] Exemplarily, the comprehensive control module 5 may include: a vision computing unit 51, a sensor unit 52, a wireless communication unit 53, an execution controller unit 54, an STM32 data processing unit 55, and a power supply unit 56. The vision computing unit 51 is composed of a Jetson board 511 and a mounting housing 512. The Jetson board 511 is used to process the images collected by the optical detection module 1 and calculate and obtain the optical positioning information of the hybrid-wing unmanned aerial vehicle. The sensor unit 52 is used to obtain the IMU information of the hybrid-wing unmanned aerial vehicle. The wireless communication unit 53 is used for data communication during the autonomous recovery system guiding the hybrid-wing unmanned aerial vehicle to land. The execution controller unit 54 is composed of four-way relays 541, encoders 542 (including a first encoder 5421 and a second encoder 5422), and motor controllers 543 (including a first motor controller 5431 and a second motor controller 5432). Two of the relays 541 are used to control the first electromagnet 351 and the second electromagnet 352, and two are used to control the operation of the first encoder 5421 and the second encoder 5422. The first encoder 5421 is used to control the first limit switch 341 and control the corresponding drive motor through the first motor controller 5431. The second encoder 5422 is used to control the second limit switch 342 and control the corresponding drive motor 321 through the second motor controller 5432. The STM32 data processing unit 55 is used to perform operations such as motion compensation and data fusion on the IMU information obtained by the sensor unit 52 and the optical positioning information calculated by the vision computing unit 51, improve the accuracy and reliability of the positioning information, and use the positioning information to guide the hybrid-wing unmanned aerial vehicle in real time through the wireless communication unit. At the same time, estimate the landing point of the hybrid-wing unmanned aerial vehicle and adjust the mechanical clamping rod 33 through the execution controller unit 54. The power supply unit 56 is composed of a 24V power supply 561 and a 24V to 19V power module 562.

[0179] Exemplarily, during operation, the autonomous sea surface recovery system for the hybrid-wing unmanned aerial vehicle (UAV) is carried on the remotely operated vehicle (ROV). When the hybrid-wing UAV needs energy replenishment after completing the aerial detection mission, the autonomous recovery system receives the landing request through the wireless communication unit 53, and then the STM32 data processing unit 55 sends an instruction to the ROV to lift the autonomous recovery system to the sea surface. The hybrid-wing UAV uses the Beidou / GPS positioning signal to return to the airspace near the autonomous recovery system. The autonomous recovery system identifies the cooperative identification image of the hybrid-wing UAV through the distal binocular camera 11, and the vision computing unit 51 processes the image and obtains the positioning information of the hybrid-wing UAV. The STM32 data processing unit 55 sends an instruction to the hybrid-wing UAV through the wireless communication unit 53 based on this positioning information to guide the hybrid-wing UAV to land on the autonomous recovery system; when the hybrid-wing UAV descends to a lower altitude, it is identified by the proximal binocular camera 12. When the hybrid-wing UAV lands on the landing platform, it slides horizontally under the guidance of the first V-shaped passive guiding structure 21 and the second V-shaped passive guiding structure 22 to complete the horizontal recovery. At the same time, the STM32 data processing unit 55 sends an instruction to the execution controller unit 54 to control the corresponding drive motors to rotate through the first motor controller 5431 and the second motor controller 5432, driving the first positive and negative lead screw slider module 31 and the second positive and negative lead screw slider module 32 to move, realizing the inward closing movement of the mechanical clamping rod 33 to complete the longitudinal recovery of the hybrid-wing UAV. Subsequently, an instruction is sent by the STM32 data processing unit 55, and the first electromagnet 351 and the second electromagnet 352 magnetically lock the hybrid-wing UAV to complete the precise recovery of the hybrid-wing UAV, ensuring the alignment of the wireless charging receiver carried by the hybrid-wing UAV with the wireless charging transmitter of the autonomous recovery system, and realizing contactless energy replenishment of the hybrid-wing UAV through electromagnetic wave energy transfer. The ROV can carry the hybrid-wing UAV underwater for carrying according to the mission requirements. After the energy replenishment is completed, the ROV can float up again to release the hybrid-wing UAV to enable it to continue to perform the detection mission.

[0180] Based on the innovative mechanical structure design, the sea surface recovery system for the hybrid-wing UAV provided by the present invention effectively solves the key problems such as large sea water disturbance, the need for manual intervention assistance, low recovery success rate, and low recovery efficiency when the existing hybrid-wing UAV is recovered on the sea surface by introducing vision algorithms and autonomous guidance control. The mechanical structure is designed for the recovery scenario, simplifying the design of the sea surface recovery mechanism of the hybrid-wing UAV, and thus reducing the load requirement on the underwater ROV. Using a wireless method to replenish energy for the hybrid-wing UAV simplifies the operation process and does not affect the underwater carrying ability of the ROV, improving the autonomy of the overall system and meeting the mission requirements of the cross-domain unmanned system cooperation paradigm.

[0181] It should be noted that the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0182] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more of the above-mentioned modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0183] Specifically, according to the embodiments of the present invention, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present invention include a computer program product, which includes a computer program carried on a storage medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), various functions defined in the system of the present application are executed.

[0184] It should be noted that the storage medium shown in the embodiments of the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any storage medium other than a computer-readable storage medium, and this storage medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0186] The units involved in the embodiments of the present invention can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0187] It should be noted that, on the other hand, the present application also provides a storage medium, which can be included in an electronic device; or it can exist alone without being assembled into the electronic device. The above storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device is caused to implement the methods described in the following embodiments. For example, the electronic device can implement each step of the method as Figure 1 shown.

[0188] In one embodiment, the present application provides a computer program product, including a computer program, which when executed by a processor implements the steps in the above method embodiments.

[0189] In addition, the above drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above drawings do not indicate or limit the time sequence of these processes. Additionally, it is also easy to understand that these processes can be executed synchronously or asynchronously in, for example, multiple modules.

[0190] Those skilled in the art will readily think of other embodiments of the present invention after considering the specification and practicing the invention herein. The present application is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed in the present invention. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the claims.

[0191] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for autonomous recovery of a blended-wing unmanned aerial vehicle on the sea surface based on a vision algorithm, characterized in that, The method includes: When a hybrid-wing unmanned aerial vehicle (UAV) to be recovered is identified within a preset recovery airspace of an autonomous recovery platform, image acquisition is performed on the hybrid-wing UAV to obtain at least one frame of UAV image containing a cooperation identifier; wherein, the UAV image includes an optical image and a corresponding depth image; cooperation identifiers are respectively arranged at different positions on the bottom and side of the hybrid-wing UAV; Identify at least one frame of UAV image containing a cooperation identifier, and screen for preferred cooperation identifiers based on the distortion degree of the cooperation identifier in the UAV image; Acquire a current UAV image of the hybrid-wing UAV containing the preferred cooperation identifier, and determine the current position information of the hybrid-wing UAV according to the current UAV image; Predict the motion trajectory of a hybrid-wing unmanned aerial vehicle (UAV) based on the current position information to obtain the position estimate of the hybrid-wing UAV at the next moment, including: defining a system state vector based on the physical quantities predicted for the hybrid-wing UAV and initializing; wherein, the physical quantities include at least one of position coordinates, velocity, and acceleration; the initialization includes: initializing the state estimate and initializing the error covariance matrix; at each time , based on the posterior state estimate value at the moment predict the prior state estimate value at the current moment ; use the measurement value at the current moment , the measurement matrix and the measurement error covariance matrix to calculate the Kalman gain coefficient , and update the posterior estimate value at the moment ; and, update the error covariance matrix; repeat the above method using the currently obtained measurement values for iteration to update the state estimate of the target and configure it as the position estimate of the hybrid-wing UAV at the next moment; Configure an autonomous guidance control instruction for the hybrid-wing UAV according to the position estimation to be used to achieve closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position; wherein, the autonomous guidance control instruction includes a position adjustment instruction for at least one of the x-axis direction, y-axis direction, and z-axis direction.

2. The method according to claim 1, wherein The identifying at least one frame of UAV image containing a cooperation identifier and screening for preferred cooperation identifiers based on the distortion degree of the cooperation identifier in the UAV image includes: When a frame of optical image contains multiple cooperation identifiers, evaluate the distortion degree of the cooperation identifier based on the area and perimeter of the cooperation identifier in the optical image; Screen for preferred cooperation identifiers according to the evaluation result of the distortion degree of the cooperation identifier.

3. The method according to claim 1 or 2, characterized in that, The identifying at least one frame of UAV image containing a cooperation identifier and screening for preferred cooperation identifiers based on the distortion degree of the cooperation identifier in the UAV image includes: When the same cooperation identifier is included in multiple frames of UAV images, evaluate the distortion degree of the cooperation identifier according to the resolution of the optical image and the position information of the center of the cooperation identifier in the image; Determine the preferred cooperation identifier according to the evaluation result of the distortion degree of the cooperation identifier, and configure the corresponding optical image for calculating the current position information of the hybrid-wing UAV.

4. The method according to claim 1, wherein The determining the current position information of the hybrid-wing UAV according to the current UAV image includes: Determine the conversion relationship between the pixel coordinate system of the depth image and the camera coordinate system according to the camera internal parameters; Based on the conversion relationship, convert the pixel coordinates of each corner of the preferred cooperation identifier in the pixel coordinate system into the corresponding three-dimensional point cloud coordinates in the camera coordinate system; When the preferred cooperation identifier is a bottom cooperation identifier, configure the positioning coordinates of the preferred cooperation identifier in the camera coordinate system according to the mean value of the three-dimensional point cloud coordinates of each corner; or, When the preferred cooperation identifier is a side cooperation identifier, convert the mean value of the three-dimensional point cloud coordinates of each corner of the preferred cooperation identifier according to a preset conversion matrix between the side cooperation identifier and the bottom cooperation identifier to determine the positioning coordinates of the preferred cooperation identifier; wherein, the positioning coordinates are the center coordinates of the preferred cooperation identifier; Based on the rotation matrix between the camera coordinate system and the platform coordinate system, convert the positioning coordinates in the camera coordinate system into the relative position coordinates in the platform coordinate system, and configure them as the current position information of the hybrid-wing UAV.

5. The method according to claim 4, wherein The rotation matrix between the camera coordinate system and the platform coordinate system is determined based on the product of the motion rotation matrix between two consecutive frames of the image and the platform coordinate system rotation matrix.

6. The method according to claim 1, wherein The method further includes: At each time , based on the posterior state estimate value at the moment, predict the prior state estimate value at the current moment , expressed as: Among them, represents the state transition matrix, represents the control matrix, represents the control input at time; the current prediction error covariance matrix at time represents the estimate of the uncertainty of the predicted state; Use the current measurement value at the moment , measurement matrix and measurement error covariance matrix to calculate the Kalman gain coefficient , and update the posterior estimate value at the moment , expressed as: Among them, represents the transpose of the measurement matrix, represents the estimation of the uncertainty of the state estimation, represents the inverse matrix of the measurement error covariance matrix and the prediction error covariance matrix; and updating the error covariance matrix, expressed as: Among them, represents the estimation of the uncertainty of the state estimation, represents the identity matrix.

7. The method according to claim 4, characterized in that Configuring the autonomous guidance control instruction of the hybrid-wing unmanned aerial vehicle according to the position estimation to be used for implementing the closed-loop control of the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands at a predetermined position, including: Using the PID closed-loop control method to decouple the adjustment of the hybrid-wing unmanned aerial vehicle in the x-axis direction, y-axis direction, and z-axis direction, expressed as: Among them, is for controlling the output, is the deviation between the measured value and the control value, is the proportionality coefficient, is the integral time coefficient, is the derivative time coefficient, is the time.

8. The method according to claim 1, characterized in that, The method further includes: After identifying that the hybrid-wing unmanned aerial vehicle touches the autonomous recovery platform, the autonomous recovery platform controls the mechanical clamping rod to directly contact the landing gear of the hybrid-wing unmanned aerial vehicle to finely adjust the position of the hybrid-wing unmanned aerial vehicle to a predetermined landing point and lock the landing gear of the hybrid-wing unmanned aerial vehicle using the mechanical clamping rod.

9. A hybrid-wing UAV sea surface autonomous recovery system based on a vision algorithm, which is applied to the method described in any one of claims 1-8, and is characterized in that, The system includes: a remotely operated underwater vehicle (ROV) subsystem, and an autonomous recovery system provided on the ROV subsystem; The autonomous recovery system includes: A landing platform (2) for carrying the hybrid-wing unmanned aerial vehicle; An optical detection module (1) provided at the edge of the landing platform (2) for collecting optical and depth images at different angles and heights; A correction and locking module (3) provided on the landing platform (2) for locking the hybrid-wing unmanned aerial vehicle; An underwater wireless charging module (4) provided at the bottom of the landing platform (2) for performing fast contactless charging on the hybrid-wing unmanned aerial vehicle at a short distance; An integrated control module (5) provided at the bottom of the landing platform (2) for performing real-time processing of data and controlling the working states of each module.

10. The system according to claim 9, wherein, The landing platform (2) includes: A platform bracket (23); A V-shaped passive guiding structure symmetrically provided on the platform bracket (23); wherein, the center distance of the V-shaped groove of the passive guiding structure matches the distance between the skid-type landing gears of the hybrid-wing unmanned aerial vehicle; A platform support (24) provided within the frame structure of the platform bracket (23) for providing support for the V-shaped passive guiding structure.

Images