Hybrid wing unmanned aerial vehicle sea surface autonomous recovery method and system based on visual algorithm

Abstract

The invention particularly relates to a hybrid wing unmanned aerial vehicle sea surface autonomous recovery method and system based on a visual algorithm, and the method comprises the steps: carrying out the image collection when a to-be-recovered hybrid wing unmanned aerial vehicle is recognized in a preset recovery airspace, and obtaining at least one frame of unmanned aerial vehicle image containing a cooperation identifier; identifying at least one frame of unmanned aerial vehicle image containing the cooperation identifier, and screening a preferred cooperation identifier based on the distortion degree of the cooperation identifier in the unmanned aerial vehicle image; acquiring a current unmanned aerial vehicle image containing the optimal cooperation identifier for the hybrid-wing unmanned aerial vehicle, and determining current position information of the hybrid-wing unmanned aerial vehicle according to the current unmanned aerial vehicle image; performing motion track prediction on the hybrid wing unmanned aerial vehicle based on the current position information, and obtaining position estimation of the hybrid wing unmanned aerial vehicle at the next moment; and configuring an autonomous guidance control instruction of the hybrid-wing unmanned aerial vehicle according to the position estimation so as to realize closed-loop control of the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands to a preset position.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous recovery of sea drones, and particularly to a method and system for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm. Background Art

[0002] In the related art, the high flight speed, high flexibility, and long endurance of drones are the key factors for performing sea tasks. The hybrid-wing drone combines the advantages of fixed-wing drones and rotor drones in terms of speed and flexibility. However, limited by the sea recovery conditions, when the existing skyhook recovery methods are used to recover small drones, there are problems of insufficient reliability; moreover, the volume of the skyhook recovery equipment is relatively large, making it difficult to implement on small unmanned platforms. In addition, the existing net-collision recovery methods are likely to cause damage to the recovery device and the structure of the drone itself; at the same time, the preparation period of the net-collision recovery equipment is relatively long. Therefore, there is a lack of relevant technical solutions for the sea recovery of hybrid-wing drones in the prior art, which also restricts the marine application of hybrid-wing drones.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present invention, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] The present invention provides a method for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm, a system for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm, a computer program product, and a storage medium, thereby being able to overcome the defects existing in the prior art to a certain extent.

[0005] Other features and advantages of the present invention will become apparent through the following detailed description, or will be partially learned through the practice of the present invention.

[0006] According to a first aspect of the present invention, there is provided a method for autonomous recovery of a hybrid-wing drone on the sea surface based on a vision algorithm, the method comprising: When a hybrid-wing drone to be recovered is identified within a preset recovery airspace of an autonomous recovery platform, image acquisition is performed on the hybrid-wing drone to obtain at least one frame of drone image containing a cooperation mark; wherein, the drone image includes an optical image and a corresponding depth image; cooperation marks are respectively arranged at different positions on the bottom and side of the hybrid-wing drone; Identifying at least one frame of drone image containing a cooperation mark, and screening out preferred cooperation marks based on the distortion degree of the cooperation marks in the drone image; Collecting a current drone image of the hybrid-wing drone containing the preferred cooperation mark, and determining the current position information of the hybrid-wing drone according to the current drone image; Predict the motion trajectory of the hybrid-wing unmanned aerial vehicle (UAV) based on the current position information, and obtain the position estimate of the hybrid-wing UAV at the next moment. Configure the autonomous guidance control instruction of the hybrid-wing UAV according to the position estimate to be used for realizing the closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position; wherein, the autonomous guidance control instruction includes: position adjustment instructions for at least one of the x-axis direction, y-axis direction, and z-axis direction.

[0007] In some exemplary embodiments, the identifying at least one frame of UAV images containing cooperation identifiers and screening preferred cooperation identifiers based on the distortion degree of the cooperation identifiers in the UAV images includes: When a frame of optical image contains multiple cooperation identifiers, evaluate the distortion degree of the cooperation identifiers based on the area and perimeter of the cooperation identifiers in the optical image. Screen the preferred cooperation identifiers according to the evaluation result of the distortion degree of the cooperation identifiers.

[0008] In some exemplary embodiments, the identifying at least one frame of UAV images containing cooperation identifiers and screening preferred cooperation identifiers based on the distortion degree of the cooperation identifiers in the UAV images includes: When the same cooperation identifier is included in multiple frames of UAV images, evaluate the distortion degree of the cooperation identifier according to the resolution of the optical image and the position information of the center of the cooperation identifier in the image. Determine the preferred cooperation identifier according to the evaluation result of the distortion degree of the cooperation identifier, and configure the corresponding optical image for calculating the current position information of the hybrid-wing UAV.

[0009] In some exemplary embodiments, the determining the current position information of the hybrid-wing UAV according to the current UAV image includes: Determine the conversion relationship between the pixel coordinate system and the camera coordinate system of the optical image according to the camera internal parameters. Based on the conversion relationship, convert the pixel coordinates of each corner of the preferred cooperation identifier in the pixel coordinate system into the corresponding three-dimensional point cloud coordinates in the camera coordinate system. When the preferred cooperation identifier is the bottom cooperation identifier, configure the positioning coordinates of the preferred cooperation identifier in the camera coordinate system according to the mean value of the three-dimensional point cloud coordinates of each corner; or, When the preferred cooperation identifier is the side cooperation identifier, convert the mean value of the three-dimensional point cloud coordinates of each corner of the preferred cooperation identifier according to the preset conversion matrix between the side cooperation identifier and the bottom cooperation identifier to determine the positioning coordinates of the preferred cooperation identifier; wherein, the positioning coordinates are the center coordinates of the preferred cooperation identifier. Based on the rotation matrix between the camera coordinate system and the platform coordinate system, the positioning coordinates in the camera coordinate system are converted into the relative position coordinates in the platform coordinate system and configured as the current position information of the hybrid-wing unmanned aerial vehicle.

[0010] In some exemplary embodiments, the rotation matrix between the camera coordinate system and the platform coordinate system is determined based on the product of the motion rotation matrix between two consecutive frames of the image and the platform coordinate system rotation matrix.

[0011] In some exemplary embodiments, the motion trajectory prediction of the hybrid-wing unmanned aerial vehicle based on the current position information to obtain the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment includes: Defining a system state vector based on the physical quantities predicted for the hybrid-wing unmanned aerial vehicle and initializing it; wherein, the physical quantities include at least one of position coordinates, velocity, and acceleration; the initialization includes: initializing the state estimate and initializing the error covariance matrix; At each time , based on the posterior state estimate value at the moment predict the prior state estimate value at the current moment , expressed as:

[0012] wherein, represents the state transition matrix, represents the control matrix, represents the control input at the moment; the predicted error covariance matrix at the current moment represents the estimate of the uncertainty of the predicted state; Using the measurement value at the current moment , the measurement matrix and the measurement error covariance matrix calculate the Kalman gain coefficient , and update the posterior estimate value at the moment , expressed as:

[0013]

[0014] wherein, represents the transpose of the measurement matrix, represents the estimate of the uncertainty of the state estimate , represents the inverse matrix of the measurement error covariance matrix and the predicted error covariance matrix; And, update the covariance matrix of the error, expressed as:

[0015] Wherein, represents the estimation of the state estimation uncertainty; Repeat the above steps iteratively using the currently obtained measurement values to update the state estimation of the target, and configure it as the position estimation of the hybrid-wing unmanned aerial vehicle at the next moment.

[0016] In some exemplary embodiments, configuring the autonomous guidance control instruction of the hybrid-wing unmanned aerial vehicle according to the position estimation to be used for realizing the closed-loop control of the hybrid-wing unmanned aerial vehicle until the hybrid-wing unmanned aerial vehicle lands at a predetermined position includes: Using the PID closed-loop control method to decouple the adjustment of the hybrid-wing unmanned aerial vehicle in the x-axis direction, y-axis direction, and z-axis direction, expressed as:

[0017] Wherein, is the control output, is the deviation between the measurement value and the control value, is the proportionality coefficient, is the integral time coefficient, is the differential time coefficient.

[0018] In some exemplary embodiments, the method further includes: After identifying that the hybrid-wing unmanned aerial vehicle touches the autonomous recovery platform, the autonomous recovery platform controls the mechanical clamping rod to directly contact the landing gear of the hybrid-wing unmanned aerial vehicle to finely adjust the position of the hybrid-wing unmanned aerial vehicle to a predetermined landing point, and locks the landing gear of the hybrid-wing unmanned aerial vehicle using the mechanical clamping rod.

[0019] According to the second aspect of the present invention, there is provided a hybrid-wing unmanned aerial vehicle sea surface autonomous recovery system based on a vision algorithm, which is applied to the method in the above embodiments; the system includes: a remotely operated underwater vehicle (ROV) subsystem, and an autonomous recovery system provided on the ROV subsystem; The autonomous recovery system includes: A landing platform for carrying the hybrid-wing unmanned aerial vehicle; An optical detection module provided at the edge of the landing platform for collecting optical and depth images at different angles and heights; A correction and locking module provided on the landing platform for locking the hybrid-wing unmanned aerial vehicle; An underwater wireless charging module provided at the bottom of the landing platform for performing short-range and fast contactless charging on the hybrid-wing unmanned aerial vehicle; The comprehensive control module is arranged at the bottom of the landing platform and is used for real-time processing of data and controlling the working states of each module.

[0020] In some exemplary embodiments, the landing platform includes: A platform support; A V-shaped passive guiding structure symmetrically arranged on the platform support; wherein, the center distance between the V-shaped grooves of the passive guiding structure matches the distance between the skid-type landing gears of the hybrid-wing unmanned aerial vehicle; A platform support is arranged within the frame structure of the platform support and is used for providing support for the V-shaped passive guiding structure.

[0021] According to a third aspect of the present invention, there is provided a computer program product having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm is implemented.

[0022] According to a fourth aspect of the present invention, there is provided a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm is implemented.

[0023] According to a fifth aspect of the present invention, there is provided an electronic device, including: A processor and a memory; Wherein, the memory is used for storing executable instructions of the processor; the processor is configured to implement the above-mentioned method for autonomous recovery of a hybrid-wing unmanned aerial vehicle on the sea surface based on a vision algorithm when executing the executable instructions.

[0024] The method for autonomous recovery of a hybrid-wing unmanned aerial vehicle (UAV) on the sea surface based on a vision algorithm provided by an embodiment of the present invention collects corresponding optical images and depth images after the hybrid-wing UAV enters a preset airspace of an autonomous recovery platform, screens out preferred cooperation identifiers for subsequent calculations by recognizing the optical images, then calculates the current position information of the hybrid-wing UAV using the depth images containing the preferred cooperation identifiers, predicts the motion trajectory of the hybrid-wing UAV based on the current position information to obtain the position estimate at the next moment, and further performs closed-loop control on the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position. This solution realizes the recovery control strategy for the hybrid-wing UAV based on the vision algorithm, achieves the autonomous guided recovery of the hybrid-wing UAV under sea surface conditions, reduces the intervention of manual labor, and improves the autonomy of the recovery process. Using image data for positioning the hybrid-wing UAV is fast and accurate, and can quickly and reliably respond to the motion of the hybrid-wing UAV. Through trajectory prediction and control strategies, the error between the actual landing point and the predetermined landing point of the hybrid-wing UAV can be reduced, and the success rate of recovering the hybrid-wing UAV on the sea surface can be improved. The autonomous recovery process ensures its recovery efficiency.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. Brief Description of the Drawings

[0026] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0027] Figure 1 A schematic diagram schematically showing an autonomous recovery method for a hybrid-wing UAV on the sea surface based on a vision algorithm according to an exemplary embodiment of the present invention; Figure 2 A schematic diagram schematically showing the flow of an autonomous recovery method for a hybrid-wing UAV on the sea surface based on a vision algorithm according to an exemplary embodiment of the present invention; Figure 3 A schematic diagram schematically showing the overall architecture of an autonomous recovery system for a hybrid-wing UAV on the sea surface according to an exemplary embodiment of the present invention; Figure 4 A schematic diagram schematically showing the explosion of an autonomous recovery system for a hybrid-wing UAV on the sea surface according to an exemplary embodiment of the present invention; Figure 5 A schematic diagram schematically showing the structure of a positive and negative thread screw rod slider module of a correction and locking module according to an exemplary embodiment of the present invention; Figure 6Schematically show a schematic diagram of the composition of a comprehensive control module according to an exemplary embodiment of the present invention.

[0028] Among them, the reference numerals include: optical detection module 1, distal binocular camera 11, proximal binocular camera 12, landing platform 2, first V-shaped passive guiding structure 21, second V-shaped passive guiding structure 22, platform bracket 23, platform support 24, correction and locking module 3, first positive and negative thread screw slider module 31, second positive and negative thread screw slider module 32, driving motor 321, coupling 322, positive and negative thread screw 323, screw base 324, first slider 3251, second slider 3252, first mechanical clamp rod connecting piece 3261, second mechanical clamp rod connecting piece 3262, mechanical clamp rod 33, first limit switch 341, second limit switch 342, first electromagnet 351, second electromagnet 352, underwater wireless charging module 4, comprehensive control module 5, vision computing unit 51, sensor unit 52, wireless communication unit 53, execution controller unit 54, STM32 data processing unit 55, power supply unit 56, Jetson board 511, installation housing 512, four-way relay 541, encoder 542 includes: first encoder 5421 and second encoder 5422, motor controller 543 includes: first motor controller 5431 and second motor controller 5432, 24V power supply 561, 24V to 19V power module 562; hybrid wing unmanned aerial vehicle 301, sea surface autonomous recovery system 302, ROV subsystem 303. Detailed implementation

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this invention will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0030] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0031] In the related art, compared with land recovery, the sea surface recovery of a hybrid-wing unmanned aerial vehicle (UAV) faces various difficulties. The UAV recovery system that requires manual intervention not only limits the usage scenarios of the system, but also reduces the recovery success rate and efficiency in complex environments such as sea surface wind and wave interference, making it difficult to meet the requirements of such cross-domain unmanned system cooperation paradigms.

[0032] Aiming at the shortcomings and deficiencies of the existing technology, in this exemplary embodiment, a method for autonomous sea surface recovery of a hybrid-wing UAV based on a vision algorithm is provided. This method solves the technical problem that the existing sea surface recovery method of a hybrid-wing UAV requires manual intervention, resulting in low efficiency and safety. Refer to Figure 1 As shown in the figure, the method may include the following steps: Step S11: When a hybrid-wing UAV to be recovered is identified within the preset recovery airspace of the autonomous recovery platform, image acquisition is performed on the hybrid-wing UAV to obtain at least one frame of UAV image containing a cooperation mark; wherein, the UAV image includes an optical image and a corresponding depth image; cooperation marks are respectively arranged at different positions on the bottom and side of the hybrid-wing UAV. Step S12: Identify at least one frame of UAV image containing a cooperation mark, and screen out the preferred cooperation mark based on the distortion degree of the cooperation mark in the UAV image. Step S13: Acquire the current UAV image containing the preferred cooperation mark for the hybrid-wing UAV, and determine the current position information of the hybrid-wing UAV according to the current UAV image. Step S14: Predict the motion trajectory of the hybrid-wing UAV based on the current position information to obtain the position estimate of the hybrid-wing UAV at the next moment. Step S15: Configure the autonomous guidance control instruction for the hybrid-wing UAV according to the position estimate to realize the closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position; wherein, the autonomous guidance control instruction includes at least one of the position adjustment instructions in the x-axis direction, y-axis direction, and z-axis direction.

[0033] Next, each step of the method for autonomous sea surface recovery of a hybrid-wing UAV based on a vision algorithm in this exemplary embodiment will be described in more detail with reference to the accompanying drawings and embodiments.

[0034] In step S11, when a hybrid-wing UAV to be recovered is identified within the preset recovery airspace of the autonomous recovery platform, image acquisition is performed on the hybrid-wing UAV to obtain at least one frame of UAV image containing a cooperation mark; wherein, the UAV image includes an optical image and a corresponding depth image; cooperation marks are respectively arranged at different positions on the bottom and side of the hybrid-wing UAV.

[0035] Exemplarily, for a hybrid-wing unmanned aerial vehicle (UAV), cooperation identifiers can be respectively set at different positions such as the bottom and sides of the UAV body. For example, a cooperation identifier can be set at the bottom of the UAV body, and a cooperation identifier can be set on each of the four side walls of the UAV body. Among them, the cooperation identifier can be an Aruco code. Each Aruco code set on the UAV body has a unique ID, and the IDs of the Aruco codes are different from each other. By installing Aruco codes at different positions on the hybrid-wing UAV and using them for detection and positioning, it is ensured that during the entire process of UAV recovery, at least one cooperation identifier can be collected and recognized at each moment.

[0036] For example, at least one set of binocular cameras can be set on the autonomous recovery platform, and the binocular cameras are used to collect optical images and corresponding depth images of the hybrid-wing UAV.

[0037] Among them, the optical image can be an RGB image.

[0038] Specifically, the hybrid-wing UAV can be controlled to fly into the preset recovery airspace of the autonomous recovery platform using Beidou positioning information and / or GPS positioning information, and then start to reduce its altitude. For example, after the hybrid-wing UAV enters the preset recovery airspace, it can send a recovery instruction message to the autonomous recovery platform and establish a communication link between the autonomous recovery platform and the hybrid-wing UAV; after receiving the recovery instruction message sent by the hybrid-wing UAV, the autonomous recovery platform can start to collect images of the hybrid-wing UAV. Or, it can also be that after the autonomous recovery platform recognizes the appearance of a UAV object in the preset recovery airspace, it first establishes a communication link with the UAV object, conducts UAV identity authentication, and after the identity authentication is successful, triggers the start of collecting images of the hybrid-wing UAV.

[0039] In step S12, at least one frame of the UAV image containing the cooperation identifier is recognized, and the preferred cooperation identifier is screened based on the distortion degree of the cooperation identifier in the UAV image.

[0040] Exemplarily, the autonomous recovery platform can control the binocular cameras to collect multiple frames of UAV images of the hybrid-wing UAV to be recovered, and respectively perform image recognition processing on each UAV image to determine whether there is a cooperation identifier in the current UAV image. If it is determined that there is a cooperation identifier in the image, it can be used for subsequent screening of the preferred cooperation identifier.

[0041] Exemplarily, the recognition of at least one frame of the UAV image containing the cooperation identifier and the screening of the preferred cooperation identifier based on the distortion degree of the cooperation identifier in the UAV image include: When multiple cooperative identifiers are included in an optical image, the distortion degree of the cooperative identifiers is evaluated based on the area and perimeter of the cooperative identifiers in the optical image; according to the evaluation result of the distortion degree of the cooperative identifiers, preferred cooperative identifiers are screened.

[0042] Specifically, image recognition can be performed on the optical image, and preferred cooperative identifiers can be screened. Among them, if two or more cooperative identifiers are recognized in one frame of optical image, for example, a bottom cooperative identifier and at least one side cooperative identifier exist simultaneously in one frame of optical image, the distortion of each cooperative identifier can be evaluated through a preset evaluation function. Among them, the first evaluation function can be expressed as:

[0043] Among them, is the area of the cooperative identifier detected by the camera in the image, is the perimeter of the cooperative identifier detected by the camera in the image, is the area of the standard square under this perimeter.

[0044] Based on this formula, the closer the value of is to 1, the smaller the degree of tangential distortion of the corresponding cooperative identifier. Therefore, for each cooperative identifier, the one with the largest value can be selected and configured as the preferred cooperative identifier.

[0045] Exemplarily, the recognition of at least one frame of drone image containing cooperative identifiers and the screening of preferred cooperative identifiers based on the distortion degree of the cooperative identifiers in the drone image include: When the same cooperative identifier is included in multiple frames of drone images, the distortion degree of the cooperative identifier is evaluated according to the resolution of the optical image and the position information of the center of the cooperative identifier in the image; according to the evaluation result of the distortion degree of the cooperative identifier, the preferred cooperative identifier is determined, and the corresponding optical image is configured for calculating the current position information of the hybrid-wing drone.

[0046] Specifically, if the same cooperative identifier is detected in multiple frames of images, the images can be selected through an evaluation function. Specifically, the second evaluation function can be expressed as:

[0047] Among them, 、 are the resolutions of the acquired images respectively, 、 are the centers of the cooperative identifier.

[0048] Based on this evaluation function, The smaller the value is, the closer the detected cooperation identification center is to the image center, which means the lower the possibility of radial distortion. Therefore, the image with the smallest R value can be selected as the initial image for subsequent calculation of the current position information of the hybrid-wing unmanned aerial vehicle.

[0049] For example, if multiple frames of optical images are collected simultaneously and each frame of optical image contains multiple cooperation identifications, the above two methods can be used simultaneously to screen the preferred cooperation identifications and the subsequent images for calculating the current position information.

[0050] In step S13, the hybrid-wing unmanned aerial vehicle collects the current unmanned aerial vehicle image containing the preferred cooperation identification, and determines the current position information of the hybrid-wing unmanned aerial vehicle according to the current unmanned aerial vehicle image.

[0051] Exemplarily, after determining the preferred cooperation identification, the unmanned aerial vehicle image can be recollected for the preferred cooperation identification and used for calculating the current position information. Or, the already collected unmanned aerial vehicle image containing the preferred cooperation identification can also be used for calculating the current position information.

[0052] Exemplarily, the determining the current position information of the hybrid-wing unmanned aerial vehicle according to the current unmanned aerial vehicle image includes: Step S21, determining the conversion relationship between the pixel coordinate system and the camera coordinate system of the optical image according to the camera internal parameters; Step S22, based on the conversion relationship, converting the pixel coordinates of each end angle of the preferred cooperation identification in the pixel coordinate system into the corresponding three-dimensional point cloud coordinates in the camera coordinate system; Step S23, when the preferred cooperation identification is the bottom cooperation identification, configuring the positioning coordinates of the preferred cooperation identification in the camera coordinate system according to the mean value of the three-dimensional point cloud coordinates of each end angle; or, Step S24, when the preferred cooperation identification is the side cooperation identification, converting the mean value of the three-dimensional point cloud coordinates of each end angle of the preferred cooperation identification according to the preset conversion matrix between the side cooperation identification and the bottom cooperation identification to determine the positioning coordinates of the preferred cooperation identification; wherein, the positioning coordinates are the center coordinates of the preferred cooperation identification; Step S25, based on the rotation matrix between the camera coordinate system and the platform coordinate system, converting the positioning coordinates in the camera coordinate system into the relative position coordinates in the platform coordinate system and configuring them as the current position information of the hybrid-wing unmanned aerial vehicle.

[0053] Exemplarily, the rotation matrix between the camera coordinate system and the platform coordinate system is determined based on the product of the motion rotation matrix between two consecutive frames of the image and the platform coordinate system rotation matrix.

[0054] Specifically, it can be to calculate the positioning coordinates using the depth image containing the preferred cooperation identifier; obtain the relative position coordinates of the hybrid-wing UAV through information fusion; and continuously predict its relative position at the next moment according to the motion trajectory of the hybrid-wing UAV.

[0055] Specifically, first, taking the coordinate system of the binocular camera as the reference, the pixels in the depth image are known as , and its corresponding depth map projection coordinates are , and the overall projection coordinates are . Using the camera internal parameters, the conversion can be carried out, and the formula is expressed as:

[0056]

[0057] Among them, , represent the pixel coordinates of the optical center in the RGB optical image; , are the focal lengths of the camera in the axis and the axis directions respectively.

[0058] Then, using the three-dimensional point cloud coordinates of the cooperation identifier in the camera coordinate system, calculate the positioning coordinates in the camera coordinate system. Specifically, convert the detected pixel coordinates , , , of the four corners of the cooperation identifier into three-dimensional point cloud coordinates , , , , and the formula can be expressed as:

[0059] That is, the position coordinates of the center point of the preferred cooperation identifier can be configured as the position coordinates of the hybrid-wing UAV.

[0060] In addition, for the side cooperation identifier, the positioning coordinates in the camera coordinate system can be obtained through conversion, and the formula can be expressed as:

[0061] Among them, is the conversion matrix between the side cooperation identifier and the bottom cooperation identifier, is the center coordinate of the cooperation identifier.

[0062] Specifically, obtain the relative position coordinates of the hybrid-wing UAV through information fusion, and use the rotation matrix to convert the positioning coordinates in the camera coordinate system Convert to the relative position coordinates in the platform coordinate system , the formula can be expressed as:

[0063] Among them, the rotation matrix includes the motion rotation matrix between two consecutive frames of the image and the platform coordinate system rotation matrix , the formula includes:

[0064] In addition, in some exemplary embodiments, an attitude sensor may be further provided on the autonomous recovery platform for collecting the attitude information of the autonomous recovery platform in real time. According to the collected attitude information, motion compensation can be performed on the positioning information at the corresponding moment to reduce the interference caused by the sea surface water disturbance to the positioning.

[0065] In step S14, based on the current position information, the motion trajectory of the hybrid-wing unmanned aerial vehicle is predicted to obtain the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment.

[0066] Exemplarily, according to the motion trajectory of the hybrid-wing unmanned aerial vehicle, its relative position at the next moment is continuously predicted, and a target tracking algorithm based on motion prediction, such as the Kalman filter algorithm, can be used.

[0067] Specifically, the predicting the motion trajectory of the hybrid-wing unmanned aerial vehicle based on the current position information to obtain the position estimate of the hybrid-wing unmanned aerial vehicle at the next moment includes: Defining a system state vector based on the physical quantities predicted for the hybrid-wing unmanned aerial vehicle and initializing it; among them, the physical quantities include at least one of position coordinates, speed, and acceleration; the initialization includes: initializing the state estimate and initializing the error covariance matrix; At each time , based on the posterior state estimate value at the moment predict the prior state estimate value at the current moment , expressed as:

[0068] Among them, represents the state transition matrix, represents the control matrix, represents the control input at the moment; the prediction error covariance matrix at the current moment represents the estimate of the uncertainty of the predicted state; Using the measurement value at the current moment , the measurement matrix and the measurement error covariance matrix Calculate the Kalman gain coefficient , and update the posterior estimate value at time , expressed as:

[0069]

[0070] wherein, represents the transpose of the measurement matrix, represents the estimation of the state estimation uncertainty, represents the inverse matrix of the measurement error covariance matrix and the prediction error covariance matrix; and, update the error covariance matrix, expressed as:

[0071] wherein, represents the estimation of the state estimation uncertainty; Repeat the above steps iteratively using the currently obtained measurement values to update the state estimation of the target, and configure it as the position estimation of the hybrid-wing unmanned aerial vehicle at the next moment.

[0072] For example, the system state vector can be defined and initialized first. Among them, the state vector of the system can be defined according to the predicted physical quantities of the target (i.e., the hybrid-wing unmanned aerial vehicle), and can include position coordinates , speed , acceleration and other physical parameters.

[0073] For initialization, it includes: 1) Initialize the state estimation: Assign an initial value to the initial state estimation ; 2) Initialize the error covariance: Assign an initial value to the initial error covariance ;

[0074] In the prediction stage, at each time , based on the posterior state estimation value at time predict the prior state estimation value at the current time , and the prediction state process can be expressed as:

[0075] wherein, represents the state transition matrix, Represents the control matrix, denotes the control input at time

[0076] Current prediction error covariance matrix at time represents the estimation of the uncertainty of the predicted state, and the formula can be expressed as:

[0077] where represents the prediction error covariance matrix at time represents the transpose of the state transition matrix, represents the covariance matrix of the system noise.

[0078] During the update, the measurement value at the current time , the measurement matrix and the measurement error covariance matrix are used to calculate the Kalman gain coefficient , and the posterior estimate value at time is updated, expressed as:

[0079]

[0080] where represents the transpose of the measurement matrix, represents the estimation of the uncertainty of the state estimate , represents the inverse matrix of the measurement error covariance matrix and the prediction error covariance matrix.

[0081] Meanwhile, for the iterative calculation of the next time, the covariance matrix of the error is updated, and the formula is expressed as:

[0082] where represents the estimation of the uncertainty of the state estimate .

[0083] During the iterative prediction update process, the above steps are continuously repeated using the currently obtained measurement value (the position information of the hybrid-wing UAV), and the state estimate of the target can be continuously updated, that is, the position estimate of the hybrid-wing UAV at the next time.

[0084] In step S15, an autonomous guidance control instruction for the hybrid-wing unmanned aerial vehicle (UAV) is configured according to the position estimation to achieve closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position; wherein, the autonomous guidance control instruction includes a position adjustment instruction for at least one of the x-axis direction, y-axis direction, and z-axis direction.

[0085] Exemplarily, according to the currently predicted relative position coordinates (i.e., position estimation), closed-loop control is performed on the hybrid-wing UAV to autonomously guide it to perform horizontal and longitudinal position adjustments and altitude descent (i.e., the x-axis direction, y-axis direction, and z-axis direction) until the hybrid-wing UAV lands at a predetermined position.

[0086] Exemplarily, the configuring of the autonomous guidance control instruction for the hybrid-wing UAV according to the position estimation to achieve closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position includes: Using the PID closed-loop control method to perform decoupling control on the adjustments of the x-axis direction, y-axis direction, and z-axis direction of the hybrid-wing UAV, expressed as:

[0087] Wherein, is the control output, is the deviation between the measured value and the control value, is the proportionality coefficient, is the integral time coefficient, is the derivative time coefficient.

[0088] Specifically, the PID closed-loop control algorithm is used to perform decoupling control on the adjustments of the lateral position, longitudinal position, and altitude of the hybrid-wing UAV. The closed-loop control has a feedback mechanism, and the output of the system can participate in the control, avoiding the potential risk of unstable open-loop control guidance and ensuring the reliability of the system.

[0089] Exemplarily, the method further includes: after identifying that the hybrid-wing UAV contacts the autonomous recovery platform, the autonomous recovery platform controls the mechanical clamping rod to directly contact the landing gear of the hybrid-wing UAV to finely adjust the position of the hybrid-wing UAV to a predetermined landing point and lock the landing gear of the hybrid-wing UAV using the mechanical clamping rod.

[0090] Specifically, after the hybrid-wing UAV contacts the recovery platform, the platform will further finely adjust its position using the mechanical clamping rod and lock it. Specifically, after the hybrid-wing UAV contacts the platform, the recovery platform will control the mechanical clamping rod to directly contact the landing gear of the hybrid-wing UAV, finely adjust its position to a predetermined landing point, and lock the landing gear of the hybrid-wing UAV using the mechanical clamping rod.

[0091] Exemplarily, during the process of guiding the hybrid-wing unmanned aerial vehicle (UAV) to land on the autonomous recovery platform, it is also possible to determine the degree of distortion of the cooperative identifier in the collected optical image, that is, to determine the degree of distortion of the preferred cooperative identifier in the current image containing the preferred cooperative identifier; if the degree of distortion is greater than the preset threshold, an update of the preferred cooperative identifier can be triggered. Alternatively, if the preferred cooperative identifier cannot be recognized in the real-time collected UAV images during the closed-loop control process, an update of the preferred cooperative identifier can be triggered, and the updated preferred cooperative identifier can be used for image acquisition and position estimation.

[0092] Exemplarily, referring to Figure 2 As shown, the hybrid-wing UAV returns to the recovery airspace of the recovery platform through Beidou / GPS positioning information. The recovery platform uses a binocular camera to continuously obtain the cooperative identifiers installed at the bottom and side of the hybrid-wing UAV in real time, selects the most suitable cooperative identifier for detection, and continuously collects its optical image and depth image. Calculate the positioning coordinates for the optical image and depth image of the preferred cooperative identifier; obtain the relative position coordinates of the hybrid-wing UAV through information fusion; continuously predict its relative position at the next moment according to the movement trajectory of the hybrid-wing UAV; the recovery platform performs closed-loop control on the hybrid-wing UAV based on the predicted relative position coordinates, autonomously guides it to adjust its horizontal and vertical positions, and descend in altitude until the hybrid-wing UAV lands at a predetermined position. After the hybrid-wing UAV contacts the recovery platform, the platform will use a mechanical clamping rod to further finely adjust its position and lock it. Among them, the cooperative identifiers collected in the image can be evaluated, and the optimal identifier can be dynamically selected for continuous collection. When calculating the position coordinates of the hybrid-wing UAV using the preferred cooperative identifier, the depth coordinates can be solved by the parallax geometry method according to the position of the center of the cooperative identifier corresponding to the depth image; the positioning coordinates in the camera coordinate system can be solved using the pixel relationship; the positioning coordinates in the camera coordinate system are converted into relative positioning coordinates using coordinate transformation; the motion compensation of the positioning information is combined with the angle data of the attitude sensor of the recovery platform to reduce the interference caused by the disturbance of the sea surface water body to the positioning.

[0093] The method for recovering a blended-wing unmanned aerial vehicle (UAV) on the sea surface provided by the present invention uses far-end and near-end binocular cameras with different elevation angles to dynamically select the optimal identification mark from the cooperation identification marks installed at different positions on the fuselage of the blended-wing UAV, and continuously collect its optical images and depth images; uses the collected image data to calculate the relative positioning information of the blended-wing UAV, compensates the positioning information for motion based on the angle data of the attitude sensor of the recovery platform, and continuously predicts its relative position at the next moment according to the motion trajectory of the blended-wing UAV; based on the predicted relative position information, autonomously guides and controls the blended-wing UAV to perform horizontal and vertical position adjustments and altitude descent until the predetermined landing position. This solution, based on vision algorithms and control strategies, realizes the autonomous guidance and recovery of the blended-wing UAV under sea surface conditions, reduces human intervention, and improves the autonomy of the recovery process. Using image data for positioning the blended-wing UAV is fast and accurate, and can quickly and reliably respond to the movement of the blended-wing UAV. Compared with the existing recovery method that installs cooperation identification marks on the recovery platform and uses the UAV to actively identify and position, the recovery method provided by the present invention installs cameras on the recovery platform, further reducing the load requirement on the UAV, enabling it to carry more devices to improve its working ability. At the same time, the recovery method provided by the present invention avoids the risk of the cooperation identification marks being covered by sea water splashes, etc. In addition, through trajectory prediction and decoupled control strategies, the error between the actual landing point and the predetermined landing point of the blended-wing UAV can be reduced, the complexity of the system can be lowered, the response performance of the system can be optimized, and the success rate of recovering the blended-wing UAV on the sea surface can be improved. The autonomous recovery process ensures its recovery efficiency.

[0094] Exemplarily, the method of the present invention utilizes the cooperation identification marks carried by the blended-wing UAV, uses the cameras installed on the recovery platform, and based on vision algorithms, real-time locates the blended-wing UAV, and can autonomously complete the recovery of the blended-wing UAV on the sea surface, reducing the load requirement of the blended-wing UAV and improving the success rate and efficiency of the recovery while reducing the need for manual intervention. During the recovery process, the optical information and depth information of the cooperation identification marks carried by the blended-wing UAV are obtained in real time, and the relative coordinates of the blended-wing UAV are calculated through vision algorithms and information fusion, ensuring the accuracy of the recovery; by predicting the motion trajectory of the blended-wing UAV in real time and performing closed-loop control accordingly, it can respond more accurately to the dynamic changes of the blended-wing UAV, avoiding potential risks such as inaccurate landing points and too fast landing speeds during the recovery process, and improving the reliability of the recovery.

[0095] In this exemplary embodiment, a hybrid-wing unmanned aerial vehicle (UAV) autonomous recovery system based on a vision algorithm is provided. Depending on a remotely operated vehicle (ROV), it autonomously recovers, replenishes energy, and carries a hybrid-wing UAV on the sea surface to solve the technical problem that the existing sea surface recovery method for hybrid-wing UAVs requires manual intervention, resulting in low efficiency and safety. Refer to Figure 3 As shown, the system includes: an ROV subsystem 303 of the remotely operated vehicle, and an autonomous recovery system 302 provided on the ROV subsystem 303; the autonomous recovery system 302 is used to receive the hybrid-wing UAV 301.

[0096] Refer to Figure 4 As shown, the autonomous recovery system includes: a landing platform 2 for carrying the hybrid-wing UAV; an optical detection module 1 provided at the edge of the landing platform 2 for collecting optical and depth images at different angles and heights; a correction and locking module 3 provided on the landing platform 2 for locking the hybrid-wing UAV; an underwater wireless charging module 4 provided at the bottom of the landing platform 2 for performing fast non-contact charging on the hybrid-wing UAV at close range; and an integrated control module 5 provided at the bottom of the landing platform 2 for performing real-time processing of data and controlling the working states of each module.

[0097] Exemplarily, refer to Figure 4 As shown, the landing platform 2 includes: a platform bracket 23; a V-shaped passive guiding structure symmetrically provided on the platform bracket 23; wherein, the center distance between the V-shaped grooves of the passive guiding structure matches the distance between the skid-type landing gears of the hybrid-wing UAV; a platform support 24 provided within the frame structure of the platform bracket 23 for providing support for the V-shaped passive guiding structure.

[0098] Specifically, the optical detection module 1 is composed of a distal binocular camera 11 and a proximal binocular camera 12 for collecting optical and depth images at different angles and heights. The distal binocular camera 11 and the proximal binocular camera 12 can be symmetrically provided on both sides of the landing platform 2.

[0099] The landing platform 2 is composed of a first V-shaped passive guiding structure 21, a second V-shaped passive guiding structure 22, a platform bracket 23, and a platform support 24. Among them, the center distance between the V-shaped grooves of the passive guiding structures 21 and 22 matches the distance between the skid-type landing gears of the hybrid-wing UAV. Therefore, during the landing process of the hybrid-wing UAV, when the landing gear touches this structure, under the action of gravity, this structure can guide the hybrid-wing UAV to slowly slide towards the plane where the recovery position is located to complete lateral recovery. The platform bracket 23 is the installation frame of the landing platform 2. The platform support 24 provides support for the first V-shaped passive guiding structure 21 and the second V-shaped passive guiding structure 22.

[0100] Exemplarily, referring to Figure 4 and Figure 5 as shown, the correction and locking module 3 is composed of a first left - right hand screw - slider module 31 and a second left - right hand screw - slider module 32 symmetrically distributed on both sides of the landing platform 2, parallel mechanical clamping rods 33, two limit switches (the first limit switch 341 and the second limit switch 342), and a first electromagnet 351 and a second electromagnet 352, which are used to achieve the precise recovery and locking of the hybrid - wing unmanned aerial vehicle, and establish conditions for subsequent carrying and energy replenishment. As Figure 5 shown is the structural schematic diagram of the second left - right hand screw - slider module 32, including a driving motor 321, a coupling 322, a left - right hand screw 323, a screw base 324, a first slider 3251 and a second slider 3252, and a first mechanical clamping rod connecting piece 3261 and a second mechanical clamping rod connecting piece 3262. When the driving motor 321 rotates forward, it drives the first slider 3251 and the second slider 3252 to move inward, thereby driving the mechanical clamping rods 33 to close inward through the first mechanical clamping rod connecting piece 3261 and the second mechanical clamping rod connecting piece 3262. The first left - right hand screw - slider module 31 works in the same way, so as to correct the recovery position of the hybrid - wing unmanned aerial vehicle and achieve the longitudinal recovery of the hybrid - wing unmanned aerial vehicle. The first limit switch 341 is used for protection when the slider moves outward, and the second limit switch 342 is used for protection when the slider moves inward. The first electromagnet 351 and the second electromagnet 352 are used for magnetic attraction locking of the hybrid - wing unmanned aerial vehicle, so as to meet the conditions for underwater carrying with the ROV.

[0101] Exemplarily, the underwater wireless charging module 4 is mainly composed of a wireless charging transmitter. When the hybrid - wing unmanned aerial vehicle is recovered to the designated position, the wireless charging receiver under its belly is aligned with the wireless charging transmitter, and energy is transmitted through electromagnetic waves to achieve fast non - contact charging of the hybrid - wing unmanned aerial vehicle at close range.

[0102] Exemplarily, the integrated control module 5 may include: a vision computing unit 51, a sensor unit 52, a wireless communication unit 53, an execution controller unit 54, an STM32 data processing unit 55, and a power supply unit 56. The vision computing unit 51 consists of a Jetson board 511 and an installation housing 512. The Jetson board 511 is used to process the images collected by the optical detection module 1 and calculate and obtain the optical positioning information of the hybrid-wing unmanned aerial vehicle. The sensor unit 52 is used to obtain the IMU information of the hybrid-wing unmanned aerial vehicle. The wireless communication unit 53 is used for data communication during the autonomous recovery system's guided landing of the hybrid-wing unmanned aerial vehicle. The execution controller unit 54 consists of a four-way relay 541, encoders 542 (including a first encoder 5421 and a second encoder 5422), and motor controllers 543 (including a first motor controller 5431 and a second motor controller 5432). Two of the relays 541 are used to control the first electromagnet 351 and the second electromagnet 352, and two are used to control the operation of the first encoder 5421 and the second encoder 5422. The first encoder 5421 is used to control the first limit switch 341 and control the corresponding drive motor through the first motor controller 5431. The second encoder 5422 is used to control the second limit switch 342 and control the corresponding drive motor 321 through the second motor controller 5432. The STM32 data processing unit 55 is used to perform operations such as motion compensation and data fusion on the IMU information obtained by the sensor unit 52 and the optical positioning information calculated by the vision computing unit 51, improve the accuracy and reliability of the positioning information, and use the positioning information to guide the hybrid-wing unmanned aerial vehicle in real time through the wireless communication unit. At the same time, estimate the landing point of the hybrid-wing unmanned aerial vehicle and adjust the mechanical clamping rod 33 through the execution controller unit 54. The power supply unit 56 consists of a 24V power supply 561 and a 24V to 19V power module 562.

[0103] Exemplarily, during operation, the autonomous sea surface recovery system for the hybrid-wing unmanned aerial vehicle (UAV) is carried on the remotely operated vehicle (ROV). When the hybrid-wing UAV needs energy replenishment after completing the aerial detection mission, the autonomous recovery system receives the landing request through the wireless communication unit 53. Then, the STM32 data processing unit 55 sends an instruction to the ROV, and the ROV lifts the autonomous recovery system to the sea surface. The hybrid-wing UAV uses the Beidou / GPS positioning signal to return to the airspace near the autonomous recovery system. The autonomous recovery system identifies the cooperative identification image of the hybrid-wing UAV through the distal binocular camera 11, and the vision computing unit 51 processes the image and obtains the positioning information of the hybrid-wing UAV. The STM32 data processing unit 55 sends an instruction to the hybrid-wing UAV through the wireless communication unit 53 based on this positioning information, guiding the hybrid-wing UAV to land on the autonomous recovery system; when the hybrid-wing UAV descends to a lower altitude, it is identified by the proximal binocular camera 12. When the hybrid-wing UAV lands on the landing platform, under the guidance of the first V-shaped passive guiding structure 21 and the second V-shaped passive guiding structure 22, it slides horizontally to the recovery position to complete the horizontal recovery. At the same time, the STM32 data processing unit 55 sends an instruction to the execution controller unit 54, and controls the corresponding drive motors to rotate through the first motor controller 5431 and the second motor controller 5432, driving the first positive and negative lead screw slider module 31 and the second positive and negative lead screw slider module 32 to move, realizing the inward closing movement of the mechanical clamping rod 33 to complete the longitudinal recovery of the hybrid-wing UAV. Subsequently, the STM32 data processing unit 55 sends an instruction, and the first electromagnet 351 and the second electromagnet 352 magnetically lock the hybrid-wing UAV to complete the precise recovery of the hybrid-wing UAV, ensuring the alignment of the wireless charging receiving end carried by the hybrid-wing UAV with the wireless charging transmitting end of the autonomous recovery system, and realizing contactless energy replenishment of the hybrid-wing UAV through electromagnetic wave energy transfer. The ROV can carry the hybrid-wing UAV underwater for carrying according to the mission requirements. After the energy replenishment is completed, the ROV can float up again to release the hybrid-wing UAV to continue to perform the detection mission.

[0104] The hybrid-wing UAV sea surface recovery system provided by the present invention, based on the innovative mechanical structure design, effectively solves the key problems such as large sea water disturbance, the need for manual intervention assistance, low recovery success rate and low recovery efficiency when the existing hybrid-wing UAV is recovered on the sea surface by introducing vision algorithms and autonomous guidance control. The mechanical structure is designed for the recovery scenario, simplifying the design of the hybrid-wing UAV sea surface recovery mechanism, and thus reducing the load requirement on the underwater ROV. Using a wireless method to replenish energy for the hybrid-wing UAV simplifies the operation process, does not affect the underwater carrying ability of the ROV, improves the autonomy of the overall system, and meets the mission requirements of the cross-domain unmanned system cooperation paradigm.

[0105] It should be noted that the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0106] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more of the above-mentioned modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0107] Specifically, according to the embodiments of the present invention, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present invention include a computer program product, which includes a computer program carried on a storage medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), various functions defined in the system of the present application are executed.

[0108] It should be noted that the storage medium shown in the embodiments of the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any storage medium other than a computer-readable storage medium, and this storage medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0110] The units involved in the embodiments of the present invention can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0111] It should be noted that, on the other hand, the present application also provides a storage medium, which can be included in an electronic device; or can exist alone without being assembled into the electronic device. The above storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device is caused to implement the methods described in the following embodiments. For example, the electronic device can implement each step of the method as Figure 1 shown.

[0112] In one embodiment, the present application provides a computer program product, including a computer program, which when executed by a processor implements the steps in the above method embodiments.

[0113] In addition, the above drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes can be executed synchronously or asynchronously in, for example, multiple modules.

[0114] Those skilled in the art will readily think of other embodiments of the present invention after considering the specification and practicing the invention herein. The present application aims to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed in the present invention. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the claims.

[0115] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for autonomous sea surface recovery of hybrid wing UAV based on visual algorithm, characterized in that: The method comprises: When a hybrid wing UAV to be recovered is identified in a preset recovery airspace of the autonomous recovery platform, image acquisition is performed on the hybrid wing UAV to obtain at least one frame of UAV image containing a cooperation mark; wherein the UAV image includes an optical image and a corresponding depth image; and cooperation marks are respectively provided at different positions on the bottom and side of the hybrid wing UAV; Identify at least one frame of drone image containing the cooperation logo, and select the preferred cooperation logo based on the degree of distortion of the cooperation logo in the drone image; Collecting a current drone image containing the preferred cooperation identifier for the hybrid-wing drone, and determining current position information of the hybrid-wing drone based on the current drone image; Predict the motion trajectory of the hybrid wing UAV based on the current position information, and obtain the position estimate of the hybrid wing UAV at the next moment; According to the position estimation, the autonomous guidance control instructions of the hybrid-wing UAV are configured to achieve closed-loop control of the hybrid-wing UAV until the hybrid-wing UAV lands at a predetermined position; wherein the autonomous guidance control instructions include: position adjustment instructions in at least one of the x-axis direction, the y-axis direction, and the z-axis direction.

2. The method according to claim 1, characterized in that: The step of identifying at least one frame of drone image containing the cooperation logo, and selecting the preferred cooperation logo based on the degree of distortion of the cooperation logo in the drone image, includes: When a plurality of cooperation logos are included in one optical image, the degree of distortion of the cooperation logos is evaluated based on the area and perimeter of the cooperation logos in the optical image; Based on the distortion degree assessment results of the cooperation logo, select the preferred cooperation logo.

3. The method according to claim 1 or 2, characterized in that: The step of identifying at least one frame of drone image containing the cooperation logo, and selecting the preferred cooperation logo based on the degree of distortion of the cooperation logo in the drone image, includes: When the same cooperation logo is included in multiple frames of UAV images, the degree of distortion of the cooperation logo is evaluated based on the resolution of the optical image and the position information of the center of the cooperation logo in the image; According to the distortion degree evaluation result of the cooperation logo, the preferred cooperation logo is determined, and the corresponding optical image is configured for calculating the current position information of the hybrid wing UAV.

4. The method according to claim 1, characterized in that The determining the current position information of the hybrid-wing UAV according to the current UAV image includes: Determine the conversion relationship between the pixel coordinate system of the depth image and the camera coordinate system according to the camera intrinsic parameters; Based on the conversion relationship, the pixel coordinates of each end corner of the preferred cooperation mark in the pixel coordinate system are converted into corresponding three-dimensional point cloud coordinates in the camera coordinate system; When the preferred cooperation mark is the bottom cooperation mark, the positioning coordinates of the preferred cooperation mark in the camera coordinate system are configured according to the average of the three-dimensional point cloud coordinates of each end corner; or, When the preferred cooperation mark is a side cooperation mark, the mean of the three-dimensional point cloud coordinates of each end corner of the preferred cooperation mark is transformed according to the preset conversion matrix between the side cooperation mark and the bottom cooperation mark to determine the positioning coordinates of the preferred cooperation mark; wherein the positioning coordinates are the center coordinates of the preferred cooperation mark; Based on the rotation matrix between the camera coordinate system and the platform coordinate system, the positioning coordinates in the camera coordinate system are converted into relative position coordinates in the platform coordinate system and configured as the current position information of the hybrid wing UAV.

5. The method according to claim 4, characterized in that The rotation matrix between the camera coordinate system and the platform coordinate system is determined based on the product of the motion rotation matrix between the two frames before and after the image and the rotation matrix of the platform coordinate system.

6. The method according to claim 1, characterized in that The method of predicting the motion trajectory of the hybrid-wing UAV based on the current position information to obtain the position estimate of the hybrid-wing UAV at the next moment includes: Based on the physical quantity predicted for the hybrid wing UAV, a system state vector is defined and initialized; wherein the physical quantity includes: at least one of position coordinates, velocity, and acceleration; initialization includes: initializing state estimation and initializing error covariance matrix; At every time ,based on The posterior state estimate at time Predict the prior state estimate at the current moment , expressed as: in, represents the state transfer matrix, represents the control matrix, express Control input at the moment; current The forecast error covariance matrix at time expresses an estimate of the uncertainty of the predicted state; Use Current Measurement value at time , measurement matrix and the measurement error covariance matrix Calculate the Kalman gain coefficient , and update The posterior estimate of time , expressed as: in, represents the transpose of the measurement matrix, Represents the state estimate Uncertainty estimates, Represents the inverse matrix of the measurement error covariance matrix and the prediction error covariance matrix; And, the error covariance matrix is ​​updated, expressed as: in, Represents the state estimate Uncertainty estimates; The above method is repeated iteratively using the currently acquired measurement values ​​to update the state estimate of the target and configure it as the position estimate of the hybrid wing UAV at the next moment.

7. The method according to claim 4, characterized in that The autonomous guidance control instructions of the hybrid wing UAV are configured according to the position estimation to realize closed-loop control of the hybrid wing UAV until the hybrid wing UAV lands at a predetermined position, including: The PID closed-loop control method is used to decouple the adjustment of the x-axis, y-axis, and z-axis directions of the hybrid wing UAV, which can be expressed as: in, To control the output, is the deviation between the measured value and the control value, is the proportionality coefficient, is the integration time coefficient, is the differential time coefficient.

8. The method according to claim 1, characterized in that The method further comprises: After identifying that the hybrid-wing UAV is in contact with the autonomous recovery platform, the autonomous recovery platform controls the mechanical clamp rod to directly contact the landing gear of the hybrid-wing UAV to fine-tune the position of the hybrid-wing UAV to the predetermined landing point, and uses the mechanical clamp rod to lock the landing gear of the hybrid-wing UAV.

9. A hybrid wing UAV sea surface autonomous recovery system based on a visual algorithm, applied to the method according to any one of claims 1 to 8, characterized in that: The system comprises: a remote-controlled underwater robot ROV subsystem, and an autonomous recovery system arranged on the ROV subsystem; The autonomous recycling system comprises: A landing platform (2) for carrying a hybrid wing UAV; An optical detection module (1) is arranged at the edge of the landing platform (2) and is used to collect optical and depth images at different angles and heights; A correction locking module (3) is arranged on the landing platform (2) and is used to lock the hybrid wing UAV; An underwater wireless charging module (4) is arranged at the bottom of the landing platform (2) and is used for performing close-range rapid contactless charging of the hybrid wing UAV; The integrated control module (5) is arranged at the bottom of the landing platform (2) and is used for processing data in real time and controlling the working status of each module.

10. The system according to claim 9, characterized in that The landing platform (2) comprises: Platform support (23); A V-shaped passive guidance structure is symmetrically arranged on the platform bracket (23); wherein the center spacing of the V-shaped grooves of the passive guidance structure matches the spacing of the skid-type landing gear of the hybrid wing UAV; The platform support (24) is arranged in the frame structure of the platform bracket (23) and is used to provide support for the V-shaped passive guide structure.

Images