Multi-unmanned ship-aircraft cooperative hunting method and system for unknown target

Abstract

The invention discloses an unknown target-oriented multi-unmanned ship-aircraft cooperative hunting method and system, and relates to the fields of visual perception, hunting motion planning and control and the like. An environment image and depth data are acquired through a depth camera carried by an unmanned aerial vehicle, target identification and tracking are realized by using a water surface target detection method, coordinate transformation is carried out through internal reference of the depth camera and pose information of the unmanned aerial vehicle, and the position of a target in a world coordinate system is solved; and the surrounding members communicate with each other, and speed planning is performed by using a multi-unmanned-ship cooperative surrounding method based on a double-layer surrounding strategy, so that surrounding and capturing of an escape target are realized. The multi-unmanned ship-aircraft hunting system has autonomous sensing and planning capabilities, the hunting method is suitable for a scene where the speed of the escape target changes, the problem that hunting fails due to the fact that the escape target has certain intelligent sensing and escape capabilities in the hunting process is solved, and the hunting success rate and efficiency are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of swarm motion planning and control, in particular to a multi-unmanned vehicle-vehicle cooperative hunting method, system, equipment and medium for unknown targets. BACKGROUND

[0002] The sea-air cross-domain cooperation of unmanned systems has the advantages of functional complementation and energy efficiency multiplication, which is conducive to achieving cooperative reconnaissance and attack in complex task environments, and is an important scene in the cross-domain cooperation of unmanned systems. With the rapid development of unmanned system theory and technology, heterogeneous multi-agent systems such as unmanned aerial vehicle-unmanned surface vehicle systems have gradually become a research hotspot, and environmental perception, decision planning and cooperative control of heterogeneous multi-agent systems have attracted widespread attention.

[0003] In the unmanned aerial vehicle-unmanned surface vehicle system, the unmanned surface vehicle and the unmanned aerial vehicle can complement each other, taking advantage of the long endurance and large load of the unmanned surface vehicle, and the large perception range of the unmanned aerial vehicle. When the unmanned aerial vehicle-unmanned surface vehicle system performs cooperative reconnaissance, sea rescue and swarm attack tasks, the identification and hunting of the target is the key to completing the task. Hunting is a strategy of forming a circle around the target and gradually shrinking to control or capture the target. The difficulty of the unmanned aerial vehicle in identifying and tracking the target boat lies in the high real-time requirement of the target identification algorithm, and the identification accuracy is easily affected by light interference and shadow blocking. The difficulty of the unmanned surface vehicle in hunting lies in the accuracy of target positioning, which depends on the effect of target identification and tracking, and the hunting efficiency and success rate are low.

[0004] Based on the above, the present application provides a multi-unmanned vehicle-vehicle cooperative hunting technical scheme for unknown targets, which realizes stable identification and tracking of the unmanned aerial vehicle to the water surface target, and transmits the target positioning information to the unmanned surface vehicle for efficient hunting. SUMMARY

[0005] The present application provides a multi-unmanned vehicle-vehicle cooperative hunting method, system, equipment and medium for unknown targets, which detects the target after detecting the target through visual perception of the environment, and plans the hunting motion to improve the hunting efficiency and success rate.

[0006] To achieve the above purpose, the embodiments of the present application adopt the following technical solutions: In a first aspect, the method provided by the embodiments of the present application is used in an unmanned system composed of one unmanned aerial vehicle and at least three unmanned surface vehicles, a visual perception method is run on the unmanned aerial vehicle, and a cooperative hunting method is run on each unmanned surface vehicle. The visual perception method and the cooperative hunting method are executed cyclically, and each execution cycle includes: S1. The UAV acquires image information of the water surface environment through a depth camera, uses an improved water surface target detection algorithm to obtain the category and bounding box parameters of objects in the image, and uses the intrinsic parameter matrix of the depth camera and the measured depth data combined with the UAV pose information to perform coordinate transformation to obtain the position of the target in the world coordinate system. S2. Perform Kalman filter prior estimation based on the established target state dynamic equation, combine the target state correction estimation with the depth camera measurement, and then predict the target state at future time through the dynamic equation. S3. Distribute potential points evenly around the target based on the number of unmanned surface vessels (USVs). Assign the nearest potential point to each USV in the world coordinate system according to the USV's serial number. Plan the velocity of each USV based on the improved dynamic window method. S4. Calculate the unmanned surface vessel's (USV) occupancy angle based on the speed ratio between the USV and the target, and then calculate the coverage angle between the USVs to obtain the speed based on target encirclement behavior planning. Calculate the speed based on target-oriented behavior planning using the position deviation vector between the USV and the target, and synthesize it with the speed of target encirclement behavior planning to obtain the speed of the USV based on encirclement behavior planning.

[0007] Secondly, the multi-unmanned surface vessel-aircraft cooperative capture system for unknown targets provided by the embodiments of the present invention consists of two types of unmanned equipment: an F450 quadcopter drone and a dual-motor differential speed unmanned surface vessel. The F450 quadcopter drone in the system is a quadcopter drone with a wheelbase of 450mm and an all-carbon steel structure. A Holybro Pixhawk4 flight controller is selected as the underlying controller for the quadcopter drone. A GPS module provides external position estimation for the drone. A depth camera is mounted on the bottom of the drone for environmental perception. A Jetson Orin NX onboard computer is mounted on the drone for target detection, cluster communication, and control signal transmission. The dual-motor differential speed unmanned surface vessel in the system uses an 830mm×523mm×300mm fishing boat as its main hardware structure. A CUAV V5+ controller is selected as the underlying controller for the dual-motor differential speed unmanned surface vessel. An external position estimation is provided via a GPS module. A Jetson Xavier NX onboard computer is mounted on the unmanned surface vessel for cluster communication and control signal transmission.

[0008] The onboard computer is loaded with an autonomous perception and planning program for a multi-unmanned surface vessel-machine cooperative encirclement system for unknown targets. The autonomous perception and planning program includes: a perception module, a data transmission module, a planning module, and a control module.

[0009] The perception module is used to acquire the category and bounding box parameters of objects in the water surface environment image, and calculate the position of the target in the world coordinate system; The data transmission module is configured to broadcast the predicted position of the target and the positions of the unmanned aerial vehicle and the unmanned surface vehicle in the multi-unmanned surface vehicle-unmanned aerial vehicle hunting system. The planning module is configured to perform motion planning of the unmanned aerial vehicle and each unmanned surface vehicle, and adjust the hunting strategy based on the motion state of the target. The control module is configured to control the flight of the unmanned aerial vehicle and the sailing of the unmanned surface vehicle according to the positions or speeds obtained by the motion planning.

[0010] In a third aspect, an embodiment of the present application discloses an electronic device, comprising: a memory configured to store a computer program; a processor configured to execute the computer program to implement the steps of the multi-unmanned surface vehicle-unmanned aerial vehicle cooperative hunting method for unknown targets disclosed above.

[0011] In a fourth aspect, an embodiment of the present application discloses a computer readable storage medium configured to store a computer program; wherein the computer program is executed by a processor to implement the steps of the multi-unmanned surface vehicle-unmanned aerial vehicle cooperative hunting method for unknown targets disclosed above.

[0012] The embodiment of the present application provides a multi-unmanned surface vehicle-unmanned aerial vehicle cooperative hunting method, system, device and medium for unknown targets, collects environmental images and depth data by a depth camera carried by an unmanned aerial vehicle, realizes target recognition and tracking by using a water surface target detection method, and converts coordinates by using a depth camera internal parameter and unmanned aerial vehicle pose information to calculate the position of the target in a world coordinate system; the hunting members communicate with each other, plan speeds by using a multi-unmanned surface vehicle cooperative hunting method based on a double-layer hunting strategy, and realize surrounding and capturing of the escaping target. The multi-unmanned surface vehicle-unmanned aerial vehicle hunting system in the embodiment has the ability of autonomous perception and planning, the hunting method is suitable for the scene where the speed of the escaping target changes, solves the problem that the hunting fails due to the intelligent perception and escaping ability of the escaping target in the hunting process, and improves the hunting success rate and efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0014] Figure 1 A flowchart of the multi-unmanned surface vehicle-unmanned aerial vehicle cooperative hunting method for unknown targets of the present application; Figure 2 A schematic diagram of the target being shadowed of the present application; Figure 3This is a schematic diagram of histogram equalization according to the present invention; Figure 4 This is a schematic diagram of the HTC image enhancement method of the present invention; Figure 5 This is a schematic diagram of the Apollonius circle of the present invention; Figure 6 This is a schematic diagram of the coverage angle of the present invention; Figure 7 This is a schematic diagram of the behavior vector fusion of the present invention; Figure 8 This is a schematic diagram of the UAV hardware platform of the present invention; Figure 9 This is a schematic diagram of the hardware platform for the unmanned surface vessel of the present invention.

[0015] Figure 10 This is a schematic diagram of the electronic device for the coordinated capture of multiple unmanned surface vessels and aircraft targeting unknown targets according to the present invention. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Embodiments of the present invention will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of the present invention means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0017] The design concept of this embodiment mainly focuses on designing a multi-UAV-UAV cooperative encirclement method and system for unknown targets. It utilizes a GPS module to provide external position estimation, implementing a surface environment perception algorithm and an encirclement motion planning algorithm to achieve efficient encirclement. The algorithm scheme includes a perception module, a data transmission module, a planning module, a control module, and data interaction interfaces between these modules. Next, the hardware parameters of the UAVs and UAVs are determined, including motor parameters, propeller pitch and size, ESC parameters, battery parameters, sensors, and onboard computers, etc., to design and build high-performance UAVs and UAVs that meet the requirements. Then, through the development of software modules and the construction of the overall operational framework, the stable operation of the UAV swarm system is ensured. Finally, the performance of the algorithm is evaluated by recording the experimental results of the system.

[0018] This invention provides a multi-unmanned surface vessel (USV)-drone cooperative encirclement method for unknown targets. The method is used in an unmanned system consisting of one UAV and at least three USVs. A visual perception method runs on the UAV, and a cooperative encirclement method runs on each USV. The visual perception method and the cooperative encirclement method are executed cyclically. The UAV adopts an all-carbon steel structure design and is equipped with a flight controller, onboard computer, depth camera, and other equipment, suitable for quadcopter UAVs with different wheelbases. The USVs are equipped with motion controllers, onboard computers, and other equipment. For example, an F450 frame (450mm wheelbase) is selected as the hardware platform for the quadcopter UAV, and a Holybro Pixhawk4 flight controller is selected as the underlying controller. A GPS module provides external position estimation for the UAV. A depth camera is mounted bottom-down on the UAV for environmental perception, and a Jetson Orin NX onboard computer is mounted on the UAV for target detection, cluster communication, and control signal transmission. A fishing boat measuring 830mm × 523mm × 300mm was selected as the hardware platform for the dual-motor differential speed unmanned surface vessel (USV). The CUAV V5+ controller was chosen as the underlying controller for the USV. External position estimation is provided via a GPS module, and a Jetson Xavier NX onboard computer is used for cluster communication and control signal transmission. The software modules for the multi-USV-UAV cooperative encirclement method for unknown targets include: a perception module, a data transmission module, a planning module, and a control module. The perception module mainly realizes target detection and trajectory prediction in the water surface environment. The data transmission module mainly broadcasts the status of the UAV and USV themselves, as well as the predicted position of the target, in the communication network. The planning module mainly implements the encirclement planning method of the system and adjusts the encirclement strategy based on the target's motion state. The control module mainly converts the received instructions from the planning module into control quantities for the motors of the UAV or USV, controlling the UAV or USV to track a predetermined path or speed.

[0019] like Figure 1As shown, each execution cycle includes: S1. The UAV acquires image information of the water surface environment through a depth camera, uses a water surface target detection algorithm to obtain the category and bounding box parameters of objects in the image, and uses the intrinsic parameter matrix of the depth camera and the measured depth data combined with the UAV pose information to perform coordinate transformation to obtain the target position. The onboard computer of the drone sends the location to the drone's underlying controller to complete the drone's trajectory tracking; the target location is the target's position in the world coordinate system, and the target is an unmanned surface vessel.

[0020] The improved water surface target detection algorithm includes acquiring images of targets under different environments using aerial drones and mobile phone cameras, labeling the images with YOLO format tags using LabelImg software, marking targets in the images with specific bounding boxes, constructing a water surface target dataset, and applying it to the training of the YOLOv5 model. After weighted brightness stretching of the images acquired by the depth camera using a hyperbolic tangent curve, target detection is performed based on the trained YOLOv5 model.

[0021] S2. Perform Kalman filtering prior estimation based on the target state dynamic equation, combine the target state to correct the prior estimation to obtain the posterior estimation, and use the target state dynamic equation to update the posterior estimation to obtain the predicted target state.

[0022] The target state includes the target instantaneous velocity and the target position; the target instantaneous velocity is obtained by calculating the deviation of the target position obtained in S1 at a time interval of 0.1s, and the target position and velocity are combined into the target state.

[0023] S3. Based on the number of unmanned surface vessels (USVs), evenly set up potential points around the target, and sequentially assign the nearest potential points as the target points of the USVs in the world coordinate system according to the USV numbering order; plan the speed of the USVs.

[0024] The planned speed of the unmanned surface vessel is based on the improved dynamic window method for planning the speed of each unmanned surface vessel. The process involves unifying the world coordinate system for each unmanned surface vessel (USV) and unmanned aerial vehicle (UAV). The target's speed and heading are calculated based on changes in its position. Potential points are then evenly distributed around the target at fixed positions or angles. Following the USV's numerical order, the nearest potential point is sequentially assigned from the remaining potential points using a greedy algorithm as the USV's desired path point. An improved dynamic window method is used to plan the speed of each USV, and this position is sent to the USV's underlying controller, which then directs the USVs to their respective potential points.

[0025] S4. Calculate the unmanned surface vessel's (USV) occupancy angle based on the speed ratio between the USV and the target, and then calculate the coverage angle between the USVs to obtain the speed based on target encirclement behavior planning. Calculate the speed based on target tendency behavior planning using the position deviation vector between the USV and the target, and synthesize it with the speed of target encirclement behavior planning to obtain the speed of the USV based on encirclement behavior planning.

[0026] In this embodiment, the UAV shares environmental perception information with the unmanned surface vessel (USV), achieving collaborative target capture based on an encirclement method. Each UAV or USV member in the system has a distributed planner, capable of independently planning its own path. Communication between each UAV / USV and the ground-based host is established via Wi-Fi, enabling the transmission of pose data, target location data, and control commands. Specifically, the UAV acquires environmental images and depth data using its onboard depth camera. On the onboard computer, target recognition and tracking are achieved using a surface target detection method. Coordinate transformation is performed using the depth camera's intrinsic parameters and the UAV's pose information to calculate the target's position in the world coordinate system. This position is then sent to the UAV's underlying controller to complete trajectory tracking. The USV obtains the target's predicted trajectory through a ROS distributed network and uses a multi-USV collaborative encirclement method based on a two-layer encirclement strategy for velocity planning. This velocity is sent to the USV's underlying controller to achieve the encirclement and capture of the escaping target. The multi-unmanned surface vessel-machine encirclement system in this embodiment has autonomous perception and planning capabilities. The encirclement method is applicable to scenarios where the speed of the escaping target changes. It solves the problem of encirclement failure caused by the escaping target having a certain degree of intelligent perception and escape capabilities, and improves the success rate and efficiency of encirclement.

[0027] This embodiment of the multi-UAV-UAV cooperative encirclement method for unknown targets addresses the problems that the recognition accuracy of UAV surface target detection methods is easily affected by light interference and shadow occlusion, and that the encirclement efficiency and success rate of single-layer encirclement strategies are low for agile escape targets. Corresponding algorithm modules and hardware platforms have been invented. The multi-UAV-UAV cooperative encirclement system for unknown targets can also be used as an experimental platform to test the performance of other encirclement algorithms.

[0028] In this embodiment, S1 includes: improving the input side of the YOLOv5 model using an image enhancement strategy based on hyperbolic tangent curve (HTC) weighted brightness stretching. For RGB images, let their pixel values ​​be... , , and These represent the red, green, and blue components of the image's three color channels, respectively, and are normalized to... Let the coordinates of the pixel position be... The following HTC function is used to implement non-linear mapping of color image brightness: in, , It's a scaling factor. HTC is... The function is a monotonically increasing function within a certain range. When processing an image, smaller image intensity values ​​increase more than larger image intensity values. Therefore, HTC is used to enhance low-light performance of drone imagery.

[0029] To suppress highlight enhancement, the brightness of the output image is weighted as follows: in These are weighting coefficients, which are assigned as the average of the RGB channel values ​​to preserve the image intensity distribution.

[0030] The image should cover all pixel gray levels from 0 to 1 to ensure global image enhancement; therefore, image intensity is normalized. in, and These are the maximum and minimum values ​​of the image intensity, respectively.

[0031] The result of image enhancement is affected by the scaling factor The weighting parameters are determined based on the maximum image entropy to achieve good image enhancement results. Therefore, it is necessary to select appropriate parameters. Image entropy, as a parameter characterizing image information, can be used to measure image quality. The higher the entropy, the more information the image carries, and the clearer the image. Image entropy is defined as: in It is the first The ratio of image intensities, this formula is used to calculate the proportion of different image intensities. of The entropy. When the entropy is at its maximum, the corresponding... The value is the optimal value. Therefore, the optimal value is adaptively obtained through iterative search based on image entropy. value.

[0032] Figure 2 , Figure 3 , Figure 4 This illustrates the effects of different image enhancement methods on low-light images. Figure 2 It can be seen that unmanned surface vessels in shadow are easily confused with them. After... Figure 3 After histogram equalization, the image brightness is significantly improved, but this is accompanied by the generation of a large amount of artifact noise. Figure 4The results of using the HTC-based image enhancement proposed in this invention show that the contrast between the unmanned surface vessel and the background is improved, and the generated artifact noise is less than that of the histogram equalization method.

[0033] Images captured by a depth camera are processed by HTC image enhancement and then input into the YOLOv5 model for real-time detection. The model obtains the object category and bounding box parameters in the image, determines the consistency between the object category and the target category, and calculates the size of the center point of the bounding box based on the pixel position of the top left point of the bounding box and the length and width of the bounding box. This is the position of the target in the pixel coordinate system.

[0034] By using the intrinsic parameter matrix of the depth camera and the measured depth values ​​to perform coordinate transformation, the coordinates of the target in the camera coordinate system are calculated. Let the coordinates of the target in the pixel coordinate system be... These coordinates are not the physical coordinates for imaging; they need to be transformed to the image coordinate system. The transformation formula from pixel coordinates to image coordinates is as follows: in, , , This refers to the camera's focal length, measured in millimeters. for Normalized focal length on the axis, for Normalized focal length on the axis.

[0035] Let the target's coordinates in the camera coordinate system be... The transformation from the image coordinate system to the camera coordinate system can be further calculated using the principle of central projection, as shown in the following formula: in, The depth value measured by the depth camera.

[0036] Substituting the transformation relationship from pixel coordinates to image coordinates into the above formula, we can obtain the transformation relationship from the pixel coordinate system to the camera coordinate system: Based on the mounting angle and distance of the depth camera on the UAV, construct the rotation and translation transformation matrices from the camera coordinate system to the UAV coordinate system. Let the target's coordinates in the UAV coordinate system be... The transformation relationship from the camera coordinate system to the UAV coordinate system is as follows: Similarly, using the UAV's own GPS and IMU modules to obtain the UAV's position and attitude information, a rotation and translation transformation matrix is ​​constructed from the UAV coordinate system to the world coordinate system. Let the target's coordinates in the world coordinate system be... The transformation relationship from the UAV coordinate system to the world coordinate system is as follows: in, and These represent the rotation and translation transformation matrices from the UAV coordinate system to the world coordinate system, respectively.

[0037] In this embodiment, S2, the target trajectory prediction based on the Kalman filter equation includes: the Kalman filter discrete-time system dynamic equation and the system measurement equation can be expressed as: in, for State estimation at time 10:00 Represents the state transition matrix. It has a mean of zero and a positive definite covariance matrix of... And a white noise sequence that follows a Gaussian distribution. It is the process noise gain coefficient. For system observations, The observation coefficient matrix, The vector has a mean of zero and a positive definite covariance matrix. And a white noise sequence that follows a Gaussian distribution, and and They are independent of each other.

[0038] The Kalman filter system state variables are selected as follows: Based on the target's motion characteristics, the dynamic equations are constructed as follows: in, , and They are respectively Always the goal is Position, velocity, and acceleration along the axis , and They represent Always the goal is Position, velocity, and acceleration along the axis. T is the sampling time interval between two adjacent observations; when the sampling frequency is around 10Hz, the interval can be taken as 0.1s. The above equation can be written in matrix form as follows: The state transition matrix is ​​then: The system dynamic noise vector is: The measurement equation is constructed based on the target state information obtained by the sensor as follows: The observation coefficient matrix is ​​then... Then, the Kalman filter parameters are initialized, the target state and error covariance are estimated a priori, the Kalman filter gain is calculated based on the prior estimate of the error covariance, the prior estimate of the target state is corrected by the sensor measurement value, and the prior estimate of the error covariance is corrected by the Kalman filter gain. Finally, the target state at future time is predicted by the dynamic equation, the predicted position of the target is input into the UAV flight control for target tracking, and the predicted position of the target is published to the cluster communication network.

[0039] After the unmanned surface vessel (USV) obtains the predicted target position through the swarm communication network, a potential point-based encirclement strategy is used for collaborative target tracking. Specifically, in S3, this includes: unifying the world coordinate system of each USV and UAV; calculating the target velocity based on changes in the target position; and uniformly setting potential points around the target at fixed positions or angles. The Individual trend points With position goal The positional relationships between them are as follows: in This represents the target's yaw angle in the world coordinate system. The radius of the encirclement. for The range of values ​​for .

[0040] The nearest potential points are assigned sequentially according to the unmanned surface vessels' (USVs) number, serving as the USVs' target points in the world coordinate system. The speed of each USV is planned using an improved dynamic window method. Once all USVs have reached their respective potential points, step S4 is initiated. This improved dynamic window method addresses dynamic obstacle avoidance and maritime rule compliance, improving upon the traditional method. First, a dynamic obstacle model is introduced, updating the obstacle's position, speed, and heading angle in real time, enabling the USVs to predict and avoid moving obstacles. Second, a steering evaluation function based on the International Maritime Collision Avoidance Regulations (COLREGs) is designed. By calculating the relative heading angle between the USV and the obstacle, compliant steering maneuvers are enforced in specific scenarios such as head-on or cross-road encounters; otherwise, the trajectory score is reduced. Furthermore, kinematic constraints are optimized, combining dynamic windows and braking distance to ensure path safety. A weighted trade-off between multiple objective evaluation functions (orientation, distance, speed, and steering compliance) balances path efficiency and rule compliance. The multiple objective evaluation functions are as follows: Wherein, distance function This represents the estimated distance from the obstacle; a larger value indicates a greater distance from the obstacle. (Heading function) This represents the supplementary angle between the estimated trajectory's endpoint heading and the destination direction; a larger value indicates a higher degree of alignment between the estimated trajectory and the destination. (Velocity function) This represents the estimated speed of the unmanned surface vessel (USV) along its trajectory; a higher value indicates that the USV can travel towards its destination at a faster speed. Since the three components of the evaluation function have different dimensions, each component needs to be normalized before calculation. ,in ; and They are respectively The minimum and maximum values. (Rule function) Its weighting factor is This function determines whether a turn is needed based on the relative motion direction between the unmanned surface vessel and the obstacle. If a turn is determined to be needed, the evaluation function is adjusted.

[0041] Once the target is surrounded by the unmanned surface vessel (USV), the system switches to a behavior-based encirclement strategy to further narrow the encirclement and capture the target. Specifically, in S4, this includes: decomposing the USV's encirclement behavior into target encirclement behavior and target approach behavior. The USV's occupancy angle is calculated based on the speed ratio between the USV and the target; this occupancy angle can be obtained using the geometric properties of the Apollonius circle. The system includes... unmanned surface vessel For the target To carry out the encirclement and capture mission, among which Indicates the first A group of hunters, unmanned boats and target The distance can be expressed as When these unmanned surface vessels move at maximum speed in a specific direction, within a limited time... Inside, they will be in a certain location. An encounter. Like... Figure 5 As shown, according to geometric principles, the first The hunters With the goal Encounter Located in With the center of the circle, On a circle with radius r, this circle is called the Apollonius circle. (This is achieved by considering the target...) coordinates Draw two tangents and They are tangent to the Apollonius circle, and the points of tangency are assumed to be... and Then the tangent point , The angle formed by the position of the target This is called the occupancy angle. According to geometric relationships, the occupancy angle... Ratio to speed The relationship between them is: in, and These represent unmanned surface vessels. and target speed magnitude, It is the speed ratio between the unmanned surface vessel and the target.

[0042] Consider the existence of multiple pursuers (e.g., pursuers) and In this scenario, the Apollonius circles formed by each hunter may be tangent, disjoint, or intersecting. For example... Figure 6 As shown, the pursuers and According to polar angle The increasing order of the numbers is distributed counterclockwise around the target, and their corresponding occupancy angles are respectively... and Define the coverage angle between the surroundrs. for: when When the target is within the overlapping occupancy angle formed by adjacent pursuers along a fixed direction, this is called the overlapping occupancy angle. Exercise, such as Figure 7 As shown, then the adjacent pursuers and Both can effectively intercept the target and prevent it from escaping. Conversely, when When the target is within the escape angle and moves in a fixed direction, it may escape capture because the pursuers have not covered that angle.

[0043] To achieve target encirclement, the relationship between linear velocity and angular velocity is used to determine the coverage angle between the encirclers. The target encirclement behavior velocity model established as input is as follows: in, The bounding factor is the coefficient of encirclement. .

[0044] to surround and capture To the target distance As input, the velocity model of the target tendency behavior is established as follows: like Figure 7 As shown, the design goal tends towards behavior. Directly pointing to the target location And target encirclement behavior Then proceed in a direction perpendicular to the target's direction, either clockwise or counterclockwise. Indicates the goal In a polar coordinate system established for the pole, the pursuers polar angle, Indicating the pursuers The extreme diameter.

[0045] Finally, the two behaviors are weighted and merged to make each hunter... Not only the target The forces surrounding and concentrating within the Apollonius circle capture area formed by the pursuers can further narrow the current encirclement by approaching the target. The synthesized velocity model is represented as follows: This embodiment also provides a multi-unmanned surface vessel-aircraft cooperative capture system for unknown targets. The system consists of two types of unmanned equipment: an F450 quadcopter drone and a dual-motor differential speed unmanned surface vessel. The F450 quadcopter drone in the system is a 450mm wheelbase quadcopter with an all-carbon steel structure. A Holybro Pixhawk4 flight controller is selected as the underlying controller for the quadcopter drone. A GPS module provides external position estimation for the drone. A depth camera is mounted on the bottom of the drone for environmental perception. The drone is equipped with a Jetson Orin NX onboard computer for target detection, swarm communication, and control signal transmission. The dual-motor differential speed unmanned surface vessel uses an 830mm×523mm×300mm fishing boat as its main hardware structure. A CUAV V5+ controller is selected as the underlying controller for the dual-motor differential speed unmanned surface vessel. An external position estimation is provided via a GPS module. The unmanned surface vessel is equipped with a Jetson Xavier NX onboard computer for swarm communication and control signal transmission.

[0046] The onboard computer is loaded with an autonomous perception and planning program for a multi-unmanned surface vessel-machine cooperative encirclement system for unknown targets. The autonomous perception and planning program includes: a perception module, a data transmission module, a planning module, and a control module.

[0047] The perception module is used to acquire the category and bounding box parameters of objects in the water surface environment image, and calculate the position of the target in the world coordinate system; The data transmission module is used to broadcast the predicted location of the target and the positions of the UAV and unmanned surface vessel themselves in the multi-UAV-UAV encirclement system; The planning module is used to plan the movement of UAVs and unmanned surface vessels, and to adjust the capture strategy based on the target's movement state. The control module is used to control the flight of the UAV and the navigation of the unmanned surface vessel based on the position or speed obtained from the motion planning.

[0048] In the preferred solution, such as Figure 8As shown, the UAV uses an F450 frame (450mm wheelbase) as its main hardware structure and a Holybro Pixhawk 4 flight controller as its underlying controller. A GPS module provides external position estimation for the UAV. The power components of the quadcopter mainly include: brushless DC motors, an electronic speed controller, propellers, and batteries. The UAV uses a 900KV brushless DC motor, paired with a 4-speed 5300mAh lithium battery, 10-inch propellers, and an electronic speed controller with a maximum allowable current of 30A. The single motor pull is 1.6kg. An Intel D435i depth camera is mounted bottom-down on the UAV for environmental perception, capable of outputting depth images from 0.1-5m to obtain the distance to objects. An Nvidia Jetson OrinNX is selected as the onboard computer for the quadcopter in this invention, mounted on the UAV frame. The Jetson Orin NX features an 8-core Arm® Cortex®-A78AE v8.2 64-bit CPU with 2MB L2 + 4MB L3 cache and a maximum clock speed of 2GHz. It also includes a 1024-core NVIDIA Ampere architecture GPU with 32 Tensor Cores, providing 157 TOPS of computing power, sufficient for real-time target detection. It communicates with the PixHawk4 flight controller via a USB-to-TTL module for control signal transmission, using a serial communication protocol.

[0049] like Figure 9 As shown, the unmanned surface vessel (USV) in the system uses a fishing boat measuring 830mm × 523mm × 300mm as its main hardware structure. The USV's power system primarily consists of a brushed DC motor, electronic speed controller (ESC), propellers, and batteries. The USV uses a DEVICT 550 60W motor, paired with two 2s 11000mAh lithium batteries, a DEVICT 40 4mm reversible propeller, and a DEVICT ESC with a maximum allowable current of 35A. The single motor has a pulling force of 1.6kg. A CUAV V5+ controller is selected as the underlying controller for the dual-motor differential speed USV, providing external position estimation via a GPS module. A Jetson Xavier NX is selected as the onboard computer on the USV, enabling cluster communication via Wi-Fi and communication with the CUAV V5+ controller via a USB-to-TTL module for control signal transmission. The data transmission protocol is a serial port protocol.

[0050] In addition to the onboard computer carried on each of the aforementioned drones and unmanned surface vessels, a Dell G3 3590 laptop was used as the host on the ground to establish the cluster communication network. The Dell G3 3590 is equipped with an Intel Core i7 9750H processor, which has 4 cores and 8 threads and a base frequency of 2.4GHz.

[0051] In practical applications, the corresponding software design running on a quadcopter UAV includes the software design of the perception module, data transmission module, and control module, as well as the data interaction interfaces between these modules. The perception module enhances the depth images acquired by the depth camera and then inputs them into the YOLOv5 model for inference. It obtains the object category and bounding box parameters in the image, determines the consistency between the object category and the target category, and calculates the size of the bounding box center point based on the pixel position of the top-left point and the length and width of the bounding box when the object category matches the target category. It calls the pyrealsense2 library of the D435i depth camera to obtain the camera's intrinsic parameter matrix and acquires the pose data of the PixHawk4 flight controller via serial port to construct the extrinsic parameter matrix. Then, it converts the target pixel coordinates output by the target detection module into coordinates in the world coordinate system through a coordinate transformation process. Next, it uses the target's coordinates in the world coordinate system as a measurement value, initializes the parameters using the established target dynamic model, and iteratively updates the target state, error covariance, and Kalman filter gain using the Kalman filter algorithm to calculate the predicted target position and publish the predicted target position to the cluster communication network. The data transmission module broadcasts the predicted position of the target, as well as the positions of the UAV and unmanned surface vessel (USV), within the multi-USV-UAV encirclement system. The control module receives the predicted target position from the ` / rover_3 / mavros / setpoint_position / local` topic of the Mavros intermediate communication module between the Jetson Orin NX and PixHawk4 flight controllers. This position is used as the desired position for position tracking control, and is converted into the rotational speeds of the UAV's four motors to control the UAV to follow a predetermined trajectory.

[0052] The corresponding software design for the dual-motor differential speed unmanned surface vessel (USV) includes the software design of the data transmission module, planning module, and control module, as well as the data interaction interfaces between these modules. The data transmission module broadcasts the predicted target position and the positions of both the UAV and the USV itself within the multi-USV-UAV encirclement system. The planning module, primarily implemented by a ground-based laptop computer using the encirclement method, writes ROS nodes to calculate potential points based on the predicted target position and assigns them to the USVs. It then plans the expected velocities of the USVs to track their respective potential points using an improved dynamic window method. The USVs communicate with the CUAV V5+ controller via a mavros topic through the Jetson Xavier NX onboard computer, using the namespace `rover_x` at the beginning of the topic to distinguish different USV numbers. The control module receives the expected velocities of the USVs from the `mavros / setpoint_velocity / cmd_vel_unstamped` topic in the mavros intermediate communication module between the Jetson Xavier NX onboard computer and the CUAV V5+ controller, converts this into the rotational speeds of the USV's two motors, and controls the USVs to track their speeds.

[0053] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the multi-unmanned surface vessel-machine cooperative encirclement method for unknown targets disclosed in any of the foregoing embodiments.

[0054] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0055] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0056] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0057] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the monocular vision-based UAV formation cooperative target tracking method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0058] Furthermore, this application also discloses a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the steps of the multi-unmanned surface vessel-machine cooperative encirclement method for unknown targets disclosed in any of the foregoing embodiments.

[0059] This embodiment relates to the fields of environmental perception, cooperative encirclement algorithms, and hardware and software implementation of a multi-UAV-UAV system, enabling cooperative encirclement after the system detects an escaping target. The invention includes: the system acquiring environmental images via a depth camera mounted on a UAV, using a UAV surface target detection method for target identification and tracking, and performing coordinate transformation using camera intrinsic parameters and pose information to calculate the target's position in the world coordinate system, predicting the target's trajectory, and providing accurate target positioning data for subsequent encirclement planning; communication between encirclement members, and using a multi-UAV cooperative encirclement method based on a two-layer encirclement strategy to encircle the escaping target, effectively improving encirclement efficiency and success rate. This invention combines an improved surface target detection method with a multi-UAV cooperative encirclement method based on a two-layer encirclement strategy, and provides the system's hardware and software modules.

[0060] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0061] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0062] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0063] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0064] The foregoing has provided a detailed description of a multi-unmanned surface vessel-machine collaborative encirclement method, system, device, and storage medium for unknown targets provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for the cooperative encirclement and capture of multiple unmanned surface vessels (USVs) and unmanned aerial vehicles (UAVs) targeting unknown targets, the method being used in an unmanned system consisting of one UAV and at least three USVs, characterized in that... The unmanned aerial vehicle (UAV) performs a visual perception method, and the unmanned surface vessel (USV) performs a cooperative encirclement method. The visual perception method and the cooperative encirclement method are executed cyclically, including: S1. The UAV acquires image information of the water surface environment through a depth camera, uses a water surface target detection algorithm to obtain the category and bounding box parameters of objects in the image, and uses the intrinsic parameter matrix of the depth camera and the measured depth data combined with the UAV pose information to perform coordinate transformation to obtain the position of the target. S2. Perform Kalman filtering prior estimation based on the target state dynamic equation, combine the target state to correct the prior estimation to obtain the posterior estimation, and use the target state dynamic equation to update the posterior estimation to obtain the predicted target state. S3. Distribute potential points evenly around the target based on the number of unmanned surface vessels (USVs), and assign the nearest potential point as the target point of the USV in the world coordinate system according to the USV numbering order; plan the speed of the USVs. S4. Calculate the unmanned surface vessel's occupancy angle based on the speed ratio between the unmanned surface vessel and the target, and then calculate the coverage angle between the unmanned surface vessels to obtain the speed based on the target encirclement behavior planning; calculate the speed based on the target tendency behavior planning through the unmanned surface vessel and the target position deviation vector, and synthesize it with the speed of the target encirclement behavior planning to obtain the speed of the unmanned surface vessel based on the encirclement behavior planning.

2. The method according to claim 1, characterized in that, The target state includes the target instantaneous velocity and the target position. The target instantaneous velocity is obtained by the deviation of the target position within a time interval. The target position and the target instantaneous velocity are then combined into the target state.

3. The method according to claim 1, characterized in that, S1 includes: After obtaining the object category and bounding box parameters in the image using the water surface target detection algorithm, it is determined whether the object category and the target category are consistent. When the object category and the target category are consistent, the size of the center point of the bounding box is calculated based on the pixel position of the upper left point of the bounding box and the length and width of the bounding box to obtain the position of the target in the pixel coordinate system. Using the intrinsic parameter matrix of the depth camera and the measured depth data, coordinate transformation is performed to calculate the position of the target in the camera coordinate system. Based on the installation angle and distance of the depth camera on the UAV, a rotation transformation matrix and a translation transformation matrix are constructed from the camera coordinate system to the UAV coordinate system to calculate the position of the target in the UAV coordinate system. Obtain the UAV's position and attitude information, construct the translation and rotation transformation matrices from the UAV coordinate system to the world coordinate system, and calculate the target's position in the world coordinate system.

4. The method according to claim 1, characterized in that, S2 includes: Based on the target motion characteristics, a dynamic equation for the target state is constructed, and a measurement equation is constructed based on the target state information acquired by the depth camera. Kalman filter parameters are initialized, and prior estimates of the target state and error covariance are made. The Kalman filter gain is calculated based on the prior estimate of the error covariance. The prior estimate of the target state is corrected by the depth camera measurement value, and the prior estimate of the error covariance is corrected by the Kalman filter gain. The target state is predicted through the dynamic equation, and the predicted position of the target is published to the cluster communication network and input into the UAV flight control for target tracking.

5. The method according to claim 4, characterized in that, The process of constructing the target state dynamic equation based on the target motion characteristics includes: in, for State estimation at time 10:00 Represents the state transition matrix. It has a mean of zero and a positive definite covariance matrix of... And a white noise sequence that follows a Gaussian distribution. It is the process noise gain coefficient; For system observations, X k For the state variables of the Kalman filter system, The observation coefficient matrix, The vector has a mean of zero and a positive definite covariance matrix. And a white noise sequence that follows a Gaussian distribution, and and They are independent of each other.

6. The method according to claim 4, characterized in that, The step of publishing the predicted location of the target to the cluster communication network and inputting it into the UAV flight control for target tracking includes: The drone is equipped with a depth camera facing downwards, uses an onboard computer for target detection and trajectory prediction, communicates with the unmanned surface vessel, and tracks the target through a position control interface provided by the flight controller.

7. The method according to claim 1, characterized in that, S3 includes: The specific method for setting the potential point is as follows: the position is... The k Individual trend points With position goal The positional relationship between them is as follows: in This represents the target's yaw angle in the northeast-northeast coordinate system. The radius of the encirclement; for The range of values ​​for ; The dynamic window method calculates a comprehensive score for the predicted trajectory based on the distance from the predicted trajectory to the obstacle, the alignment between the predicted trajectory and the destination, the speed of the unmanned surface vessel on the predicted trajectory, and the potential value under the potential field of the obstacle on the predicted trajectory, and plans the speed of each unmanned surface vessel.

8. The method according to claim 7, characterized in that, The dynamic window method includes: The evaluation function for the dynamic window method is: Wherein, distance function This represents the estimated distance from the obstacle; a larger value indicates a greater distance from the obstacle. (Heading function) This represents the supplementary angle between the estimated trajectory's endpoint heading and the destination direction; a larger value indicates a higher degree of alignment between the estimated trajectory and the destination. (Velocity function) This represents the estimated speed of the unmanned surface vessel (USV) along its trajectory. A higher value indicates that the USV can travel towards its destination at a faster speed. Let be the rule function, and its weight factor be... The rule function represents the potential value of the predicted trajectory under an obstacle potential field that conforms to COLREGS. The larger the value, the higher the requirement for the predicted trajectory to turn and avoid other vessels.

9. The method according to claim 1, characterized in that, S4 includes: Occupied corner Ratio to speed The relationship between them is ; The coverage angle between unmanned surface vessels is calculated as follows: in, Indicates the goal In a polar coordinate system established for the pole, the pursuers The polar angle; Based on the relationship between linear velocity and angular velocity, the coverage angle between the pursuers As input, the velocity model for the target encirclement behavior is established as follows: in, The bounding factor is the coefficient of encirclement. ; to surround and capture To the target distance As input, the velocity model of the target tendency behavior is established as follows: 。 10. A multi-unmanned surface vessel-machine cooperative encirclement system for unknown targets, employing the method described in any one of claims 1-9, characterized in that, The onboard computer is loaded with an autonomous perception and planning program for a multi-unmanned surface vessel-machine cooperative encirclement system for unknown targets. The autonomous perception and planning program includes: a perception module, a data transmission module, a planning module, and a control module. The perception module is used to acquire the category and bounding box parameters of objects in the water surface environment image, and calculate the position of the target in the world coordinate system; The data transmission module is used to broadcast the predicted location of the target and the positions of the UAV and unmanned surface vessel themselves in the multi-UAV-UAV encirclement system; The planning module is used to plan the movement of UAVs and unmanned surface vessels, and to adjust the capture strategy based on the target's movement state. The control module is used to control the flight of the UAV and the navigation of the unmanned surface vessel based on the position or speed obtained from the motion planning.

Images