Surface-underwater unmanned vehicle cross-domain cooperative motion control method

Abstract

The application provides a water surface and underwater unmanned vehicle cross-domain cooperative motion control method, and relates to the technical field of water surface and underwater unmanned vehicle control, and the cooperative control system is realized based on a wide-angle camera of an underwater robot and a light combination installed on a water surface unmanned ship. The water surface and underwater unmanned vehicle cooperative motion control system is used for realizing the following motion of the underwater robot to the water surface unmanned ship. The underwater robot detects the light on the water surface unmanned ship through the camera, and realizes the motion regulation and control of the underwater robot by monitoring the coordinate change amount of the light in the image. By using the cooperative algorithm, the underwater robot adjusts the sailing direction and speed according to the motion information of the water surface unmanned ship, so as to continuously follow the water surface unmanned ship. The cooperative control system utilizes the visual perception of the underwater robot, realizes the intelligent cooperative motion between the water surface and underwater unmanned vehicles, and provides a simple and easy-to-implement solution for the cooperative operation of underwater detection and water surface monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control application, and particularly relates to a water-surface and underwater unmanned aerial vehicle cross-domain cooperative motion control method. BACKGROUND

[0002] In order to better detect marine resources, various countries in the world have developed different types of marine assemblies for detecting marine resources. These marine assemblies can perform various tasks, such as marine investigation, ecological monitoring and environmental protection. However, in the past research, the water surface device and the underwater device are two separate systems. At the same time, due to the short endurance time, small operation range and difficult underwater communication of single type underwater robots, they are difficult to carry out effective cooperative operation, thereby limiting their application range and performance. This water-surface and underwater cooperative motion control technology has important application value for improving the endurance of underwater robots and expanding their operation range in the field of marine scientific research. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a water-surface and underwater unmanned aerial vehicle cross-domain cooperative motion control method, which realizes efficient cooperative motion operation of the water-surface and underwater unmanned aerial vehicle.

[0004] To achieve the above purpose, the present application adopts the following technical scheme: a water-surface and underwater unmanned aerial vehicle cross-domain cooperative motion control method, the unmanned aerial vehicle comprising an underwater robot ROV and a water-surface unmanned ship USV; the underwater robot ROV body comprising left and right side plates, upper and lower side plates, a pressure-resistant cabin, and sealed components, the overall frame of the underwater robot being composed of the pressure-resistant cabin, 4 horizontal thrusters and 2 vertical thrusters being installed on the overall frame of the underwater robot, electronic components, an embedded computer and an underwater wide-angle camera being sealed in the pressure-resistant cabin body; the water-surface unmanned ship USV being a double-paddle integrated type, adjusted by differential, three illumination light sources being installed on the outside of the water-surface unmanned ship USV, the illumination light sources serving as important identification for the water-surface unmanned ship USV to be recognized by the underwater robot ROV, the cabin of the water-surface unmanned ship USV being internally provided with a Beidou-GPS positioning device, a wireless communication device, a power supply, an electronic speed regulator and an embedded computer.

[0005] In a preferred embodiment, the underwater robot ROV and the water-surface unmanned ship USV further comprise an electronic compass, which determines the heading of the unmanned aerial vehicle by detecting the size of the geomagnetic field.

[0006] In a preferred embodiment, the underwater robot ROV further comprises a depth sensor and an attitude sensor, which are used for detecting the attitude and depth control of the underwater robot ROV.

[0007] In a preferred embodiment, the underwater robot ROV detects the coordinate position of the light on the unmanned surface vehicle USV in the image according to its own wide-angle camera, since the light on the unmanned surface vehicle is installed in a triangular shape, the change of the coordinate value of the three light and the change of the pose of the triangle are used to realize the continuous following of the underwater robot ROV to the heading and position of the unmanned surface vehicle USV.

[0008] The application also provides a cross-domain cooperative motion control method of a surface-underwater unmanned vehicle, comprising the following steps:

[0009] Step S1: The camera of the underwater robot ROV is configured to ensure that it can be normally used in different underwater environments and can shoot and detect the light on the unmanned surface vehicle USV for identification;

[0010] Step S2: An image processing algorithm capable of recognizing the light source on the unmanned surface vehicle and detecting the pixel coordinate value of the light source in the image is used;

[0011] Step S3: The underwater robot captures each frame of image of the unmanned surface vehicle in real time through the camera, and then uses the image processing algorithm to detect the position of the light in the image coordinate and obtain the corresponding pixel coordinate value;

[0012] Step S4: Based on the change amount of the pixel coordinate value in the image, the speed and direction angle change of the underwater robot relative to the unmanned surface vehicle are calculated;

[0013] Step S5: A cooperative motion planning algorithm is designed, based on the continuous acquisition of the position information of the light on the unmanned surface vehicle, the motion control strategy is adjusted in real time;

[0014] Step S6: Based on the above steps, the final result is converted to the propeller of the unmanned vehicle to realize the cooperative motion of the unmanned vehicle.

[0015] In a preferred embodiment, the Otsu method is used to ensure good binarization effect in different environments; the Otsu method adaptively determines the image threshold by minimizing the intra-class variance and maximizing the inter-class variance; the Otsu method assumes that the image contains two classes, i.e. foreground and background, and finds a threshold value that maximizes the variance between the two classes and minimizes the intra-class variance; the Otsu method first needs to calculate the gray level histogram of the image:

[0016]

[0017] Where n i is the number of pixels with a gray level of i, N is the total number of pixels of the image, and p is the probability of occurrence; therefore, the cumulative probability P i accumulated to the gray level i is:

[0018]

[0019] For gray levels from 0 to k, the average gray level u is:

[0020]

[0021] According to the above formula, for each possible threshold t, assuming that the average gray level less than the threshold t is u0, and the average gray level greater than the threshold t is u1, the local variances on both sides of the threshold t are:

[0022]

[0023]

[0024] For each possible threshold t, its intra-class variance σ ω 2 and the between-class variance σ b 2 The calculation is as follows:

[0025]

[0026]

[0027] Where u T is the average gray value of the entire image, and the inter-class variance and intra-class variance The following conclusions are drawn:

[0028]

[0029] In the above formula Indicates that the total variance of the image is a constant. When the inter-class variance is maximized, that is, the intra-class variance is minimized, the formula The maximized gray level k is the optimal threshold.

[0030] In a preferred embodiment, to obtain the optimal threshold, it is only necessary to traverse and calculate the inter-class variance of each gray level. The maximum between-class variance calculated The corresponding grayscale value k is used as the threshold to binarize the image, and edge detection is performed on the binarized image to further extract the center coordinate position of the light spot.

[0031] In a preferred embodiment, the pixel coordinates obtained after extracting the feature edge present a triangular geometry, and the relative angle change between the ROV and the USV can be obtained according to the change amount of the pixel coordinates of the feature points in each frame of image and the rotational change of the triangle, and the relative angle change is obtained according to the rotation of the rigid body, so only the included angle between the straight lines corresponding to the sides of the triangle before and after the change is required, and the two point coordinates of the triangle are (x0, y0) and (x1, y1), and the changed coordinates are (x'0, y'0) and (x'1, y'1), and the relative angle θ change can be obtained according to the inner product of vectors as follows:

[0032]

[0033] In the above formula, V1 represents the direction vector before the change, and V2 represents the direction vector after the change, and the angle obtained is the rotation angle of the ROV relative to the USV, at this time, only the rotation θ angle of the ROV needs to be adjusted, and the adjustment direction can be judged by the positive and negative of the vector cross product; at the same time, in order to ensure that the USV is within the detectable range of the camera, the coordinate range of the light in the image needs to be limited, when the pixel coordinates of the light source reach the image boundary point, the ROV needs to move away from the boundary in the reverse direction, to ensure that it can be within the effective detection range.

[0034] Compared with the prior art, the present application has the following beneficial effects: the cooperative control system utilizes the visual perception of the underwater robot, realizes the intelligent cooperative motion between the water surface unmanned aerial vehicle and the underwater unmanned aerial vehicle, and provides a simple and easy-to-implement solution for cooperative operations such as underwater detection and water surface monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a device ROV structure schematic diagram of the embodiment of the present application;

[0036] Figure 2 is a device USV structure schematic diagram of the embodiment of the present application;

[0037] Figure 3 is a detection principle block diagram of the embodiment of the present application;

[0038] Figure 1 In the middle: 1-buoyancy block protection shell, 2-right side support, 3-underwater thruster, 4-lead counterweight, 5-lower support, 6-waterproof connection plug, 7-threaded screw, 8-depth sensor, 9-left side support, 10-sensor and electronic control board, 11-wide-angle camera, 12-tao underwater light, 13-transparent pressure-resistant cabin, 14-front view transparent cover and front view camera, 15-upper transparent support.

[0039] Figure 2Middle: 101-integrated hull, 201-communication antenna, 301-Beidou-GPS signal receiving antenna, 401-unmanned boat hull light identification, 501-unmanned boat sealing cover, 601-thruster. DETAILED DESCRIPTION

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0042] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0043] like Figures 1-3 As shown, this embodiment provides a measurement device for a cross-domain collaborative motion control method of a surface and underwater UAV, including an ROV and a USV. ROV is an underwater robot such as Figure 1 As shown, USV is an unmanned surface vessel. Figure 2The ROV body includes four buoyancy block protective shells 1, a right side plate 2, six underwater thrusters 3, four counterweight lead blocks 4, two lower side plates 5, three water-tight connection plugs 6, three threading screws 7, a depth sensor 8, a left side plate 9, electronic components and a main control board 10, an underwater wide-angle camera 11, two underwater illuminating lamps 12, a transparent pressure-resistant cabin 13, a front-view transparent cover and a front-view camera 14, and an upper side plate 15. The buoyancy block protective shells 1 and the horizontal four thruster 3 mounting seats are arranged on the upper side plate 15; the vertical thrusters 3 and the illuminating lamps are symmetrically arranged on the left side plate 2 and the right side plate 9; the counterweight lead blocks 4 are symmetrically arranged on the lower side plates 5; the water-tight connection plugs 6 are used for connecting the thrusters 3 and the main control board 10; the threading screws 7 are used for connecting the underwater illuminating lamps 12 and the main control board 10; the depth sensor 8 is connected with the main control board 10 through an IIC protocol and is used for collecting depth data; and the wide-angle camera 11 communicates with the main control board 10 through a Port port after data processing by a built-in MCU. The USV is a double-paddle integrated type, and the USV body includes an integrated hull 101, a communication module 201 and an antenna 2, a Beidou-GPS positioning module 301, three unmanned ship light identifiers 401, a hull sealing cover 501, and two thrusters 601. The USV realizes differential adjustment through the two thrusters; three unmanned ship light identifiers 401 are arranged on the outer side of the USV, two on the left side and one on the right side, and the light source serves as an important identifier for identifying the USV by the ROV; and the unmanned ship cabin is internally provided with the communication module 201, the Beidou-GPS positioning module 301, a power supply, an electronic speed regulator, an embedded computer and other equipment.

[0044] The application provides a water surface and underwater unmanned vehicle cross-domain cooperative control method based on the above device, which comprises the following steps:

[0045] Step S1: The camera of the ROV is configured to ensure that it can be normally used in different underwater environments and can shoot and detect the light on the USV for identification;

[0046] Step S2: An image processing algorithm capable of identifying the light source on the unmanned ship and detecting the pixel coordinate value of the light source in the image is used;

[0047] Step S3: The underwater robot captures each frame of image of the water surface unmanned ship in real time through the camera, and then uses the image processing algorithm to detect the position of the light in the image coordinate and obtain the corresponding pixel coordinate value;

[0048] Step S4: Based on the change amount of the pixel coordinate value in the image, the speed and direction angle change of the underwater robot relative to the water surface unmanned ship are calculated;

[0049] Step S5: A cooperative motion planning algorithm is designed, and the position information of the light on the water surface unmanned ship is continuously obtained to adjust the motion control strategy in real time;

[0050] Step S6: Based on the above steps, the final result is converted to the propeller of the UAV, realizing the cooperative movement of the UAV.

[0051] Further, the light image detection method is Otsu method. This method adaptively determines the threshold by minimizing the intra-class variance and maximizing the inter-class variance. This method assumes that the image contains two classes (foreground and background), and finds a threshold that maximizes the variance between the two classes and minimizes the intra-class variance. This method first needs to calculate the gray histogram of the image:

[0052]

[0053] Where n i is the number of pixels with gray level i, N is the total number of pixels in the image, and p is the probability of occurrence. Therefore, the cumulative probability P i accumulated to the gray level i is:

[0054]

[0055] The average gray level u from 0 to k is:

[0056]

[0057] According to the above formula, for each possible threshold t, assuming that the average gray level less than the threshold t is u0, and the average gray level greater than the threshold t is u1, then the local variance on both sides of the threshold t is:

[0058]

[0059]

[0060] For each possible threshold t, the intra-class variance σ ω 2 and the inter-class variance σ b 2 are calculated as follows:

[0061]

[0062]

[0063] In the formula, u T is the average gray value of the entire image, and for the inter-class variance and the intra-class variance , the following conclusions are drawn:

[0064]

[0065] In the above formula The total variance of the image is a constant value, when the inter-class variance is maximized, that is, the intra-class variance is minimized, the formula The gray level k which is maximized is the optimal threshold value.

[0066] Further, according to the above, to obtain the optimal threshold value, only the inter-class variance of each gray level needs to be traversed and calculated The calculated maximum inter-class variance The corresponding gray value k is the threshold value, the image is binarized, and the edge of the binarized image is detected to further extract the center coordinate position of the light spot.

[0067] The pixel coordinates obtained after extracting the feature edge present a triangular geometric shape, according to the change amount of the feature point coordinate pixel coordinates in each frame image and the rotation change of the triangle, the relative angle change of the ROV and the USV can be obtained, according to the rotation of the rigid body, therefore, only the included angle before and after the straight line corresponding to the side of the triangle is required to be the change amount of the front and rear angles, here, it is assumed that the two point coordinates of the triangle are (x0, y0) and (x1, y1), the changed coordinates are (x'0, y'0) and (x1', y1'), according to the inner product of the vector, the relative angle θ change is:

[0068]

[0069] In the above formula, V1 represents the direction vector before the change, and V2 represents the direction vector after the change, according to the obtained angle, the rotation angle of the ROV relative to the USV is obtained, at this time, only the ROV needs to be adjusted by θ angle, and the adjustment direction can be judged by the positive and negative of the vector cross product. At the same time, in order to ensure that the USV is within the detectable range of the camera, the coordinate range of the light in the image needs to be limited, when the pixel coordinates of the light source reach the image boundary point, the ROV needs to move reversely to ensure that it can be within the effective detection range.

[0070] The above is only a preferred embodiment of the present application, and is not intended to limit the present application in other forms, any skilled person in the art can modify or change the above disclosed technical content into equivalent embodiments. However, any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present application, which does not deviate from the technical solution content of the present application, still belongs to the protection scope of the present application.

Claims

1. A control method for a drone with cross-domain coordinated motion on the surface and underwater, characterized in that: The drones include underwater robots (ROVs) and surface unmanned vessels (USVs); the ROV body includes left and right side panels, upper and lower side panels, a pressure-resistant cabin, and sealing components to form the overall frame of the ROV. Four horizontal thrusters and two vertical thrusters are installed on the overall frame of the underwater robot, and electronic components, an embedded computer, and an underwater wide-angle camera are sealed in the pressure-resistant cabin. The surface unmanned vessel (USV) is a dual-propeller integrated type that is adjusted by differential speed. Three lighting sources are installed on the outside of the USV, which serve as important identification for the ROV to identify the surface unmanned vessel (USV). The USV cabin is equipped with Beidou-GPS positioning, wireless communication, power supply, electronic speed regulator, and embedded computer. The following steps are involved: Step S1: Configuring the camera of the underwater robot (ROV) to ensure that it can be used normally in different underwater environments and that it can capture and detect the marking lights on the surface unmanned vessel (USV); Step S2: using an image processing algorithm that can identify the light source on the unmanned boat and detect the pixel coordinate values ​​of the light source in the image; Step S3: The underwater robot captures each frame of the surface unmanned boat through a camera in real time, and then uses an image processing algorithm to detect the position of the light in the image coordinates to obtain the corresponding pixel coordinate values; Step S4: Calculate the speed and direction angle change of the underwater robot relative to the surface unmanned vehicle based on the change in pixel coordinate values ​​in the image; Step S5: Design a collaborative motion planning algorithm to adjust the motion control strategy in real time based on the continuous acquisition of the position information of the lights on the surface unmanned vehicle; Step S6: Based on the above steps, the final result is converted to the propeller of the UAV to achieve the coordinated movement of the UAV; The Otsu method is used to ensure good binarization results in different environments. The Otsu method adaptively determines the image threshold by minimizing the intra-class variance and maximizing the inter-class variance. The Otsu method assumes that the image contains two classes, foreground and background, and finds a threshold that maximizes the variance between the two classes while minimizing the variance within each class. The Otsu method first requires calculating the grayscale histogram of the image: , where n i is the number of pixels with gray level i, N is the total number of pixels in the image, and p is the probability of occurrence; therefore, the cumulative probability P for gray level i is i for: , the average grayscale u for grayscale levels from 0 to k is: , according to the above formula, for each possible threshold t, assuming that the average gray level less than the threshold t is , the average gray level greater than the threshold t is , then the local variances on both sides of the threshold t are: , , for each possible threshold t, its intra-class variance and between-class variance The calculation is as follows: , , where is the average gray value of the entire image, and the inter-class variance and intra-class variance The following conclusions are drawn: , in the above formula Indicates that the total variance of the image is a constant. When the inter-class variance is maximized, that is, the intra-class variance is minimized, the formula The maximized gray level k is the optimal threshold.

2. The control method of a surface-underwater cross-domain coordinated motion UAV according to claim 1 is characterized in that: According to the above, in order to obtain the optimal threshold, it is only necessary to traverse and calculate the inter-class variance of each gray level , the maximum between-class variance calculated The corresponding grayscale value k is used as the threshold to binarize the image, and edge detection is performed on the binarized image to further extract the center coordinate position of the light spot.

3. The control method of a surface-underwater cross-domain coordinated motion UAV according to claim 2 is characterized in that: The pixel coordinates obtained after extracting the feature edge present a triangular geometric shape. The relative angle change between ROV and USV can be obtained according to the change in the pixel coordinates of the feature point coordinates in each frame image and the rotation change of the triangle. According to the rotation of the rigid body, it is only necessary to obtain the angle between the front and back of the corresponding side straight line of the triangle, which is the change in the front and back angle. Here, it is assumed that the coordinates of the two points of the triangle are and The changed coordinates are and , the relative angle can be obtained according to the inner product of the vector Changes to: In the above formula, V1 represents the direction vector before the change, and V2 represents the direction vector after the change. The required angle is the rotation angle of ROV relative to USV. In this case, you only need to adjust the ROV rotation. The angle can be adjusted, and its adjustment direction can be determined by the sign of the vector cross product. At the same time, to ensure that the USV is within the detectable range of the camera, the coordinate range of the light in the image needs to be limited. When the pixel coordinate of the light source reaches the boundary point of the image, the ROV needs to move in the opposite direction away from the boundary to ensure that it is within the effective detection range.

Images