Differentiated beacon optical positioning method for underwater robots

Abstract

The present invention discloses an underwater differentiated beacon optical positioning system and method. The system comprises a differentiated positioning array consisting of a long-range primary beacon and a short-range secondary beacon, connected to a power supply within a control cabin. An underwater camera and a visual recognition module within an underwater robot cabin form an underwater visual recognition system. The visual recognition module receives camera video signals and outputs positioning information to the underwater robot controller. The system also includes a neural network-based positioning recognition algorithm that processes the differentiated beacon array image through a recognition, extraction, tracking, and pose estimation computational framework to obtain real-time positioning information. The system meets the real-time requirements for underwater robot mobile positioning, exhibits strong adaptability and stability, and can effectively extend the range of underwater optical positioning, making it suitable for underwater robot navigation and positioning.

Description

Technical Field

[0001] The present invention discloses an underwater differentiated beacon optical positioning method based on a neural network, which is suitable for the mobile positioning of underwater robots. Technical Background

[0002] my country is currently vigorously developing autonomous underwater vehicles (AUVs). These vehicles boast autonomy, intelligence, a wide range of motion, and maneuverability. They can carry a variety of sensors and operating components, significantly expanding the scope of marine exploration operations. AUVs primarily consist of autonomous driving, navigation and positioning, self-diagnosis and troubleshooting, measurement equipment, and energy sources. Navigation and positioning are crucial for AUVs to plan their underwater mission paths. At close range, optical positioning enables AUVs to operate with extremely high precision and stability, making it the primary method for precise underwater positioning at close range.

[0003] Key indicators for underwater positioning are range, accuracy, and update frequency. Common methods include optical and acoustic methods. Optical methods offer higher accuracy and frequency than underwater acoustic positioning. Compared to acoustic methods' meter-level positioning accuracy and 1Hz positioning frequency, optical methods can achieve centimeter-level positioning and 30Hz real-time processing. Therefore, increasing the optical range is a future trend in optical positioning methods. Light source and imaging system design are key to improving the effective range of underwater robot optical positioning. By designing differentiated light sources, the optical characteristics of different light source distributions underwater can be fully utilized, providing different positioning strategies at different distances.

[0004] Visual recognition requires specific algorithms to identify and locate multi-light source arrays. Currently, the predominant localization algorithm is based on feature extraction, which offers high computational speed but cannot effectively detect changes in the target. In recent years, the development of neural network algorithms has led to a shift in the field of target recognition due to their superior performance in recognition accuracy and generalizability to target changes. Compared to traditional algorithms, neural network algorithms offer higher accuracy but slower computational speeds. With the advancement of hardware, high-performance embedded computing platforms dedicated to neural networks are now available, suitable for mobile robots.

[0005] However, there are no reports on optical positioning methods for underwater robots using differentiated beacons and neural networks. Summary of the Invention

[0006] The present invention discloses an underwater differentiated beacon optical positioning device and an identification method thereof, and proposes a detection-tracking technical route based on a neural network for a differentiated beacon array.

[0007] In order to achieve the above objectives, the technical solution of the present invention is as follows: an underwater robot differentiated beacon optical positioning system, comprising:

[0008] The optical navigation node includes multiple light sources of different colors and is located on the outer surface of the cable.

[0009] A camera, used to collect light spot images of the optical navigation node;

[0010] The positioning unit is used to extract the light source coordinates and calculate the position and posture of the light spot image to achieve optical positioning of the underwater robot.

[0011] The wavelengths of light emitted by the multiple light sources are the same.

[0012] The wavelength of light emitted by the multiple light sources is 450 nm.

[0013] The multiple light sources are all blue light sources.

[0014] Among the multiple light sources, at least one light source has a brightness greater than that of the other light sources; and the other light sources have the same brightness.

[0015] The method for optical positioning of an underwater robot using differentiated beacons includes the following steps:

[0016] Collect the light spot image of the optical navigation node through the camera;

[0017] The positioning unit extracts the light source coordinates and calculates the position and posture of the light spot image to achieve optical positioning of the underwater robot.

[0018] The positioning unit extracts the light source coordinates and calculates the position and posture of the light spot image to achieve optical positioning of the underwater robot, including the following steps:

[0019] Coordinate extraction:

[0020] 1) Identify the target light source from the spot image through a neural network;

[0021] 2) Using the BLOB method in the RGB blue channel, light source extraction is performed in the target light source area to obtain the independent image coordinates of multiple target light sources;

[0022] 3) Using the Hungarian matching algorithm, the independent image coordinates of multiple target light sources are matched and sorted to obtain the coordinates of multiple target sequence images;

[0023] 4) Determine the overlapping area between the image sequence coordinates of the current frame and the image sequence coordinates of the previous frame, and calculate the intersection over union (IoU) of the target area of ​​the current frame and the target area of ​​the previous frame, that is, the ratio of the overlapping area to the total area of ​​the two areas; if the IoU is greater than the threshold, the data is considered valid and the next step is executed; if not, the data is sent to exception handling and returns to step 1);

[0024] Position and attitude calculation:

[0025] When the number of light sources n meets the array positioning conditions, two-degree-of-freedom and six-degree-of-freedom solutions are performed on the sequence image coordinates to obtain positioning information with different degrees of freedom.

[0026] The differentiated beacon optical positioning method for underwater robots also includes a reprojection closed-loop mechanism: the positioning information is obtained by inversely solving the PnP algorithm to obtain the coordinate projection information of the light source on the spot image under the current position and posture of the vehicle, thereby realizing parallel real-time positioning of tracking and detection.

[0027] The array positioning condition is n≥4.

[0028] The six-degree-of-freedom solution obtains the three-dimensional position coordinates and posture of the underwater robot relative to the optical positioning array through the PnP algorithm; the two-degree-of-freedom solution obtains the heading information of the underwater robot's pitch and heading two Euler angles relative to the optical positioning array based on the target light source.

[0029] The present invention has the following advantages:

[0030] 1. The present invention can greatly improve the working distance of optical positioning;

[0031] 2. By combining the characteristics of different beacon forms, the present invention can meet different underwater positioning accuracy requirements at different distances;

[0032] 3. The present invention greatly improves the processing capability of the system by utilizing the extremely high computing speed of the high-performance embedded system;

[0033] 4. The present invention can flexibly modify the array configuration according to actual use requirements, and has strong practicality and scalability;

[0034] 5. The recognition algorithm of the present invention has strong recognition capabilities for different beacon forms, and the visual recognition system is reliable and robust to the transformation, combination, and loss of the array in water.

[0035] 6. The present invention combines the advantages of high precision of neural networks with the high speed of traditional methods, thus taking into account both the accuracy and speed of the positioning system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a structural diagram of the device of the present invention;

[0037] Figure 2a This is a top-down diagram of the beacon array light path distribution;

[0038] Figure 2b This is a front view schematic diagram of the beacon array light path distribution diagram;

[0039] Figure 3 Flowchart of the control program adopted for the control method. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below with reference to the embodiments and accompanying drawings.

[0041] The underwater vision positioning system device includes:

[0042] The long-range primary beacon and the short-range secondary beacon array are connected to the underwater positioning control cabin via watertight connecting cables;

[0043] The underwater camera is connected to the visual recognition module through a network to form an underwater visual recognition system;

[0044] The visual recognition module receives the camera video signal and outputs the positioning information to the underwater robot controller

[0045] The neural network module trains the YOLO neural network by collecting 2,000 real underwater beacon images and manually annotating the results, and obtains the ability to identify whether there are beacons in color images and the location information of the beacons.

[0046] The visual recognition method of the present invention is as follows:

[0047] First, start the system and initialize the embedded computing platform; the system initializes communication, waits for receiving network video signal data, and performs data processing;

[0048] Data processing is divided into scheduled data transmission and differentiated beacon real-time positioning algorithm based on neural network:

[0049] Timed data transmission: Regularly judge whether there is new data in the sending buffer. If there is new data in the sending buffer, write the sending data, start sending, and send the data to the host computer for processing; if there is no new data in the sending buffer, report an error and perform exception processing;

[0050] Differentiated beacon real-time positioning algorithm based on neural network: First, the neural network is used to detect and identify the color image transmitted by the camera, and the position and size of the target in the image are obtained. Then, the image is cropped and feature extracted at this position, and the center of each beacon light is located. Then, each light is tracked and the relative position and posture of the camera and the positioning beacon are given through the position settlement algorithm. Finally, the data is processed to determine the validity of the data; if the data is invalid, the data is put into exception processing and re-circulated; if the data is valid, the data is entered into the cache area to be sent; since the feature extraction positioning algorithm is faster than the neural network detection, the target tracking self-loop is performed based on the historical data prediction, and the recognition correction is performed after the settlement in the neural network module is completed.

[0051] The primary beacon and secondary beacon are connected to the positioning control cabin; the underwater camera is connected to the visual recognition module through the network to transmit video signals; the visual recognition module is connected to the robot control cabin through the network to transmit positioning signals;

[0052] The primary and secondary beacons form different light source characteristics underwater. The primary beacon has a large operating range and a small angle, and has a guiding capability; the secondary beacon has a small operating range and a large angle. Multiple secondary beacons form a positioning light array with positioning capability.

[0053] The primary and secondary beacons form differentiated light source distribution underwater, as shown in Figure 2.

[0054] The underwater camera is connected to the visual recognition module through the network to transmit video data; the visual recognition module is connected to the robot control cabin through the network to transmit positioning signals.

[0055] First, start the system and initialize the embedded computing platform; the system initializes communication, waits for receiving network video signal data, and performs data processing;

[0056] Data processing is divided into scheduled data transmission and differentiated beacon real-time positioning algorithm based on neural network:

[0057] Timed data transmission: Regularly judge whether there is new data in the sending buffer. If there is new data in the sending buffer, write the sending data, start sending, and send the data to the host computer for processing; if there is no new data in the sending buffer, report an error and perform exception processing;

[0058] Differentiated beacon real-time positioning algorithm based on neural network: First, the neural network is used to detect and identify the color image transmitted by the camera, and the position and size of the target in the image are obtained. Then, the image is cropped and feature extracted at this position, and the center of each beacon light is located. Then, each light is tracked and the relative position and posture of the camera and the positioning beacon are given through the position settlement algorithm. Finally, the data is processed to determine the validity of the data; if the data is invalid, the data is put into exception processing and re-circulated; if the data is valid, the data is entered into the cache area to be sent; since the feature extraction positioning algorithm is faster than the neural network detection, the target tracking self-loop is performed based on the historical data prediction, and the recognition correction is performed after the settlement in the neural network module is completed.

[0059] The primary and secondary beacons generate distinct light sources underwater. The primary and secondary beacon arrays are mounted on the recovery base station, perpendicular to the horizontal plane. An underwater vehicle, at the same depth as the center of the optical beacon array, moves horizontally toward the recovery base station's optical guidance array. When the vehicle enters the beacon's light range, the system initiates optical guidance.

[0060] like Figure 2aAs shown in the side view of the optical path distribution diagram, the main beacon has a large operating distance and a small angle, and has a guiding ability; the secondary beacon has a small operating distance and a large angle, and multiple secondary beacons form a positioning light array with positioning ability; the beacon distribution form is as follows: Figure 2b As shown in the front view diagram of the optical path distribution diagram, the primary beacon and the secondary beacon together form a surrounding light array, whose positioning center is consistent with the array center. The underwater robot moves to the positioning center through the circular array navigation and positioning.

[0061] The underwater differentiated beacon positioning system structure of the present invention is as follows: Figure 1 As shown, the primary beacon and the secondary beacon are electrically connected to the positioning control cabin through watertight connecting cables; the underwater camera is connected to the visual Jetson TX2 recognition module through the network to transmit video signals; the Jetson TX2 visual recognition module is connected to the robot control cabin through the network to transmit positioning signals;

[0062] The controller in this example uses Nvidia's Jetson TX2 platform. The Jetson TX2 is equipped with two CPUs, a GPU with 256 CUDA cores, 8GB of RAM, and 32GB of flash storage. The Jetson TX2 offers two operating modes: MAX Q, which maximizes energy efficiency with power consumption below 7.5W; and MAX P, which maximizes performance with power consumption below 15W. These modes can be switched based on computing speed and requirements. At 15W, the Jetson TX2 can achieve 1.3 TFLOPS of computing power.

[0063] The underwater positioning method of the present invention is implemented by a neural network detection and recognition program stored in a visual module. For the specific process, see Figure 3 .

[0064] First, start the system and initialize the embedded computing platform; the system initializes communication, waits for receiving network video signal data, and performs data processing;

[0065] The data processing part is divided into scheduled data transmission and differentiated beacon positioning algorithm based on neural network;

[0066] Timed data transmission: Regularly judge whether there is new data in the sending buffer. If there is new data in the sending buffer, write the sending data, start sending, and send the data to the host computer for processing; if there is no new data in the sending buffer, report an error and perform exception processing;

[0067] Differentiated beacon positioning algorithm based on neural network:

[0068] Coordinate extraction: First, the color image is processed through YOLO neural network recognition to obtain the image coordinates of the overall target of the guidance array; then, the RGB blue channel BLOB method is used to extract the light source within the range of the target image coordinates, effectively filtering out other interfering light sources and obtaining the independent image coordinates of the multi-target light sources; the Hungarian matching algorithm is used to match and sort the independent image coordinates of the multi-target light sources to obtain the multi-target sequence image coordinates; the sequence image coordinates are judged for overlapping areas with the previous frame results, and the intersection of union (IoU) of the two areas is calculated, that is, the ratio of the overlapping area to the total area of ​​the two areas. If the IoU is greater than 0.8, the data is judged to be valid and the process continues; if it is invalid, it is sent to exception processing and enters the next loop;

[0069] Position and attitude solution: Determine whether the number of beacons n meets the array positioning requirements, and enter the two-degree-of-freedom and six-degree-of-freedom solution processes respectively to obtain positioning information of different degrees of freedom. The six-degree-of-freedom uses the PnP algorithm, which requires n ≥ 4, while extracting the two-degree-of-freedom information only requires the image coordinates of one beacon to convert it into the heading information of the two Euler angles of pitch and heading. Here, for n = 2, 3, the overall image coordinates are used; the positioning information is compared with the previous frame result to determine the validity of the position error. If invalid, it is sent to exception processing. If valid, the data is used as the output of this positioning and stored in the cache;

[0070] Reprojection closed-loop mechanism: The positioning information in the cache area is obtained by reversely solving the PnP algorithm to obtain the coordinate projection information of the beacon on the image under the current position and attitude of the spacecraft; a closed loop is formed through the reprojection mechanism. Since the neural network is slower than the light source extraction to degree of freedom positioning solution process, reprojection is used instead of the neural network recognition result for fast matching, realizing parallel real-time positioning of tracking and detection.

[0071] This algorithm combines the advantages of traditional algorithms and neural network algorithms to achieve fast and accurate target recognition. It uses traditional algorithms to quickly extract features from images, and then uses neural network algorithms to accurately identify these features.

Claims

1. A differentiated beacon optical positioning method for underwater robots, characterized in that: The following steps are involved: Collect the light spot image of the optical navigation node through the camera; The positioning unit extracts the light source coordinates and calculates the position and posture of the light spot image to achieve optical positioning of the underwater robot; The positioning unit extracts the light source coordinates and calculates the position and posture of the light spot image to achieve optical positioning of the underwater robot, including the following steps: Coordinate extraction: 1) Identify the target light source from the spot image through a neural network; 2) Using the BLOB method in the RGB blue channel, light source extraction is performed in the target light source area to obtain the independent image coordinates of multiple target light sources; 3) Using the Hungarian matching algorithm, the independent image coordinates of multiple target light sources are matched and sorted to obtain the coordinates of multiple target sequence images; 4) Determine the overlapping area between the image sequence coordinates of the current frame and the image sequence coordinates of the previous frame, and calculate the intersection over union (IoU) of the target area of ​​the current frame and the target area of ​​the previous frame, that is, the ratio of the overlapping area to the total area of ​​the two areas; if the IoU is greater than the threshold, the data is considered valid and the next step is executed; if not, the data is sent to exception handling and returns to step 1); Position and attitude calculation: When the number of light sources n meets the array positioning conditions, two-degree-of-freedom and six-degree-of-freedom solutions are performed on the sequence image coordinates to obtain positioning information with different degrees of freedom. It also includes a reprojection closed-loop mechanism: the positioning information is obtained by inversely solving the PnP algorithm to obtain the coordinate projection information of the light source on the spot image under the current position and attitude of the spacecraft, realizing parallel real-time positioning of tracking and detection.

2. The underwater robot differentiated beacon optical positioning method according to claim 1, characterized in that: The array positioning condition is n≥4.

3. The underwater robot differentiated beacon optical positioning method according to claim 1, characterized in that: The six-degree-of-freedom solution obtains the three-dimensional position coordinates and posture of the underwater robot relative to the optical positioning array through the PnP algorithm; the two-degree-of-freedom solution obtains the heading information of the underwater robot's pitch and heading two Euler angles relative to the optical positioning array based on the target light source.

Images