A real-time detection method and device for underwater visual guidance based on deep learning

Abstract

A deep learning-based underwater visual guidance real-time detection method comprises: S1, acquiring underwater images as a data set; using the data set to train a YOLOv5s detection model until the detection accuracy reaches a preset accuracy requirement, completing the training; calibrating an underwater camera to obtain internal parameters of the underwater camera; S2, during the autonomous docking and recovery process of an underwater vehicle, the underwater camera captures underwater images; inputting the captured underwater images into a trained YOLOv5s detection model to obtain the category and corresponding coordinates of a visual guidance target; according to the coordinates of the visual guidance target, using the internal parameters of the underwater camera to eliminate distortion and convert to the underwater camera coordinate system, and obtaining the relative position of the underwater vehicle and the visual guidance target.

Description

Technical Field

[0001] The present invention relates to a deep learning-based underwater visual guidance real-time detection method and device, and relates to the technical field of underwater autonomous docking and recovery of underwater autonomous vehicles. Background Art

[0002] For underwater vehicles that operate for a long time, they need to be recovered to complete the energy replenishment, mission instruction update, data download and equipment maintenance of the vehicle. The underwater vehicle autonomous docking and recovery technology enables the vehicle and the docking device to achieve autonomous homing, approach, docking, locking, etc. through guidance and positioning, which is of great significance for improving the endurance of the vehicle and improving the operation efficiency of the vehicle.

[0003] In the terminal guidance stage when the distance between the vehicle and the docking device is relatively close, visual guidance can be used to provide the vehicle with the relative position information of the docking device. However, due to the nature of the water body and the presence of microorganisms, underwater images have defects such as uneven illumination, high noise interference, low contrast, and blurred texture features. Traditional guidance detection methods have poor detection effects on underwater guidance light sources, resulting in a low success rate for underwater vehicle docking and recovery. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and solve the problem of underwater autonomous docking and recovery of an underwater vehicle with a docking device.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A real-time underwater visual guidance detection method based on deep learning, comprising:

[0007] S1. Obtain underwater images as a data set; use the data set to train the YOLOv5s detection model until the detection accuracy reaches the preset accuracy requirement, completing the training;

[0008] Calibrate the underwater camera to obtain the internal parameters of the underwater camera;

[0009] S2. During the autonomous docking and recovery process of the underwater vehicle, the underwater camera captures underwater images;

[0010] Input the captured underwater images into the trained YOLOv5s detection model to obtain the category and corresponding coordinates of the visual guidance target;

[0011] According to the coordinates of the visual guidance target, the internal parameters of the underwater camera are used to eliminate the distortion and transform it into the underwater camera coordinate system to obtain the relative position of the underwater vehicle and the visual guidance target.

[0012] Preferably, after the relative position between the underwater vehicle and the visual guidance target is obtained, the underwater vehicle adjusts its posture according to the relative position, and S2 is repeated until the underwater vehicle reaches a predetermined position.

[0013] Preferably, the categories of the visual guidance targets include guiding light sources and docking devices;

[0014] Before the underwater vehicle head enters the docking device, a guiding light source is used as an effective visual guidance target;

[0015] After the head of the underwater vehicle enters the docking device, the docking device is used as an effective visual guidance target.

[0016] Preferably, when the guiding light source is used as an effective visual guiding target, the underwater vehicle navigates to the same depth as the docking device and performs a fixed depth movement;

[0017] When the docking device is used as an effective visual guidance target, the relative position is calculated by detecting the channel center coordinates of the docking device. The underwater vehicle adjusts its position and locks the effective visual guidance target, and continues to navigate under visual guidance until the underwater vehicle completely enters the docking device.

[0018] Preferably, the data set includes positive samples and negative samples; wherein the positive samples are images of underwater guidance light sources and docking devices taken at different angles during multiple simulations of autonomous docking and recovery of underwater vehicles; and the negative samples are underwater images without guidance targets.

[0019] Preferably, the training of the YOLOv5s detection model is specifically as follows: dividing the data set into a training data set, a verification data set, and a test data set, using the training data set to train the YOLOv5s detection model, each time an iteration of the entire training set is completed, the YOLOv5s detection model is verified using the verification data set, and when the accuracy reaches the preset accuracy requirement, it is saved, and the test data set is used for testing.

[0020] An underwater visual guidance real-time detection device based on deep learning, comprising:

[0021] An image acquisition module, used to acquire underwater visual guidance images and send them to a processor;

[0022] The processor uses the existing underwater images as a data set to train the YOLOv5s detection model until the detection accuracy reaches the preset accuracy requirement, completing the training; during the autonomous docking and recovery process of the underwater vehicle, the captured underwater images are input into the pre-trained YOLOv5s detection model to obtain the category and corresponding coordinates of the visual guidance target; according to the coordinates of the visual guidance target, the internal parameters of the underwater camera are used to eliminate the distortion and convert it to the underwater camera coordinate system to obtain the relative position of the underwater vehicle and the visual guidance target.

[0023] Preferably, the detection device further comprises a control module, and the control module is used to call the processor and obtain the relative position, and control the underwater vehicle to adjust its posture until the underwater vehicle reaches a predetermined position.

[0024] Preferably, the categories of the visual guidance targets include guiding light sources and docking devices;

[0025] Before the underwater vehicle head enters the docking device, the processor uses the guidance light source as an effective visual guidance target;

[0026] After the head of the underwater vehicle enters the docking device, the processor uses the docking device as an effective visual guidance target.

[0027] Preferably, when the guiding light source is used as an effective visual guiding target, the control module controls the underwater vehicle to navigate to the same depth as the docking device and perform fixed-depth movement;

[0028] When the docking device is used as an effective visual guidance target, the processor calculates the relative posture by detecting the channel center coordinates of the docking device, the control module adjusts the posture of the underwater vehicle, the processor locks the effective visual guidance target, and the control module controls the underwater vehicle to completely enter the docking device.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] (1) The present invention uses a deep learning target detection algorithm to perform visual guidance detection, thereby improving the success rate of autonomous underwater docking and recovery of the vehicle, enabling it to perform underwater operations efficiently and autonomously, and promoting the intelligent development of underwater unmanned equipment;

[0031] (2) The present invention uses a deep learning method, which has a better detection effect on underwater guidance light sources compared with traditional guidance detection methods, thereby improving the success rate of underwater autonomous docking and recovery of the vehicle.

[0032] (3) The present invention improves the deep learning model to address the problem of target incompleteness caused by the limited viewing angle of underwater cameras. The input data is randomly cropped and an attention mechanism is introduced to enable the network to extract discriminative information of the target and alleviate the problem of missed detection of incomplete targets.

[0033] (4) The present invention is portable and can be installed on an underwater navigation platform to efficiently complete autonomous docking and recovery;

[0034] (5) The present invention has real-time characteristics, meeting the requirements for real-time detection in underwater navigation vehicle operations and the requirements for rapid detection of guidance targets during autonomous docking and recovery;

[0035] (6) The present invention can adapt to different guidance types and more diverse underwater environments by using different guidance data sets for training, and has greater engineering applicability; BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the composition of the device of the present invention.

[0037] Figure 2 Schematic diagram of the real-time detection process of the present invention. DETAILED DESCRIPTION

[0038] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0039] Embodiment 1:

[0040] A real-time underwater visual guidance detection method based on deep learning, the detection method comprising the following steps:

[0041] 1) Obtain underwater guidance light source images, including multiple simulations of underwater vehicle autonomous docking and recovery processes, taking underwater guidance light sources (4) and docking device images at different angles as positive samples, and marking the visual guidance target area; collect underwater images without guidance targets as negative samples. Positive samples and negative samples form an underwater image dataset, referred to as the dataset; organize and annotate the underwater image dataset, set the i-th image x i Contains n targets to be detected o i，n , each target to be detected has a label {b i,n ,c i,n}, b i,n is the candidate box coordinate of the target to be detected, where b i，n ={x1,x2,y1,y2,},c i,n is the category of the target to be detected.

[0042] Among them, the visual guidance target categories include guiding light sources and docking devices.

[0043] The autonomous docking and recovery process of the underwater vehicle includes: approaching the docking device, entering the recovery cage and clamping. The guiding light source is 4 green LED light sources.

[0044] 2) Divide the data set into training data set D train , validation data set D val And the test dataset D test .

[0045] 3) Use the training data set to train the YOLOv5s detection model. After each iteration of the entire training set, the model is verified using the validation data set. When the detection accuracy reaches the accuracy requirement, the training is completed and saved, and the test data set is used for test analysis.

[0046] 4) Calibrate the underwater camera packaged into the head of the underwater vehicle to obtain the camera's internal parameter matrix and distortion coefficient.

[0047] 5) Take underwater images during the autonomous docking and recovery process of underwater vehicles j .

[0048] 6) Process the underwater image and input it into the pre-trained YOLOv5s to detect the guidance light source and docking device, and output the visual guidance target o j，n The candidate box coordinate information b j,n and guidance category information c j,n .

[0049] 7) To effectively guide the AUV, remove the impact of incomplete targets on detection. When the coordinates of multiple targets are obtained, the target with the largest area in the candidate box is selected as the guidance target. j .

[0050] o j = argmax n∈N area(P j，n ) (1)

[0051] Among them, P j，n For image x j The area of ​​the target candidate box detected in the image is j The number of detected target candidate boxes.

[0052] 8) Further, according to the candidate frame coordinate information b j Calculate the center coordinates of the guidance target (x o ,y o ) as the docking device location information.

[0053]

[0054]

[0055] 8) The coordinate information of the visual guidance target (x o ,y o) is used to solve the position, and the distortion parameters of the underwater camera are used to obtain the center coordinates of the visual guidance target after eliminating the distortion. Then, the center coordinates of the visual guidance target are converted from the image coordinate system to the camera coordinate system through coordinate transformation to obtain the relative position of the underwater vehicle and the docking device.

[0056] 9) After obtaining the relative position of the underwater vehicle and the docking device, the heading of the vehicle can be adjusted and the positioning result can be displayed.

[0057] 10) When the underwater vehicle is far away from the docking device, the effective visual guidance target is the guidance light source. The underwater vehicle sails to the same depth as the docking device and performs fixed-depth movement. The center position coordinates of the guidance light source obtained by detection are used to provide feedback and guidance to the underwater vehicle, so that the underwater vehicle gradually approaches the docking device;

[0058] 11) After the underwater vehicle head enters the docking device, the guidance light source becomes invalid, and the effective visual guidance target is the docking device. The relative position is calculated by detecting the channel center coordinates of the docking device, and the underwater vehicle is adjusted and locked to the target, and continues to navigate under visual guidance until it completely enters the docking device.

[0059] 12) Repeat the above steps and continue to analyze the new underwater images collected by the underwater camera until the underwater vehicle is docked with the docking device. Figure 2 shown.

[0060] Embodiment 2:

[0061] A real-time underwater visual guidance detection device based on deep learning, such as Figure 1 Said device comprises:

[0062] An image acquisition module, used for acquiring underwater visual guidance images, connected to the processor, and transmitting the underwater visual guidance images to the processor;

[0063] The processor is connected to the image acquisition module, the display module and the expansion interface respectively. The detection method described in Example 1 is used for processing. The processor receives the underwater visual guidance image and detects and locates the visual guidance in the underwater image according to the depth detection model. Finally, the underwater visual guidance image detection result is transmitted to the display module for display;

[0064] A power supply module, connected to the image acquisition module, the processor and the display module respectively, for providing a working voltage during operation;

[0065] The expansion interface is connected to the processor and is used to connect to other expansion modules for data interaction and more expansion functions.

[0066] The docking device is suspended and fixed on the support ship or the surface platform, about 3 meters above the water surface. The docking device mainly includes a recovery cage and a clamping device, and is equipped with a light source guide device, which is a guide light source of 4 green LED light sources.

[0067] Wherein, the image acquisition module includes a monocular industrial camera and a glass waterproof tooling;

[0068] The processor may be any type of airborne board, which implements the underwater visual guidance real-time detection method by running executable instructions.

[0069] Wherein, the power module is a built-in battery or an external power interface.

[0070] Wherein, the processor is arranged in the watertight compartment.

[0071] Embodiment 3:

[0072] An underwater visual guidance real-time detection device based on deep learning, comprising:

[0073] An image acquisition module, used to acquire underwater visual guidance images and send them to a processor;

[0074] The processor uses the existing underwater images as a data set to train the YOLOv5s detection model until the detection accuracy reaches the preset accuracy requirement, completing the training; during the autonomous docking and recovery process of the underwater vehicle, the captured underwater images are input into the pre-trained YOLOv5s detection model to obtain the category and corresponding coordinates of the visual guidance target; according to the coordinates of the visual guidance target, the internal parameters of the underwater camera are used to eliminate the distortion and convert it to the underwater camera coordinate system to obtain the relative position of the underwater vehicle and the visual guidance target.

[0075] Preferably, the detection device further comprises a control module, and the control module is used to call the processor and obtain the relative position, and control the underwater vehicle to adjust its posture until the underwater vehicle reaches a predetermined position.

[0076] Preferably, the categories of the visual guidance targets include guiding light sources and docking devices;

[0077] Before the underwater vehicle head enters the docking device, the processor uses the guidance light source as an effective visual guidance target;

[0078] After the head of the underwater vehicle enters the docking device, the processor uses the docking device as an effective visual guidance target.

[0079] Preferably, when the guiding light source is used as an effective visual guiding target, the control module controls the underwater vehicle to navigate to the same depth as the docking device and perform fixed-depth movement;

[0080] When the docking device is used as an effective visual guidance target, the processor calculates the relative posture by detecting the channel center coordinates of the docking device, the control module adjusts the posture of the underwater vehicle, the processor locks the effective visual guidance target, and the control module controls the underwater vehicle to completely enter the docking device.

[0081] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

[0082] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A real-time underwater visual guidance detection method based on deep learning, characterized in that: include: S1. Obtain underwater images as a data set; use the data set to train the YOLOv5s detection model until the detection accuracy reaches the preset accuracy requirement, completing the training; Calibrate the underwater camera to obtain the internal parameters of the underwater camera; S2. During the autonomous docking and recovery process of the underwater vehicle, the underwater camera captures underwater images; Input the captured underwater images into the trained YOLOv5s detection model to obtain the category and corresponding coordinates of the visual guidance target; According to the coordinates of the visual guidance target, the internal parameters of the underwater camera are used to eliminate distortion and transform to the underwater camera coordinate system to obtain the relative position of the underwater vehicle and the visual guidance target; After obtaining the relative position between the underwater vehicle and the visual guidance target, the underwater vehicle adjusts its position and posture according to the relative position, and repeats S2 until the underwater vehicle reaches a predetermined position; The categories of the visual guidance targets include guiding light sources and docking devices; Before the underwater vehicle head enters the docking device, a guiding light source is used as an effective visual guidance target; After the underwater vehicle head enters the docking device, the docking device is used as an effective visual guidance target; When the guiding light source is used as an effective visual guidance target, the underwater vehicle sails to the same depth as the docking device and performs depth-fixed movement; When the docking device is used as an effective visual guidance target, the relative position is calculated by detecting the channel center coordinates of the docking device. The underwater vehicle adjusts its position and locks the effective visual guidance target, and continues to navigate under visual guidance until the underwater vehicle completely enters the docking device.

2. The real-time detection method according to claim 1, characterized in that: The data set includes positive samples and negative samples; the positive samples are underwater guidance light sources and docking device images taken at different angles during multiple simulations of autonomous docking and recovery of underwater vehicles; the negative samples are underwater images without guidance targets.

3. The real-time detection method according to claim 1, characterized in that: The training of the YOLOv5s detection model is specifically as follows: dividing the data set into a training data set, a verification data set, and a test data set, using the training data set to train the YOLOv5s detection model, each time an iteration of the entire training set is completed, the YOLOv5s detection model is verified using the verification data set, and when the accuracy reaches the preset accuracy requirement, it is saved and tested using the test data set.

4. A real-time underwater visual guidance detection device based on deep learning, characterized in that: include: An image acquisition module, used to acquire underwater visual guidance images and send them to a processor; The processor uses the existing underwater images as a data set to train the YOLOv5s detection model until the detection accuracy reaches the preset accuracy requirement, and the training is completed; during the autonomous docking and recovery process of the underwater vehicle, the underwater images taken are input into the pre-trained YOLOv5s detection model to obtain the category and corresponding coordinates of the visual guidance target; according to the coordinates of the visual guidance target, the internal parameters of the underwater camera are used to eliminate the distortion and convert it to the underwater camera coordinate system to obtain the relative position of the underwater vehicle and the visual guidance target; The detection device further includes a control module, which is used to call the processor and obtain the relative position, and control the underwater vehicle to adjust its posture until the underwater vehicle reaches a predetermined position; The categories of the visual guidance targets include guiding light sources and docking devices; Before the underwater vehicle head enters the docking device, the processor uses the guidance light source as an effective visual guidance target; After the underwater vehicle head enters the docking device, the processor uses the docking device as an effective visual guidance target; When the guiding light source is used as an effective visual guidance target, the control module controls the underwater vehicle to navigate to the same depth as the docking device and perform depth-fixed movement; When the docking device is used as an effective visual guidance target, the processor calculates the relative posture by detecting the channel center coordinates of the docking device, the control module adjusts the posture of the underwater vehicle, the processor locks the effective visual guidance target, and the control module controls the underwater vehicle to completely enter the docking device.

Images