Underwater docking guidance light recognition and positioning method for underwater unmanned vehicle

Abstract

This invention relates to the field of autonomous docking and recovery of underwater autonomous vehicles, and in particular to a method for underwater docking guide light identification and positioning for unmanned underwater vehicles. The method includes the following steps: S1, real-time acquisition of guide light image sequences, classification and filtering using an image classification model to obtain pre-filtered guide light images; S2, guide light signal spot detection and brightness-based noise removal to obtain spot detection information; S3, determination of whether the image contains apparent guide light signals to obtain actual guide light spot detection information; S4, calculation of the relative position between the underwater vehicle and the docking station based on the guide light spot information obtained in step S3. A lightweight neural network model is used for real-time optical image recognition, improving the extraction capability of optical feature information, reducing environmental noise interference, and increasing the success rate of guide light identification, positioning accuracy, and docking success rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of autonomous docking and recovery of underwater autonomous vehicles, and particularly to a method for underwater docking guidance light recognition and positioning for underwater unmanned vehicles. BACKGROUND

[0002] The autonomous docking and recovery of underwater autonomous vehicles generally includes two stages: 1) long-distance homing: returning from a long distance to a specific area close to the docking station, which usually uses acoustic and integrated navigation to guide; 2) close-range docking: when the underwater autonomous vehicle reaches the docking site, from the initial docking position area to the completion of docking, it is a close-range guidance stage or terminal docking stage. In the close-range guidance stage, it is necessary to obtain a high-precision relative pose relationship between the vehicle and the docking station to ensure the docking success rate.

[0003] At present, the close-range guidance stage mainly includes three types of schemes based on acoustic guidance, electromagnetic guidance and visual guidance. Among them, the acoustic signal data update frequency is low, and the stability and accuracy are poor in the process of close-range accurate docking, and the docking success rate is low. With the continuous development of image sensors and computer vision technology in recent years, visual sensors have the advantages of high frame rate, low noise and low delay, and target positioning based on machine vision has unique advantages in underwater terminal autonomous docking and recovery.

[0004] The current underwater docking scheme using visual guidance mainly includes pure visual guidance, monocular single light guidance, monocular multi-light guidance, binocular guidance, etc. Pure visual guidance has high requirements for water quality, relatively short guidance distance and poor environmental adaptability; at the same time, since the docking station does not arrange active guidance light sources, it is not easy for underwater vehicles to capture the docking station target. Monocular single light guidance cannot obtain distance information and can only guide through horizontal and vertical deflection angles, which cannot guarantee that the orientation of the robot entering the dock is consistent with the orientation of the opening of the docking station, and the collision risk is large, and the docking success rate is relatively low. The binocular system itself is relatively complex, and the poor underwater imaging quality leads to low accuracy of parallax map solving, which also has certain application limitations. In comparison, the monocular multi-light guidance scheme has high reliability and accuracy, but its efficient execution depends on accurate guidance light recognition and positioning methods, and in addition, the guidance light sources are basically arranged on the same plane of the opening of the docking station. Due to the limited field of view of the camera, the camera will lose the guidance light source when the distance is very close (about 1-2 meters), so that the last stage cannot be corrected. This makes accurate recognition and positioning of the guidance light source and rapid pose adjustment particularly important for improving the docking success rate.

[0005] In addition, the identification and positioning of the current guide light source mostly uses the classical image binarization method to extract the guide light. These methods rely on the brightness information of the image pixels, and the underwater imaging is special. When the underwater vehicle approaches the water surface, the water surface reflection effect causes the imaging to have a false imaging spot, and there is a serious misidentification problem. Moreover, the existing identification method cannot distinguish between bright spots and guide lights, and the success rate of guide light extraction is low. SUMMARY

[0006] The purpose of the present application is to overcome the above-mentioned defects existing in the prior art, according to the inherent characteristics of the optical image in underwater docking and the computing performance of the AUV-mounted operation platform, a kind of underwater docking guide light identification and positioning method for underwater unmanned vehicle is proposed, light neural network model is used for optical image real-time identification, improve the extraction ability of optical feature information, reduce the interference of environmental noise, improve the success rate of guide light identification, positioning accuracy and docking success rate.

[0007] The technical scheme of the present application is: a kind of underwater docking guide light identification and positioning method for underwater unmanned vehicle, comprising the following steps:

[0008] S1, real-time acquisition guide light image sequence, utilizes image classification model to carry out classification screening, obtains pre-screened guide light image;

[0009] S2, guide light signal spot detection and noise removal based on brightness, obtain spot detection information;

[0010] S3, determine whether the image contains apparent guide light signal:

[0011] If it exists, the real guide light detection is carried out using guide light detection model, and then identification and numbering are carried out;

[0012] Otherwise, directly identify and number the guide light to obtain the real guide light spot detection information;

[0013] S4, according to the guide light spot information obtained in step S3, the relative position of underwater vehicle and dock station is calculated.

[0014] In the present application, step S1 includes the following specific steps:

[0015] S1.1, adopts double thread real-time acquisition data:

[0016] Double thread includes thread 0 and thread 1, in thread 0, build global empty queue Q, start camera to carry out real-time data acquisition, and the image data collected is temporarily stored in the queue;Through thread 1, read the latest one frame image I(w,h,3) from the queue obtained from thread 0 in real time;

[0017] S1.2, input the image I(w, h, 3) into the lightweight image classification model, perform classification screening, determine whether the current image I(w, h, 3) contains a pilot light signal, if there is a pilot light signal, keep it, otherwise, the camera does not capture the pilot light signal, and a new image I(w, h, 3) is obtained.

[0018] In step S1.2, the construction and training method of the lightweight image classification model is as follows:

[0019] In step S1.2.1, the network model is as follows:

[0020] The lightweight deep neural network MobileNetV2 is used to complete the binary classification task of underwater pilot light images;

[0021] In step S1.2.2, the loss function is designed:

[0022] Two classes of true and false lights are distinguished, and the loss function of binary classification is:

[0023] L(θ)=-ylog(h θ (x))-(1-y)log(1-h θ (x))

[0024]

[0025] Where, θ is the parameter value in the optimization process, x is the original output of the network prediction under the parameter; h θ (x) represents the probability of x being converted to a positive class by the sigmoid function; y is the true class label, label 1 is true light, and label 0 is false light; e is the exponential function;

[0026] In step S1.2.3, the pilot light and false pilot light image dataset is constructed.

[0027] In step S1.2.4, the image dataset obtained in step S1.2.3 is used to train the lightweight image classification model, and the lightweight image classification model is obtained to predict whether the image contains a pilot light signal.

[0028] Step S1.2.3 includes the following specific steps:

[0029] In step S1.2.3.1, the pre-experimental data is sorted and analyzed, and the image sequence recognized as a pilot light and the true light and the image sequence recognized as a pilot light but not a true light are found out.

[0030] S1.2.3.2, two category folders are established: true lamp category and false lamp category, and the two categories of image sequences identified through step S1.2.3.1 are respectively put into the corresponding folders, the unified image size is 960*540, and the guiding lamp and misrecognized guiding lamp image dataset is obtained;

[0031] S1.2.3.3, the training set and the verification set are divided according to 0.9:0.1 for model training.

[0032] Step S2 includes the following specific steps:

[0033] S2.1, image segmentation;

[0034] S2.2, spot detection:

[0035] The spot detection based on white information in the opencv library is adopted to obtain the coordinate information Key points of the center of the guiding lamp in the image;

[0036] S2.3, noise elimination:

[0037] The spots lower than the second threshold value Threshold are eliminated, and the spots greater than or equal to the second threshold value are retained, and the denoised spot coordinate information Key points_f is obtained.

[0038] Step S2.1 includes the following specific steps:

[0039] S2.1.1, the RGB distance d of each pixel point of the image is obtained,

[0040]

[0041] Wherein, d is the RGB distance of a pixel point, R, G and B are respectively the pixel value of the red, green and blue channels of the pixel point;

[0042] S2.1.2, the first threshold value (α·d max ) is set to realize the segmentation of the image, and the image I'(w, h) is obtained, and the segmentation mode of the first threshold value is as shown in the following formula,

[0043]

[0044] Wherein, α is the weight, d max is the maximum RGB distance.

[0045] Step S3 includes the following specific steps:

[0046] S3.1, the number of spots in the image detected through step S2 is less than four, which indicates that there is no apparent guiding lamp interference signal in the image, the center of the spot is numbered, and the labeled spot coordinate information Key points_d is obtained.

[0047] S3.2, if the number of spots in the image detected by step S2 exceeds four, it indicates that the image contains apparent beacon interference signals, a beacon detection model is used for real beacon detection, real beacon spot Keypoints_r is obtained, the center of the spot is numbered to obtain the labeled spot coordinate information Keypoints_d;

[0048] S3.3, the length of Keypoints_d is calculated to obtain the number of spots light_num.

[0049] The construction method of the beacon detection model in step S3.2 is as follows:

[0050] S3.2.1, design a lightweight detection network:

[0051] ShuffleNetV2 network is used as the feature extraction backbone, SPP is used to fuse feature maps of different scales, and a single lightweight detection head is used to obtain the detection result of the beacon;

[0052] S3.2.2, design the loss function:

[0053] Set the bounding box loss L IoU : the overlap area IoU of the bounding box A and the target box B

[0054]

[0055] L IoU = 1-IoU;

[0056] Set the target class classification loss L clc :

[0057] L clc = (1-y)*logp-y*log(1-p)

[0058] Wherein, y represents the class, and p represents the class probability;

[0059] Set the foreground-background classification loss L obj :

[0060] L obj = (1-y)*logp-y*log(1-p)

[0061] Wherein, y represents the class, and p represents the class probability;

[0062] Further, the loss function of the constructed detection model is determined as:

[0063] L = λ1·L iou + λ2·L obj+L clc

[0064] wherein λ1 and λ2 represent weights;

[0065] S3.2.3, constructing a guide light reflection imaging dataset:

[0066] The pre-experimental data is analyzed, and a guide light reflection imaging image sequence is selected;

[0067] Using Labelimg software, all guide light reflection imaging image sequences are labeled, and the real guide light is framed out, and the apparent imaging of the guide light is ignored.

[0068] The guide light reflection imaging dataset containing the true value label is obtained;

[0069] S3.2.4, normalization training parameter setting:

[0070] The values of the learning rate η, the weights λ1 and λ2 of the adversarial loss function and the perception loss function, and the batch size are set.

[0071] S3.2.5, based on the guide light reflection imaging dataset, the guide light detection model is trained to obtain a trained model, and the guide light real imaging in the image is detected through the guide light detection model, and the interference of the apparent imaging of the guide light caused by the water surface reflection is removed.

[0072] In step S4, the relative position between the underwater vehicle and the dock station is calculated according to the number of spots information light_num obtained in step S3, and the relative position parameters include the relative angle and the relative distance between the underwater vehicle and the dock station.

[0073] If there is one guide light in the image, the relative angle between the underwater vehicle and the guide light is calculated to guide the underwater vehicle to adjust the pose in the direction of the guide light and to proceed to dock.

[0074] If there are two guide lights in the image, the relative angle between the underwater vehicle and the center points of the two guide lights is calculated to guide the underwater vehicle to adjust the pose in the direction of the two light centers and to proceed to dock.

[0075] If there are three or four guide lights in the image, the relative angle and distance between the underwater vehicle and the center points of the guide lights are calculated to guide the underwater vehicle to adjust the pose in the direction of the center and to proceed to dock.

[0076] The calculation method of the relative distance between the underwater vehicle and the dock station is:

[0077] Let the width between the two guide lights be W, the distance between the camera and the guide light plane be D, the pixel width occupied by the distance between the two guide lights be P, and the camera focal length be focal, then the following relationship exists:

[0078]

[0079] The camera focal length focal is obtained by using the above formula;

[0080] Let the image center be (Cx, Cy) and the guide light spot positioning center be O, the pixel distance pixelwidth between O and (Cx, Cy) is directly obtained, as well as the horizontal pixel difference pixelwidthx and the vertical pixel difference pixelwidthy, and the conversion from the pixel distance to the actual distance is realized by the following formula:

[0081]

[0082] where KNOWN_HEIGHT is the actual distance between the two guide lights, and centerline is the pixel distance between the two guide lights, and according to the above formula, the actual distance realwidth between the guide light spot positioning center O and the image center, the horizontal actual distance realwidthx, and the vertical actual distance realwidthy are calculated.

[0083] The camera focal length focal and the actual distance realwidth between the guide light spot positioning center O and the image center obtained by the above calculation are brought into the following formula to calculate the vertical distance D between the camera and the guide light plane:

[0084]

[0085] The actual spatial distance dis from the docking station to the camera, i.e., the relative distance between the docking station and the unmanned vehicle, is obtained by the following formula:

[0086]

[0087] The relative angle between the underwater vehicle and the docking station includes the horizontal and vertical angles between the optical axis of the underwater vehicle camera and the positioning center point, and the calculation method is as follows:

[0088] Let the horizontal angle between the optical axis of the underwater vehicle camera and the positioning center point be angle_x, then

[0089]

[0090] where pixelwidthx is the horizontal coordinate difference between the image center (Cx, Cy) and the guide light positioning center O, and PI is a constant, so:

[0091] angle x = arctan2 (pixelwidthx / focal) * 180 / PI

[0092] Let the vertical angle between the optical axis of the underwater vehicle camera and the positioning center point be angle x, then

[0093] angle y = arctan2 (pixelwidthy / focal) * 180 / PI

[0094] Wherein, pixelwidthy is the difference value of the longitudinal coordinates between the image center (Cx, Cy) and the positioning center O of the guide light, and PI is a constant.

[0095] The beneficial effects of the present application are:

[0096] (1) The monocular four-light visual guidance scheme is proposed in the present application, which can calculate distance and angle information at the same time, can ensure that the orientation of the underwater autonomous vehicle when entering the dock and the opening orientation of the dock station are always the same, reduces the risk of collision, and improves the docking success rate;

[0097] (2) The lightweight neural network is used for filtering and identifying underwater optical images in the present application, compared with the existing image segmentation method based on pixel threshold, the interference of environmental light, background objects and water surface reflection pseudo imaging in the process of guide light identification and positioning can be effectively eliminated, and the precision and robustness of underwater guide light identification and positioning are improved;

[0098] (3) The real-time classification and identification based on lightweight neural network can meet the real-time operation performance of the embedded GPU carried by the AUV, and improve the autonomy, intelligence and data driving ability of the AUV in complex marine environment. BRIEF DESCRIPTION OF DRAWINGS

[0099] Figure 1 is a structural schematic view of the underwater dock and the guide light setting;

[0100] Figure 2 is a flowchart of the method described in the present application;

[0101] Figure 3 is a flowchart of the construction and training of the lightweight image classification model;

[0102] Figure 4 is an example diagram of the guide light and misidentified guide light image data set constructed in the present embodiment;

[0103] Figure 5 is a spot number schematic diagram when there is no apparent guide light interference signal in the image described in the present embodiment;

[0104] Figure 6is a spot number schematic diagram of the apparent guide lamp interference signal in the figure described in the embodiment;

[0105] Figure 7 is an example diagram of the guide lamp reflection imaging data set constructed by the guide lamp detection model in the embodiment. DETAILED DESCRIPTION

[0106] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0107] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in a variety of ways other than those described herein, and skilled in the art can make similar extensions without departing from the spirit of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0108] The present application proposes an underwater docking guide lamp recognition and positioning method for underwater unmanned vehicles, as shown in Figure 1 The docking station guide lamp in the present application adopts a monocular four-light guide scheme, and the method described in the present application includes the following steps, as shown in Figure 2

[0109] First, the image sequence of the guide lamp is collected in real time, and the image classification model is used to classify and screen the image sequence to obtain the pre-screened guide lamp image.

[0110] The present embodiment takes a single image input as an example to introduce the method in detail.

[0111] First, real-time data acquisition is performed by using double threads. The double threads in the present embodiment can include thread 0 and thread 1. In thread 0, a global empty queue Q is constructed for temporarily storing the collected image data, the camera is started for real-time data acquisition, and the data is temporarily stored in the queue. If the queue reaches the maximum length, the elements at the head are deleted. In thread 0, the latest image data is maintained in real time.

[0112] Thread 1 is the main thread, through which the latest image I(w, h, 3) is read from the queue in thread 0 in real time.

[0113] Secondly, the image I(w, h, 3) is input to the solidified lightweight image classification model for classification and screening to determine whether the current image I(w, h, 3) contains a guide lamp signal. If there is a guide lamp signal, it is retained, otherwise, the camera does not capture the guide lamp signal, a new image I(w, h, 3) is acquired, and the model is used again for classification and screening and determination until the guide lamp signal is contained.

[0114] ​Specifically, the construction and training process of the lightweight image classification model is as follows, as shown in the following. Figure 3

[0115] (1) Design the network model.

[0116] MobileNet was proposed by Google team in 2017, focusing on lightweight CNN network in mobile terminal or embedded device. MobileNet reduces the parameter quantity and calculation quantity by using Depthwise Convolution and Pointwise Convolution to replace the traditional convolution layer, and MobileNetsV2 improves the effect of the model by adding Linear Bottleneck and Inverted Residuals two mechanisms. The embodiment adopts lightweight deep neural network-MobileNetV2 to realize the binary classification task of underwater guide light image, and the model structure is shown in Table 1. Among them, t represents the expansion factor, c represents the depth of the output feature matrix, n represents the number of bottleneck repeats, bottleneck refers to the inverted residual structure, and n=2 represents that the inverted residual structure is repeated twice. S represents the step distance, which represents the step distance of the first layer of each block. The step distance of the first layer is equal to 1. For example, when n=2, the bottleneck is repeated twice, the step distance s of the first layer is 2, and the step distance s of the second layer is 1.

[0117] Table 1 Model structure of lightweight deep neural network-MobileNetV2

[0118]

[0119]

[0120] (2) Design the loss function.

[0121] The loss function of binary classification is:

[0122] L(θ)=-ylog(h θ (x))-(1-y)log(1-h θ (x))

[0123]

[0124] Where, θ is the parameter value in the optimization process, x is the original output of the network prediction under the parameter; h θ (x) represents the probability of x being converted by the sigmoid function to predict as a positive class; y is the real class label, in which label 1 represents a true light, and label 0 represents a false light; e is the exponential function.​

[0125] (3) Construct the guide light and misrecognized guide light image dataset.

[0126] The previous experimental data is sorted and analyzed to find the image sequence that is recognized as a guide light and is a real light, and the image sequence that is recognized as a guide light but is not a real light. Then, two category folders are established: a real light category and a fake light category, with folder names of real and fake, respectively. The above two types of image sequences are placed in the two folders, and the image size is uniformly set to 960x540, to obtain the guide light and misrecognized guide light image dataset. Finally, the training set and the validation set are divided according to 0.9:0.1 for model training. As shown in FIG. 6, the image examples of the real light and the fake light are shown. Figure 4

[0127] (4) Training parameter initialization setting.

[0128] With batch_size = 16 and epoch = 100, the image dataset obtained in the above step (3) is used as the basis to train the lightweight image classification model, and the trained model is obtained to predict whether the image contains a guide light signal.

[0129] The second step is guide light signal spot detection and noise removal based on brightness. Specifically, the following steps are included.

[0130] First, image segmentation. In this application, an image segmentation method based on RGB distance is used to perform image segmentation on the image I(w, h, 3) to obtain the image I'(w, h):

[0131] (1) The RGB distance d of each pixel point of the image is obtained, and the calculation formula is as follows,

[0132]

[0133] Where d is the RGB distance of a pixel point, R, G, and B are the pixel values of the red, green, and blue channels of the pixel point, respectively. Through the above formula, the difference between each pixel point and the pure black RGB value is calculated.

[0134] (2) By setting a suitable RGB distance range as the first threshold (α·d max ), the image is segmented to obtain the image I'(w, h). The segmentation method of the first threshold is as follows,

[0135]

[0136] Where a is the weight, d max is the maximum RGB distance. The weight a in this embodiment is 0.86.

[0137] ​Secondly, spot detection.

[0138] The spot detection based on white information in the opencv library is adopted to obtain the center coordinates of the required white spot, that is, the coordinate information Key points of the center of the pilot lamp in the image.

[0139] Finally, noise elimination.

[0140] In order to eliminate the influence of noise around the spot on the positioning of the pilot lamp, a noise elimination method based on brightness is adopted, that is, the spots below the second threshold Threshold are eliminated, and the spots greater than or equal to the second threshold are retained, to obtain the denoised spot coordinate information Key points_f.

[0141] In this embodiment, the second threshold Threshold is 200.

[0142] Thirdly, it is judged whether the image obtained through the second step contains the apparent pilot lamp signal. If the apparent pilot lamp signal exists, the real pilot lamp detection model is used for detecting the real pilot lamp, eliminating the apparent pilot lamp interference signal, and numbering and positioning the real pilot lamp signal. Otherwise, the identification and numbering of the pilot lamp are directly carried out to obtain the real pilot lamp spot detection information. Specifically, the following steps are included.

[0143] Firstly, if the number of spots in the image detected through the second step is less than four, that is, there is no apparent pilot lamp interference signal in the image, the principle of counterclockwise numbering is adopted in this embodiment, and the center of the spot is numbered as shown in Figure 5 , to obtain the labeled spot coordinate information Key points_d.

[0144] If the number of spots in the image detected through the second step is more than four, that is, the image contains the apparent pilot lamp interference signal, the pilot lamp detection model is adopted to detect the real pilot lamp, to obtain the real pilot lamp spot Key points_r, and the principle of counterclockwise numbering is adopted to number the center of the spot, as shown in Figure 6 , to obtain the labeled spot coordinate information Key points_d.

[0145] In this application, the construction of the pilot lamp detection model includes the following specific steps.

[0146] (1) Design a lightweight detection network.

[0147] Firstly, the shufflenetv2 network is adopted as the feature extraction backbone, and through the channel separation technology, the high channel number is maintained without too many dense convolution numbers and grouping numbers, so as to improve the speed and accuracy.

[0148] Then, SPP (Spatial Pyramid Pooling) is used to fuse feature maps of different scales, increase the receptive field of the image, and enrich the representation ability of the feature map to improve the detection accuracy.

[0149] Finally, a single lightweight detection head is used to simplify the processing steps, improve the running speed of the algorithm, and obtain the detection result of the guide light.

[0150] (2) Design the loss function.

[0151] Designing a suitable loss function helps the network learn more information about the target during training, resulting in higher detection accuracy.

[0152] Set the bounding box loss L IoU : Consider the overlapping area IoU of the detection box A and the target box B

[0153]

[0154] L IoU = 1 - IoU

[0155] Set the target class classification loss L clc :

[0156] L clc = (1-y) * log p - y * log (1-p)

[0157] Where y represents the class and p represents the class probability.

[0158] Set the foreground-background classification loss L obj :

[0159] L obj = (1-y) * log p - y * log (1-p)

[0160] Where y represents the class and p represents the class probability.

[0161] Further determine the loss function of the constructed detection model as:

[0162] L = λ1·L iou + λ2·L obj + L clc

[0163] Where λ1 and λ2 represent the weights.

[0164] (3) Construct the guide light reflection imaging dataset.

[0165] First, analyze the pre-experimental data and select the guide light reflection imaging image sequence.

[0166] Secondly, all image sequences are labeled by using Labelimg software, that is, the real guide light is framed, and the apparent imaging of the guide light is ignored.

[0167] Finally, the guide light reflection imaging dataset containing the true value label is obtained. As shown in the following table, the dataset data example is shown. Figure 7 The dataset data example is shown.

[0168] (4) Normalization training parameter setting.

[0169] The values of the learning rate η, the weights λ1 and λ2 of the adversarial loss function and the perception loss function, and the batch size are set. In this embodiment, η = 0.001, λ1 = 8, λ2 = 16, and the batch size is set to 48.

[0170] (5) Based on the guide light reflection imaging dataset, the guide light detection model is trained to obtain a trained model, which is used for the detection of the real imaging of the guide light in the image in the actual working process, and effectively eliminates the interference of the apparent imaging of the guide light caused by the water surface reflection.

[0171] Finally, the length of the Keypoints_d, that is, the number of light spots light_num, is calculated.

[0172] Fourthly, the relative position between the underwater vehicle and the dock station is calculated according to the guide light spot information determined in the third step.

[0173] According to the spot number information light_num obtained in the third step, the relative position between the underwater vehicle and the dock station is calculated, and the relative position parameters include the relative angle and the relative distance between the underwater vehicle and the dock station.

[0174] Firstly, according to the actual situation, it is necessary to determine which specific relative position parameters between the underwater vehicle and the dock station are calculated. In the actual working process, the following situations are usually included.

[0175] The first situation is that there is one guide light in the image.

[0176] At this time, the relative angle between the underwater vehicle and the guide light is calculated to guide the underwater vehicle to adjust the pose in the direction of the guide light, find more guide signals or adjust to the best docking pose, and then proceed to dock.

[0177] The second situation is that there are two guide lights in the image.

[0178] At this time, the relative angle between the underwater vehicle and the center points of the two guide lights is calculated to guide the underwater vehicle to adjust the pose in the direction of the two light centers, find more signals or adjust to the best docking pose, and then proceed to dock.

[0179] Thirdly, if there are three or four guide lights in the image.

[0180] At this time, the relative angle and distance between the underwater vehicle and the center of the guide light are calculated to guide the underwater vehicle to adjust the pose in the direction of the center and to the optimal docking pose, and then to proceed to dock.

[0181] In the above several cases, the specific calculation and implementation process is as follows.

[0182] The relative distance between the underwater vehicle and the dock station is calculated by using the principle of similar triangles, and the calculation process is as follows.

[0183] Let the width between the two guide lights be W, the distance between the camera and the dock light plane be D, the pixel width occupied by the distance between the two guide lights be P, and the camera focal length be focal, then the following relationship exists:

[0184]

[0185] The camera is calibrated using the above principle to obtain the camera focal length focal. In this embodiment, the calculated focal value is 931.62.

[0186] Let the image center be (Cx, Cy) and the guide light spot positioning center be O, then the pixel distance between O and (Cx, Cy) can be directly obtained, as well as the horizontal pixel difference pixelwidthx and the vertical pixel difference pixelwidthy.

[0187] The conversion from pixel distance to actual distance is realized by the following formula:

[0188]

[0189] where KNOWN_HEIGHT is the actual distance between the two guide lights, and in this embodiment, KNOWN_HEIGHT is set to 1.3 m. centerline is the pixel distance between the two guide lights, and realwidth is the actual distance between the guide light spot positioning center O and the image center.

[0190] According to the above formula, the actual distance realwidth between the guide light spot positioning center O and the image center, the horizontal actual distance realwidthx, and the vertical actual distance realwidthy can be calculated.

[0191] Then, the actual distance realwidth between the camera focal length focal, the guide light spot positioning center O and the image center is brought into the following formula by using the triangle similarity principle, and the vertical distance D of the camera to the guide light plane is calculated:

[0192]

[0193] Finally, the actual space distance dis of the docking station to the camera can be obtained by using the Pythagorean theorem, and the calculation formula is as follows:

[0194]

[0195] Since the camera is arranged on the underwater vehicle, the dis obtained by the above formula is the relative distance between the docking station and the unmanned vehicle.

[0196] The relative angle between the underwater vehicle and the docking station is the horizontal and vertical included angle of the optical axis of the underwater vehicle camera and the positioning center point, and the calculation process is as follows.

[0197] Let the horizontal included angle of the optical axis of the underwater vehicle camera and the positioning center point be angle_x, then

[0198]

[0199] Where focal is the camera focal length, pixelwidthx is the horizontal coordinate difference between the image center (Cx, Cy) and the guide light positioning center O, PI is a constant, so we can get:

[0200] anglex=arctan2(pixelwidthx / focal)*180 / PI)

[0201] Similarly, the vertical included angle of the optical axis of the underwater vehicle camera and the positioning center point is

[0202] angley=arctan2(pixelwidthy / focal)*180 / PI

[0203] Where focal is the camera focal length, pixelwidthy is the vertical coordinate difference between the image center (Cx, Cy) and the guide light positioning center O, and PI is a constant. In this embodiment, the value of PI is 3.1415926.

[0204] The above describes in detail the underwater docking guidance lamp identification and positioning method for underwater unmanned vehicles provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications of the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An underwater docking guidance light identification and positioning method for underwater unmanned vehicles, characterized in that, The method comprises the following steps: S1, real-time acquisition of a guide light image sequence, classification and screening of the guide light image sequence by using an image classification model to obtain a pre-screened guide light image; S2, guide light signal spot detection and noise elimination based on brightness to obtain spot detection information; S3, determining whether the image contains an apparent guide light signal: If so, detecting the real guide light by using a guide light detection model, and then identifying and numbering the guide light; Otherwise, directly identifying and numbering the guide light to obtain real guide light spot detection information; S4, calculating the relative position of the underwater vehicle and the dock station according to the guide light spot information obtained in step S3; The construction and training method of the image classification model is as follows: S1.2.1, using a network model: A lightweight deep neural network MobileNetV2 is used to complete the binary classification task of the underwater guide light image; S1.2.2, designing a loss function: Two categories of true and false lights are distinguished, and the loss function of binary classification is: ， ， where, is the parameter value in the optimization process, x is the predicted raw output of the network for this parameter; is the predicted probability of being a positive class; x is converted to the probability of being a positive class by the sigmoid function; y is the true class label, with label 1 for true light and label 0 for false light; e is the exponential function; S1.2.3, constructing a guide light and misidentified guide light image dataset; S1.2.4, initializing the training parameters, based on the image dataset obtained in step S1.2.3, training the lightweight image classification model to obtain a lightweight image classification model for predicting whether an image contains a guide light signal; The construction method of the guide light detection model is as follows: S3.2.1, designing a lightweight detection network: A shufflenetv2 network is used as the feature extraction backbone, SPP is used to fuse feature maps of different scales, and a single lightweight detection head is used to obtain the detection result of the guide light; S3.2.2, designing a loss function: Setting a detection box loss : an overlap area IoU of the detection box A and the target box B ， ； Setting target class classification loss : ； wherein, y represents a class, p represents a class probability; Set foreground classification loss : ， wherein, y represents a class, p represents a class probability; The loss function of the constructed detection model is: ， wherein and denotes a weight value; S3.2.3, constructing a guide light reflection imaging dataset: Analyzing the experimental data, selecting a guide light reflection imaging image sequence; Using Labelimg software, all guide light reflection imaging image sequences are labeled, and the real guide light is framed out, and the apparent imaging of the guide light is ignored; A guide light reflection imaging dataset containing true value labels is obtained; S3.2.4, normalizing the training parameters: a learning rate , weights of the adversarial loss function and the perceptual loss function , and a value of a batch size are set; S3.2.5, based on the guide light reflection imaging dataset, the guide light detection model is trained to obtain a trained model, and the guide light real imaging in the image is detected by the guide light detection model to eliminate the interference of the apparent imaging of the guide light caused by water surface reflection.

2. The underwater unmanned vehicle facing underwater docking guidance light identification and localization method according to claim 1, characterized in that, Step S1 includes the following specific steps: S1.1, using double threads to collect data in real time: The double threads include thread 0 and thread 1, in the thread 0, a global empty queue Q is constructed, a camera is started for real-time data acquisition, and the collected image data is temporarily stored in the queue; through the thread 1, the latest image frame is read from the queue obtained from the thread 0 in real time ; S1.2, input the image to the lightweight image classification model, and perform classification screening to determine whether the current image contains a pilot light signal. If there is a pilot light signal, it is retained; otherwise, the camera does not capture the pilot light signal, and a new image is acquired .

3. The underwater unmanned vehicle facing underwater docking guidance light identification and localization method of claim 1, wherein, Step S1.2.3 includes the following specific steps: S1.2.3.1, organizing and analyzing the experimental data, and finding out the image sequences identified as guide lights and real lights, and the image sequences identified as guide lights but not real lights; S1.2.3.2, establishing two category folders: true light category and false light category, respectively, and placing the two types of image sequences identified in step S1.2.3.1 into the corresponding folders, and uniformly setting the image size to 960x540 to obtain a guide light and misidentified guide light image dataset; S1.2.3.3, training set and validation set are divided according to 0.9:0.1 for training of the model.

4. The underwater unmanned vehicle facing underwater docking guidance light identification and localization method of claim 1, wherein, Step S2 includes the following specific steps: S2.1, image segmentation; S2.2, spot detection: The center of the guide lamp in the image is obtained by using the spot detection based on white information in the opencv library Keypoints ; S2.3, noise elimination: rejecting the spots below the second threshold value Threshold , retaining the spots equal to or greater than the second threshold value, to obtain the denoised spot coordinate information Keypoints_f .

5. The underwater unmanned vehicle facing underwater docking guidance light identification and localization method of claim 4, wherein, Step S2.1 includes the following specific steps: S2.1.1, obtaining the RGB distance of each pixel point of the image d , ， wherein, d is the RGB distance of a pixel point, R , G , B are the pixel values of the red, green and blue channels of the pixel point, respectively. S2.1.2, set a first threshold value (T1) ), to obtain an image , the first threshold value is segmented as follows ， wherein a is a weight, is the maximum RGB distance.

6. The underwater unmanned vehicle facing underwater docking guidance light identification and localization method of claim 1, wherein, Step S3 includes the following specific steps: S3.1, the number of the spots in the image detected by step S2 is less than four, which means that there is no apparent pilot light interference signal in the image, the centers of the spots are numbered to obtain the labeled spot coordinate information Keypoints_d ; S3.2, the number of spots in the image detected by step S2 exceeds four, indicating that the image contains apparent beacon interference signals, using the beacon detection model to detect the real beacon, and obtaining the real beacon spot Keypoints_ r , the center of the spot is numbered to obtain the labeled spot coordinate information Keypoints_d ; S3.3, compute Keypoints_d Length, get number of spots light_num .

7. The underwater docking guide light recognition and positioning method for underwater unmanned vehicles according to claim 6, characterized in that, In step S4, based on the spot count information obtained in step S3... light_num The relative position between the underwater vehicle and the dock station is calculated. The relative position parameters include the relative angle and relative distance between the underwater vehicle and the dock station. If there is one guide light in the image, the relative angle between the underwater vehicle and the guide light is calculated to guide the underwater vehicle to adjust the pose in the direction of the guide light and move forward into the dock; If there are two guide lights in the image, the relative angles between the underwater vehicle and the center points of the two guide lights are calculated to guide the underwater vehicle to adjust the pose in the direction of the center points of the two guide lights and move forward into the dock; If there are three or four guide lights in the image, the relative angles and distances between the underwater vehicle and the center points of the guide lights are calculated to guide the underwater vehicle to adjust the pose in the direction of the center points and move forward into the dock.

8. The underwater docking guide light recognition and positioning method for underwater unmanned vehicles according to claim 7, characterized in that, The relative distance between the underwater vehicle and the dock station is calculated as follows: Let the width between two guide lights be W , the distance between the camera and the plane of the guide lights be D , the pixel width occupied by the distance between two guide lights be P , and the focal length of the camera be focal , then the following relationship exists: ， Using the above formula, the camera focal length is obtained focal ; Let the center of the image be ( Cx , Cy The center of the guide light spot is O , directly obtain O Point and ( Cx , Cy Pixel distance between points pixelwidth and horizontal pixel difference pixelwidthx Vertical pixel difference pixelwidthy The following formula is used to convert pixel distance to actual distance: ， wherein, KNOWN_HEIGHT is the actual distance between the two guide lights, centerline is the pixel distance between the two guide lights, according to the above formula, the guide light spot positioning center O is the actual distance between the guide light spot positioning center and the image center realwidth , the horizontal actual distance realwidthx , the vertical actual distance realwidthy; The actual spatial distance from the docking station to the camera is obtained by the following equation dis i.e., the relative distance between the docking station and the unmanned vehicle: ， The relative angle between the underwater vehicle and the dock station includes the horizontal and vertical angles between the optical axis of the underwater vehicle camera and the positioning center point, and is calculated as follows: Let the horizontal angle between the optical axis of the camera of the underwater vehicle and the positioning center point be angle_x then ， wherein pixelwidthx is the horizontal coordinate difference between the image center (x Cx , Cy ) and the pilot light positioning center (x O , PI is a constant, thus: )， Let the vertical angle between the optical axis of the underwater vehicle camera and the positioning center point be angle_x then ， wherein pixelwidthy is the vertical coordinate difference between the image center (x, y) Cx , Cy ) and the guidance light positioning center (x, y) O , PI is a constant.

Images