A priori information based underwater visual positioning method with illumination robustness
By obtaining the brightness peak of underwater images and prior information to determine the presence of targets, combined with adaptive binarization and morphological operations, the robustness problem of underwater visual positioning in uncontrollable lighting environments is solved, and reliable detection and accurate positioning of UUVs are achieved.
Patent Information
- Application Number
- CN202310977225.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-03
- Publication Date
- 2025-10-24
- Estimated Expiration
- 2043-08-03
AI Technical Summary
Underwater visual positioning technology lacks robustness in environments with uncontrollable lighting conditions, making it difficult to effectively apply in underwater recovery scenarios on lakes or at sea.
By obtaining the brightness peak of underwater images, using prior information to determine the existence of targets, performing adaptive binarization processing and morphological operations, screening effective target contours, and determining pixel coordinates through priority sorting, the recovery of unmanned underwater vehicles is achieved by combining acoustic and optical positioning methods.
It can achieve reliable detection and accurate positioning of UUV targets under different lighting conditions, which is suitable for practical application scenarios with large lighting changes and improves positioning accuracy and robustness.
Smart Images
Figure CN117152190B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of underwater visual positioning, and more particularly relates to a light-robust underwater visual positioning method based on prior information. BACKGROUND
[0002] In recent years, acoustic positioning and guiding technologies represented by the ultra-short baseline positioning system (USBL) and sonar have been relatively mature and have been widely used in long and medium distance scenarios of underwater recovery. Compared with acoustic positioning technology, visual positioning technology has the advantages of high positioning accuracy, fast data updating, and low hardware requirements, and is thus more suitable for short distance scenarios of underwater recovery. However, underwater visual positioning technology is extremely susceptible to environmental brightness. How to effectively use visual positioning means in lake or sea underwater recovery scenarios where the lighting conditions are uncontrollable has been a hot and difficult research topic. In recent years, many scholars at home and abroad have conducted research and attempts in this regard.
[0003] Zhang Wei et al. studied the monocular visual multi-target tracking problem of UUV recovery, proposed a positioning and guiding method based on an asymmetric light array based on a nonlinear Kalman filter, and still achieved good tracking effect when part of the foreground light source was blocked and the background frame changed greatly. However, the algorithm is based on the premise of ignoring the interference of background light and pseudo light source, and the universality needs to be improved. In addition, Zhang Wei's team improved the Otsu method and solved the feature extraction problem of the guide light array target under the condition of overall low brightness, low contrast, and fuzzy details, but failed to evaluate the segmentation ability under the condition that the target and background brightness are close. Han Zekai et al. proposed an improved detection-tracking algorithm based on neural network and Pnp-P3P algorithm, combined the high generalization advantage of neural network target recognition with the high efficiency advantage of traditional algorithm segmentation, and realized the robust tracking of the guide light source. However, this method needs to train a neural network model, and how to construct a training data set and the high hardware resource overhead are two problems that need to be faced in the deployment process of the method. SUMMARY
[0004] In view of the above defects or improvement needs of the prior art, the application provides a light-robust underwater visual positioning method based on prior information, which aims to solve the technical problem that the current underwater visual positioning method is greatly affected by weather and light conditions.
[0005] To achieve the above purpose, in a first aspect, the application provides a light-robust underwater visual positioning method based on prior information, which comprises the following steps:
[0006] (1) obtaining an underwater image, and converting the underwater image into a grayscale image after filtering and denoising;
[0007] (2) acquiring a brightness peak value of the grayscale image, and determining whether the brightness peak value is greater than an ambient brightness threshold value; if not, returning to step (1); if yes, entering step (3);
[0008] (3) performing binarization processing on the grayscale image to obtain a plurality of target contours, and performing morphological operation on the target contours;
[0009] (4) screening out effective target contours based on the number of pixel points contained in the target contours, if the number of the effective target contours is not less than 1, performing priority sorting on the effective target contours, and determining the centroid of the effective target contour with the highest priority as the pixel coordinates of the underwater target, and ending; otherwise, entering step (5);
[0010] (5) acquiring the coordinates of the brightness peak pixel of the grayscale image, taking the coordinates of the brightness peak pixel as the center to frame an interest region, performing binarization processing on the interest region to obtain a plurality of target contours, and performing morphological operation on the target contours;
[0011] (6) screening out effective target contours based on the number of pixel points contained in the target contours, if the number of the effective target contours is not less than 1, performing priority sorting on the effective target contours, and determining the centroid of the effective target contour with the highest priority as the pixel coordinates of the underwater target, and ending; otherwise, determining the coordinates of the brightness peak pixel as the pixel coordinates of the underwater target.
[0012] Preferably, the ambient brightness threshold value is obtained by the following method: a plurality of frames of underwater environment images are continuously acquired in advance, the average value of the brightness peak values of the plurality of frames of underwater environment images is calculated, and the average value plus a preset offset value obtains the ambient brightness threshold value.
[0013] Preferably, in the method, the effective target contours are screened out based on the number of pixel points contained in the target contours, and specifically, a pixel point number range is preset, if the number of pixel points contained in the target contour is within the pixel point number range, the target contour is an effective target contour.
[0014] Preferably, in the method, the effective target contours are priority sorted, and specifically, the shape and the number of pixel points of the effective target contours are acquired, the closer the shape is to a regular circle and the more the number of pixel points is, the higher the priority is.
[0015] Preferably, in the method, a region of interest is framed with the coordinates of the luminance peak pixel as the center, and a plurality of target contours are obtained by binarizing the region of interest, specifically: a region within a preset distance adjacent to the coordinates of the luminance peak pixel is set as the region of interest; luminances of pixel points in the region of interest are sorted, and an average value of luminances in a front section is calculated, the front section being a preset value, and a threshold value of the region of interest is obtained by adding a preset offset to the average value of luminances, and a plurality of target contours are obtained by binarizing the region of interest based on the threshold value of the region of interest.
[0016] In a second aspect, the present application provides a method for recovering an unmanned underwater vehicle based on acoustic-optical positioning, the method comprising the following steps:
[0017] The relative position of the unmanned underwater vehicle is obtained by using the USBL underwater positioning method, and the unmanned underwater vehicle is guided into an optical positioning range through underwater acoustic communication;
[0018] An underwater image in the optical positioning range is collected, and the coordinate position of the unmanned underwater vehicle underwater is determined by using any one of the methods in the first aspect;
[0019] The unmanned underwater vehicle is recovered based on the coordinate position.
[0020] In a third aspect, the present application provides an electronic device, comprising:
[0021] A memory for storing a program;
[0022] A processor for executing the program stored in the memory, when the program stored in the memory is executed, the processor is used to execute any one of the methods in the first aspect; or the processor is used to execute the method in the second aspect.
[0023] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program runs on the processor, the processor is caused to execute any one of the methods in the first aspect; or the processor is caused to execute the method in the second aspect.
[0024] In a fifth aspect, the present application provides a computer program product, when the computer program product runs on the processor, the processor is caused to execute any one of the methods in the first aspect; or the processor is caused to execute the method in the second aspect.
[0025] Overall, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects:
[0026] The method fully utilizes prior information of the target to enhance the accuracy and robustness of target contour extraction, thereby optimizing the calculation accuracy of the spatial position, and is very suitable for practical application scenarios with large light changes; underwater experiment results show that the underwater visual positioning method can realize reliable detection and accurate positioning of the UUV target in the "early-mid-late" whole period and "cloudy-sunny-rainy" all-weather, and has good engineering application value. BRIEF DESCRIPTION OF DRAWINGS
[0027] Figure 1 is an implementation structure composition schematic diagram of an underwater recovery method of an unmanned underwater vehicle based on acousto-optic positioning provided in an embodiment of the present application;
[0028] Figure 2 is a flowchart of an optical positioning recovery method of an unmanned underwater vehicle in an underwater recovery method of an unmanned underwater vehicle;
[0029] Figure 3 is an experimental result of an underwater unmanned underwater vehicle recovery positioning experiment Figure 1 ; part a is a target gray scale diagram and a gray scale histogram under soft light conditions in the morning; part b is a target gray scale diagram and a gray scale histogram under strong light conditions at noon; part c is a target gray scale diagram and a gray scale histogram under weak light conditions in the evening;
[0030] Figure 4 is an experimental result of an underwater unmanned underwater vehicle recovery positioning experiment Figure 2 ; part a is a target contour obtained after binarization based on different threshold values under soft light conditions in the morning; part b is a target contour obtained after binarization based on different threshold values under strong light conditions at noon; part c is a target contour obtained after binarization based on different threshold values under weak light conditions in the evening;
[0031] Figure 5 is a flowchart of a light robustness underwater visual positioning method based on prior information provided in an embodiment of the present application;
[0032] Figure 6 is a target contour obtained by using the method of the present application; part a is a target contour obtained under soft light conditions in the morning; part b is a target contour obtained under strong light conditions at noon; part c is a target contour obtained under weak light conditions in the evening. DETAILED DESCRIPTION
[0033] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.
[0034] 1. Experimental system design:
[0035] As shown in Figure 1 , it is a UUV (underwater unmanned vehicle) underwater recovery system combined with acoustic and optical positioning and guiding means, and lake test verification is carried out, the system is composed of an experimental ship recovery platform, a recovery device, an acoustic-optical guiding system and a UUV,
[0036] The recovery device can move a certain distance in the horizontal and vertical directions to compensate for the motion offset of the UUV; the acoustic-optical guiding system is composed of a USBL system and a visual positioning system, the USBL system is composed of a base array installed at the bottom of the recovery platform and a beacon installed on the UUV, the recovery platform obtains the relative position of the UUV through the base array, and uses the underwater acoustic communication function of the USBL system to guide the UUV into the capture range of the recovery device. When the UUV is close, the acoustic guiding accuracy and rate are not enough to support the recovery device to capture the UUV, so the visual positioning system is used to estimate the offset of the UUV, the visual positioning system is composed of a marker light source (green LED light) installed on the UUV and an underwater binocular camera installed at the bottom of the recovery platform, when the UUV enters the field of view of the binocular camera, the binocular camera locates it through the marker light source, and the recovery device compensates for the offset of the UUV according to the visual positioning data and completes the capture, the working principle is as shown in Figure 2 .
[0037] 2. Experimental data analysis:
[0038] In the UUV recovery field test, the binocular camera is arranged at the position of 50 cm at the bottom of the test ship, which is close to the water surface and is easily affected by sunlight. Figure 3 Parts a, b and c respectively give the real images obtained by the binocular camera and the corresponding brightness histograms under the conditions of soft light in the morning, strong light at noon and weak light in the evening, the white bright spots in the three images are the marker light of the same UUV, and their sizes and characteristics are obviously different, it can be seen that the change of environmental light has a significant influence on the detection effect of the underwater camera on the target.
[0039] The analysis of the brightness histogram of the three images shows that under the soft light condition in the morning and the weak light condition in the evening, the background brightness is uniform, the target pixel brightness is prominent, and it is easy to distinguish from the background; under the strong light condition at noon, the overall background brightness is high and unevenly distributed, the target pixel brightness is close to the surrounding background brightness, and it is difficult to effectively distinguish from the background.
[0040] The target pixel coordinate acquisition is one of the key steps of the binocular vision positioning, and directly determines the calculation accuracy of the target space position. Figure 3 The a, b and c parts in the formula (1) are binarized based on different threshold values (105, 155, 205) to extract the target contour, and the effect is shown in the a, b and c parts in the formula (2). Figure 4
[0041] The binarization processing with a fixed threshold value cannot be adjusted according to the actual situation of the image, so it cannot simultaneously consider images with different background brightness, and the summary is as follows: (1) the three threshold values can usually effectively extract the target contour for images with low and uniform background brightness, and there is basically no missed detection and false detection; (2) when the gray threshold values 105 and 205 are taken, it is difficult to effectively separate the target and the background with small contrast for images with high and uneven background brightness, and missed detection and false detection may occur; (3) when the gray threshold value 155 is taken, it is difficult to distinguish the target from the virtual image around it, so it is difficult to accurately restore the true shape features of the target.
[0042] 3. The technical scheme embodiment of the application:
[0043] In view of the above problems, the application provides a light-robust underwater visual positioning method based on prior information, which firstly judges whether there is a target in the image according to the brightness feature, and only when it is confirmed that there is a target object in the image, the subsequent processing flow is entered, after the key prior information that the target exists is known, the possible area where the target (marker light source) exists is tried to be located, if the possible area where the target exists can be successfully located, the area is subjected to adaptive binarization processing, otherwise, if not, the highest brightness point can also be selected as the target position according to the prior knowledge that the marker light source has the highest brightness. Figure 5 The specific steps are shown in the formula (3), and include the following steps.
[0044] (1) after starting the binocular camera, 100 frames of images are continuously collected, the average value of the peak brightness of the 100 frames of images is calculated, and a proper offset is added to serve as the environmental brightness threshold value b env .
[0045] (2) the image is collected, and is converted into a gray scale format after filtering and denoising;
[0046] (3) the peak brightness b max of the image is obtained, and it is judged whether it is greater than benv ;
[0047] (4) If b max ≥ b env , it means that the camera has captured the identification light source (known a priori information: the identification light source brightness is much higher than the brightness of the surrounding environment), otherwise jump to step (2);
[0048] (5) According to the set threshold, the image is binarized to obtain a number of contours, and then morphological operations such as erosion and expansion are performed to modify the contours;
[0049] (6) The number of pixel points contained in each contour is obtained, and the contours with too small or too large number of pixel points are excluded, so as to screen out effective contours;
[0050] (7) If the number of effective contours is greater than or equal to 1, the priority of the effective contours is evaluated, and the evaluation method includes: the higher the contour shape is close to a regular circle, the higher the priority, and the more the number of pixel points, the higher the priority; otherwise, jump to step (9);
[0051] (8) The centroid of the contour with the highest priority is selected as the pixel coordinate of the target;
[0052] (9) The coordinates of the brightness peak pixel l max are obtained, and the adjacent 2500 pixels are framed as a region of interest (ROI) with l max as the center point;
[0053] (10) All pixel points in the ROI are sorted in descending order according to the brightness value, and the average brightness value of the top 30% pixel points is calculated. After adding a suitable offset, the ROI threshold value is obtained. Based on the threshold value, the ROI is binarized to obtain a number of contours, and then morphological operations such as erosion and expansion are performed to modify the contours;
[0054] (11) The number of pixel points contained in each contour is obtained, and the contours with too small or too large number of pixel points are excluded, so as to screen out effective contours;
[0055] (12) If the number of effective contours in the ROI is greater than or equal to 1, the priority of the effective contours is evaluated, and the evaluation method is the same as step (7). The centroid of the contour with the highest priority is selected as the pixel coordinate of the target; otherwise, l max is selected as the pixel coordinate of the target. Jump to step (2);
[0056] 4. Analysis of experimental results
[0057] The images collected under different lighting conditions are applied to the algorithm proposed in this paper to extract the contour of the target, and the results are shown in parts a, b and c of Figure 6 , wherein Figure 6 andFigure 4 It can be concluded that, due to the great influence of external light changes on the target features in the image, if direct binarization segmentation is performed with a fixed threshold, it is often difficult to consider different scenes, and false detection, missed detection or contour distortion may occur, thereby being difficult to be directly applied to outdoor scenes with large fluctuations in light conditions, and the algorithm proposed in the present application can effectively detect the target and extract the real contour shape under any light condition, has good robustness to image brightness changes, and is suitable for application in outdoor test scenes.
[0058] 5、Experimental conclusion:
[0059] The method of the present application is aimed at the UUV underwater recovery scene, and a light-robust underwater visual positioning method based on prior information is designed, on the basis of which an image processing algorithm with good robustness to environmental brightness changes is proposed, the prior information of the cooperative target is fully utilized to enhance the accuracy and robustness of target contour extraction, thereby optimizing the calculation accuracy of the spatial position, and the present application is very suitable for practical application scenes with large light changes. The lake test results show that the present application can realize reliable detection and accurate positioning of the UUV target under the conditions of "early-mid-late" whole time period and "cloudy-sunny-rain" all-weather, and has good engineering application value.
[0060] The above content is easily understood by those skilled in the art, and the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for prior information based illumination-robust underwater visual positioning, characterized in that, The method comprises the following steps: (1) obtaining an underwater image, filtering and denoising the underwater image, and converting the underwater image into a grayscale image; (2) obtaining a brightness peak value of the grayscale image, and determining whether the brightness peak value is greater than an environmental brightness threshold value; if not, returning to step (1); if yes, entering step (3); (3) performing binarization processing on the grayscale image to obtain a plurality of target contours, and performing morphological operation on the target contours; (4) screening out effective target contours based on the number of pixel points contained in the target contours; if the number of effective target contours is not less than 1, performing priority sorting on the effective target contours, determining the centroid of the effective target contour with the highest priority as the pixel coordinates of the underwater target, and ending; otherwise, entering step (5); (5) obtaining the coordinates of the brightness peak pixel of the grayscale image, framing an interest region with the coordinates of the brightness peak pixel as the center, performing binarization processing on the interest region to obtain a plurality of target contours, and performing morphological operation on the target contours; framing an interest region with the coordinates of the brightness peak pixel as the center, performing binarization processing on the interest region to obtain a plurality of target contours, specifically: setting a region within a preset distance adjacent to the coordinates of the brightness peak pixel as the interest region; sorting the brightness of the pixel points in the interest region, calculating the average brightness of the front segment, the front segment being a preset value, obtaining an interest region threshold value by adding a preset offset to the average brightness, and performing binarization processing on the interest region based on the interest region threshold value to obtain a plurality of target contours; (6) screening out effective target contours based on the number of pixel points contained in the target contours; if the number of effective target contours is not less than 1, performing priority sorting on the effective target contours, determining the centroid of the effective target contour with the highest priority as the pixel coordinates of the underwater target, and ending; otherwise, determining the coordinates of the brightness peak pixel as the pixel coordinates of the underwater target.
2. The method of claim 1, wherein, The environmental brightness threshold value is obtained by the following method: continuously collecting a plurality of underwater environment images in advance, calculating the average value of the brightness peak values of the plurality of underwater environment images, and obtaining the environmental brightness threshold value by adding a preset offset to the average value.
3. The method of claim 1, wherein, In the method, the effective target contours are screened out based on the number of pixel points contained in the target contours, specifically: a pixel point number range is preset, and if the number of pixel points contained in the target contour is within the pixel point number range, the target contour is an effective target contour.
4. The method of claim 1, wherein, In the method, the effective target contours are priority sorted, specifically: the shape and the number of pixel points of the effective target contour are obtained, and the closer the shape is to a regular circle and the more the number of pixel points is, the higher the priority is.
5. An unmanned underwater vehicle recovery method based on acousto-optic positioning, characterized by, The method comprises the following steps: obtaining the relative position of the unmanned underwater vehicle by using the USBL underwater positioning method, and guiding the unmanned underwater vehicle into the optical positioning range through underwater acoustic communication; collecting an underwater image in the optical positioning range, and determining the coordinate position of the unmanned underwater vehicle in water by using the method of any one of claims 1-4; recovering the unmanned underwater vehicle based on the coordinate position.
6. An electronic device, comprising: It comprises: a memory for storing a program; a processor for executing the program stored in the memory, the processor being configured to perform the method of any one of claims 1-4 when the program stored in the memory is executed; or the processor being configured to perform the method of claim 5.
7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. a computer program for making the processor perform the method of any one of claims 1-4 when the computer program is run on the processor; or the computer program for making the processor perform the method of claim 5.
8. A computer program product, characterised in that, a computer program product for making the processor perform the method of any one of claims 1-4 when the computer program product is run on the processor; or the computer program product for making the processor perform the method of claim 5.