Underwater obstacle segmentation method and system based on unmanned ship side scanning

Abstract

The invention discloses an underwater obstacle segmentation method and system based on unmanned ship side scanning, and relates to the technical field of underwater object detection. The method comprises the following three steps of underwater side-scan XTF file analysis, underwater side-scan waterfall plot denoising and underwater side-scan denoised image segmentation. According to the technical scheme, by means of optimization algorithms such as grid parameter adjustment and the like and PSNR scores, suitable selection parameters during BM3D denoising and graph segmentation based on QPSO + morphological parameter adjustment can be automatically obtained, compared with a traditional image segmentation algorithm, morphological segmentation can be carried out more quickly and more accurately through a threshold value, a clear and complete defect area contour can be obtained, and the defect area denoising accuracy is improved. The obstacle can be rapidly and accurately cut out, accurate positioning of underwater obstacles is achieved, and the requirements of an unmanned ship for seabed target searching and handling work are met.

Description

Technical Field

[0001] This invention relates to the field of underwater object detection technology, specifically to an underwater obstacle segmentation method and system based on unmanned surface vessel side scanning. Background Technology

[0002] With the continuous development of sonar technology, using unmanned vessels equipped with two-dimensional sonar equipment to conduct large-scale underwater scanning has become an important method. Compared with traditional optical-based detection, lasers have more scattering and refraction phenomena in underwater measurement. In contrast, underwater sonar has the advantages of wide range, is not affected by water turbidity, and has a wide coverage area.

[0003] However, existing sonar side-scan schemes generate black defect regions with high similarity, which makes it difficult to segment general images and cannot quickly and accurately cut out obstacles. To address this, we propose an underwater obstacle segmentation method and system based on unmanned surface vessel side-scanning. Summary of the Invention

[0004] The purpose of this invention is to provide an underwater obstacle segmentation method and system based on unmanned surface vessel side scanning, which can quickly and accurately cut out obstacles and achieve precise positioning of underwater obstacles.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an underwater obstacle segmentation method based on unmanned surface vessel side scanning, specifically including the following steps:

[0006] Acquire underwater side-scan XTF files, independently parse the XTF format, and reconstruct a bidirectional waterfall plot from its signal points;

[0007] An adaptive BM3D denoising algorithm was used to denoise the underwater side-scan waterfall image to obtain a denoised image.

[0008] The QPSO algorithm is used to segment the denoised image, extract large-area shadow regions, and obtain the final image segmentation contour.

[0009] Furthermore, the steps for parsing the XTF format are as follows:

[0010] First, read 1024 bytes and extract the first byte to determine if the file is in XTF format. If the first byte is 0x7B, then it is an XTF file.

[0011] Next, check if the number of channels in the header information is greater than 6. If it is greater than 6, continue reading 1024 bytes and then parse out the important header information.

[0012] Furthermore, the image reconstruction step in XTF format is as follows:

[0013] The received data lines of each transmission cycle are arranged vertically in a single line and displayed on the monitor, forming a two-dimensional acoustic map of the seabed topography. The acoustic map plane and the seabed plane are in a point-to-point mapping relationship, and the brightness of the acoustic map includes the features of the seabed.

[0014] Furthermore, the adaptive BM3D denoising algorithm includes the BM3D algorithm and an adaptive parameter tuning algorithm:

[0015] In BM3D, the similarity between image block P and matching block Pmatch is determined by the Euclidean distance between them. All matching blocks Pmatch found with P as the center are sorted, and the Pmatch that is closest to P is selected. This set is then combined with P into a three-dimensional matrix, represented by P3D.

[0016] Next, collaborative filtering is performed. First, a three-dimensional linear transformation is applied to P3D: first a two-dimensional linear transformation, then a one-dimensional linear transformation, followed by hard thresholding in the transform domain, and finally, an inverse transformation is performed to obtain the processed three-dimensional matrix P3D.

[0017]

[0018] Among them, T 2D To represent a two-dimensional linear transformation, this study uses the two-dimensional discrete cosine transform; T 1D represents a one-dimensional linear transformation, using a one-dimensional wavelet transform; Y represents hard-domain filtering. Represents a two-dimensional linear inverse transform; This represents a one-dimensional linear inverse transform.

[0019] Furthermore, the specific method for denoising underwater side-scan waterfall images using the adaptive BM3D denoising algorithm is as follows:

[0020] First, a speckle noise model for side-scan sonar images is established. Then, through power transform and logarithmic transform, the multiplicative non-normally distributed noise is transformed into additive normally distributed noise.

[0021] Secondly, the speckle noise in the side-scan sonar image is first transformed from a Rayleigh distribution to a near-Gaussian distribution, and then a logarithmic transformation is performed on the side-scan sonar image to change the noise from multiplicative to additive.

[0022] Furthermore, the speckle noise model of the side-scan sonar image is specifically described as follows:

[0023] X=CZ (1)

[0024] Where X is the image contaminated by noise, C is the real image, and Z is speckle noise that follows a Rayleigh distribution.

[0025] Furthermore, the 3D matrix P3D is evaluated using the image signal-to-noise ratio (PSNR), and the parameters of the BM3D algorithm are adaptively adjusted based on the score feedback.

[0026] The adaptive parameter tuning algorithm is a grid search method. It adjusts the parameters sequentially according to a set step size. By iterating through all the parameters, it finds the parameter with the highest accuracy on the validation set, which is the final result.

[0027] Furthermore, the specific steps for segmenting the denoised image using the QPSO algorithm are as follows:

[0028] (1) The two-dimensional maximum entropy criterion is adopted as the fitness function of QPSO. Based on QPSO, the segmentation threshold vector (r′, s′) is calculated, and then the target segmentation of the side scan sonar image is performed.

[0029] (2) Due to the obstruction of sound waves by water bodies or lake bottom targets, shadow areas are generated in the side scan sonar images. The pixel gray values ​​of the shadow areas are very small. The gray histogram of set T is statistically analyzed to obtain the first peak value peak1. The positions in set T with values ​​< peak1 are marked as shadow areas.

[0030] (3) Based on the characteristics that the target and shadow appear in pairs and the shadow appears behind the target, the shadow area is finally segmented;

[0031] (4) The target area and shadow area segmentation results of the side-scan sonar image are fused, and the edge contour lines are extracted from the segmentation results using mathematical morphology operations to obtain the final image segmentation contour.

[0032] Furthermore, the two-dimensional optimal segmentation threshold vector is calculated using the quantum particle swarm optimization algorithm based on the two-dimensional maximum entropy theory.

[0033] Based on particle swarm optimization, QPSO uses a Delta potential well as a metric. During the optimization process, it is assumed that the Kth particle is at its optimal position Γ. k =(Γ k1 ,Γ k2 ,...,Γ kD It moves within the Delta potential well centered at 0 (where D is the particle dimension).

[0034] Γ kD The local optimal position of (d∈[1,D]) can be calculated iteratively using the following formula:

[0035]

[0036] In the formula and Γ is any random number in the range [0,1]; best The optimal position among all particles (possessing the best fitness value); Γkd Γ represents the previous local optimal position of the k-th particle. d This represents the local optimal position after the evolution of the k-th particle.

[0037] According to one aspect of the present invention, the present invention provides an underwater obstacle segmentation system based on unmanned surface vessel side scanning, specifically comprising:

[0038] The parsing unit is used to acquire underwater side-scan XTF files, independently parse the XTF format, and reconstruct its signal points into a bidirectional waterfall plot.

[0039] Denoising unit: Used to denoise the underwater side-scan waterfall image using an adaptive BM3D denoising algorithm to obtain a denoised image;

[0040] The image segmentation unit is used to segment the denoised image using the QPSO algorithm, extract large-area shadow regions, and obtain the final image segmentation contour.

[0041] This invention has at least the following beneficial effects:

[0042] The technical solution of this invention utilizes optimization algorithms such as mesh parameter tuning and PSNR scoring to automatically obtain suitable selection parameters for BM3D denoising. Based on QPSO+ morphological parameter tuning, the image segmentation can perform morphological segmentation through thresholding faster and more accurately compared to traditional image segmentation algorithms. This results in clear and complete defect region contours, quickly and accurately cutting out obstacles and achieving precise positioning of underwater obstacles, thus meeting the needs of unmanned vessels for searching and handling seabed targets.

[0043] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the overall implementation of this invention;

[0045] Figure 2 This is a diagram of the XTF file structure in this invention;

[0046] Figure 3 This is a flowchart of XTF parsing in this invention;

[0047] Figure 4 This is the actual waterfall plot generated by XTF in this invention;

[0048] Figure 5 This is a flowchart of the adaptive BM3D denoising algorithm in this invention;

[0049] Figure 6 This is a diagram illustrating the effect of an embodiment of the adaptive BM3D denoising algorithm in this invention.

[0050] Figure 7 This is a flowchart of the QSPO image segmentation process in this invention;

[0051] Figure 8 This is a segmentation effect diagram of the QSPO image embodiment in this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0053] Please see Figure 1-8 This invention provides a technical solution: an underwater obstacle segmentation method based on unmanned surface vessel side scanning, specifically including the following steps:

[0054] S1. Obtain the underwater side-scan XTF file, independently parse the XTF format and reconstruct its signal points into a bidirectional waterfall plot;

[0055] like Figure 1 As shown, the XTF format is an extensible data format with strong scalability and extensibility. It can store various types of information such as sonar, navigation, telemetry, and depth sounding, and is currently a commonly used side-scan sonar data format both domestically and internationally.

[0056] like Figure 2 As shown, the XTF format begins with the XTF header structure, which is a multiple of 1024 bytes in length. This is followed by different types of data packets, including four types: sonar, depth sounding, attitude, and annotation. Each data packet contains NPing data, and each ping contains a header and channel information.

[0057] The specific steps for parsing the XTF format are as follows:

[0058] (1) First, read 1024 bytes and extract the first byte to determine whether the file is in XTF format. If the first byte is 0x7B, then it is an XTF file.

[0059] (2) Then determine whether the number of channels in the header information is greater than 6. If it is greater than 6, continue to read 1024 bytes and then parse out important header information, such as the number of samples and the sampling precision, and store it in variables to ensure the correct reading of each subsequent Ping data.

[0060] like Figure 3As shown, the received data lines of each transmission cycle are arranged vertically in a linear fashion and displayed on the monitor, forming a two-dimensional acoustic map of the seabed topography. The acoustic map plane and the seabed plane are in a point-to-point mapping relationship, and the brightness of the acoustic map includes the features of the seabed.

[0061] S2. The underwater side-scan waterfall image is denoised using an adaptive BM3D denoising algorithm to obtain a denoised image;

[0062] like Figure 4 As shown, Figure 4 This is the final seabed topography map generated from actual XTF data;

[0063] like Figure 5 The diagram shown is a flowchart of the adaptive BM3D denoising algorithm. The waterfall plot denoising method is as follows:

[0064] (1) First, the BM3D algorithm needs to establish a speckle noise model and transform the multiplicative non-normal noise into additive normal noise through power transformation and logarithmic transformation.

[0065] The speckle noise model of side-scan sonar images can be described as:

[0066] X=CZ (1)

[0067] Where X is the image contaminated by noise, C is the real image, and Z is speckle noise that follows a Rayleigh distribution;

[0068] (2) Secondly, the speckle noise of the side-scan sonar image is first transformed from Rayleigh distribution to near Gaussian distribution, and then the side-scan sonar image is logarithmically transformed to make the noise change from multiplicative to additive.

[0069] Adaptive BM3D denoising algorithms include the BM3D algorithm and adaptive parameter tuning algorithms:

[0070] In BM3D, the similarity between image block P and matching block Pmatch is determined by the Euclidean distance between them. All matching blocks Pmatch found with P as the center are sorted, and the Pmatch that is closest to P is selected. This set is then combined with P into a three-dimensional matrix, represented by P3D.

[0071] Next, collaborative filtering is performed. First, a three-dimensional linear transformation is applied to P3D: first a two-dimensional linear transformation, then a one-dimensional linear transformation, followed by hard thresholding in the transform domain, and finally, an inverse transformation is performed to obtain the processed three-dimensional matrix P3D.

[0072]

[0073] Among them, T 2D To represent a two-dimensional linear transformation, this study uses the two-dimensional discrete cosine transform; T1D represents a one-dimensional linear transformation, using a one-dimensional wavelet transform; Y represents hard-domain filtering. Represents a two-dimensional linear inverse transform; This represents a one-dimensional linear inverse transform.

[0074] The 3D matrix P3D is evaluated using PSNR, and the parameters of the BM3D algorithm are adaptively adjusted based on the score feedback.

[0075] Image signal-to-noise ratio (PSNR) is a metric used in engineering to compare the strength of a desired signal with the strength of background noise. Typical peak signal-to-noise ratio values ​​in image compression are between 30 and 40 dB, with higher values ​​being better.

[0076] The adaptive parameter tuning optimization method used is grid search. The parameters are adjusted step by step. By iterating through all possibilities, the parameter with the highest accuracy on the validation set is found. The parameter with the best performance is the final result.

[0077] Regarding the technical solution of this embodiment, BM3D uses a general range of image search radius.

[0078] [4,8,12,16,20,24], the image size is generally in the range of [2,4,6,8,10], and the search step size is generally in the range of [1,2,3,4].

[0079] like Figure 6 As shown, Figure 6 These are example images of the adaptive BM3D denoising algorithm in an underwater obstacle segmentation method based on unmanned surface vessel side scanning provided by this invention. Examples a, b, and c are used to demonstrate the effect.

[0080] S3. Use the QPSO algorithm to segment the denoised image, extract large-area shadow regions, and obtain the final image segmentation contour;

[0081] Based on the two-dimensional maximum entropy theory, the quantum particle swarm optimization algorithm is used to calculate the two-dimensional optimal segmentation threshold vector, which enables the rapid and accurate acquisition of the segmentation threshold vector, improves segmentation efficiency and accuracy, and achieves accurate and efficient segmentation of targets in high-noise side-scan sonar images.

[0082] Based on particle swarm optimization, QPSO is based on the Delta potential well. During the optimization process, it is assumed that the Kth particle is at its optimal position Γ. k =(Γ k1 ,Γ k2 ,...,Γ kD It moves within the Delta potential well centered at 0 (D is the particle dimension).

[0083] Γ kDThe local optimal position of (d∈[1,D]) can be calculated iteratively using the following formula:

[0084]

[0085] like Figure 7 As shown, in the formula and Γ is any random number in the range [0,1]; best The optimal position among all particles (possessing the best fitness value); Γ kd Γ represents the previous local optimal position of the k-th particle. d This represents the local optimal position after the evolution of the k-th particle.

[0086] The QPSO algorithm segments denoised images, and its specific steps are as follows:

[0087] Step 1: The two-dimensional maximum entropy criterion is used as the fitness function of QPSO. Based on QPSO, the segmentation threshold vector (r′, s′) is calculated, and then the target segmentation of the side-scan sonar image is performed.

[0088] Step 2: Due to the obstruction of sound waves by water bodies or lake bottom targets, shadow areas are generated in the side-scan sonar images. The pixel gray values ​​of the shadow areas are very small. The gray-level histogram of set T can be statistically analyzed to obtain the first peak value peak1. The positions in set T with values ​​< peak1 are marked as shadow areas.

[0089] Step 3: Based on the characteristics of the target and shadow appearing in pairs and the shadow appearing behind the target, the shadow area is finally segmented.

[0090] Step 4: Fuse the segmentation results of the target area and shadow area of ​​the side-scan sonar image, and use mathematical morphology operations to extract the edge contour lines of the segmentation results to obtain the final image segmentation contour.

[0091] like Figure 8 The image shown is a QSPO image segmentation result diagram, using examples a, b, and c to demonstrate the effect.

[0092] According to one aspect of the present invention, the present invention provides an underwater obstacle segmentation system based on unmanned surface vessel side scanning, specifically comprising:

[0093] The parsing unit is used to acquire underwater side-scan XTF files, independently parse the XTF format, and reconstruct its signal points into a bidirectional waterfall plot.

[0094] Denoising unit: Used to denoise the underwater side-scan waterfall image using an adaptive BM3D denoising algorithm to obtain a denoised image;

[0095] The image segmentation unit is used to segment the denoised image using the QPSO algorithm, extract large-area shadow regions, and obtain the final image segmentation contour.

[0096] In summary, the technical solution of this invention, by utilizing optimization algorithms such as mesh parameter tuning and PSNR scoring, can automatically obtain suitable selection parameters for BM3D denoising. The image segmentation based on QPSO+ morphological parameter tuning is equivalent to traditional image segmentation algorithms, which can perform morphological segmentation through thresholds faster and more accurately, obtain clear and complete defect region contours, and quickly and accurately cut out obstacles, achieving precise positioning of underwater obstacles and meeting the needs of unmanned vessels for searching and handling seabed targets.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0098] For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances. When an element is referred to as being "assembled on," "mounted on," "fixed to," or "set on" another element, it may be directly on the other element or there may be an intermediate element present. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element present. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible embodiments.

[0099] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0100] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A method for underwater obstacle segmentation based on unmanned surface vessel side scanning, characterized in that, Specifically, the following steps are included: Acquire underwater side-scan XTF files, independently parse the XTF format, and reconstruct a bidirectional waterfall plot from its signal points; An adaptive BM3D denoising algorithm was used to denoise the underwater side-scan waterfall image to obtain a denoised image. The QPSO algorithm is used to segment the denoised image, extract large-area shadow regions, and obtain the final image segmentation contour.

2. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 1, characterized in that: The steps for parsing the XTF format are as follows: First, read 1024 bytes and extract the first byte to determine if the file is in XTF format. If the first byte is 0x7B, then it is an XTF file. Next, check if the number of channels in the header information is greater than 6. If it is greater than 6, continue reading 1024 bytes and then parse out the important header information.

3. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 2, characterized in that: The steps for reconstructing an XTF format image are as follows: The received data lines of each transmission cycle are arranged vertically in a single line and displayed on the monitor, forming a two-dimensional acoustic map of the seabed topography. The acoustic map plane and the seabed plane are in a point-to-point mapping relationship, and the brightness of the acoustic map includes the features of the seabed.

4. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 1, characterized in that: The adaptive BM3D denoising algorithm includes the BM3D algorithm and an adaptive parameter tuning algorithm: In BM3D, the similarity between image block P and matching block Pmatch is determined by the Euclidean distance between them. All matching blocks Pmatch found with P as the center are sorted, and the Pmatch that is closest to P is selected. This set is then combined with P into a three-dimensional matrix, represented by P3D. Next, collaborative filtering is performed. First, a three-dimensional linear transformation is applied to P3D: first a two-dimensional linear transformation, then a one-dimensional linear transformation, followed by hard thresholding in the transform domain, and finally, an inverse transformation is performed to obtain the processed three-dimensional matrix P3D. Among them, T 2D To represent a two-dimensional linear transformation, this study uses the two-dimensional discrete cosine transform; T 1D represents a one-dimensional linear transformation, using a one-dimensional wavelet transform; Y represents hard-domain filtering. Represents a two-dimensional linear inverse transform; This represents a one-dimensional linear inverse transform.

5. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 4, characterized in that, The specific method for denoising underwater side-scan waterfall images using an adaptive BM3D denoising algorithm is as follows: First, a speckle noise model for side-scan sonar images is established. Then, through power transform and logarithmic transform, the multiplicative non-normally distributed noise is transformed into additive normally distributed noise. Secondly, the speckle noise in the side-scan sonar image is first transformed from a Rayleigh distribution to a near-Gaussian distribution, and then a logarithmic transformation is performed on the side-scan sonar image to change the noise from multiplicative to additive.

6. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 5, characterized in that: The speckle noise model of the side-scan sonar image is specifically described as follows: X=CZ (1) Where X is the image contaminated by noise, C is the real image, and Z is speckle noise that follows a Rayleigh distribution.

7. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 6, characterized in that, The 3D matrix P3D is evaluated using the image signal-to-noise ratio (PSNR), and the parameters of the BM3D algorithm are adaptively adjusted based on the score feedback. The adaptive parameter tuning algorithm is a grid search method. It adjusts the parameters sequentially according to a set step size. By iterating through all the parameters, it finds the parameter with the highest accuracy on the validation set, which is the final result.

8. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 1, characterized in that: The specific steps for segmenting the denoised image using the QPSO algorithm are as follows: (1) The two-dimensional maximum entropy criterion is adopted as the fitness function of QPSO. Based on QPSO, the segmentation threshold vector (r′, s′) is calculated, and then the target segmentation of the side scan sonar image is performed. (2) Due to the obstruction of sound waves by water bodies or lake bottom targets, shadow areas are generated in the side scan sonar images. The pixel gray values ​​of the shadow areas are very small. The gray histogram of set T is statistically analyzed to obtain the first peak value peak1. The positions in set T with values ​​< peak1 are marked as shadow areas. (3) Based on the characteristics that the target and shadow appear in pairs and the shadow appears behind the target, the shadow area is finally segmented; (4) The target area and shadow area segmentation results of the side-scan sonar image are fused, and the edge contour lines are extracted from the segmentation results using mathematical morphology operations to obtain the final image segmentation contour.

9. The underwater obstacle segmentation method based on unmanned surface vessel side scanning according to claim 8, characterized in that, Based on the two-dimensional maximum entropy theory, the quantum particle swarm algorithm is used to calculate the two-dimensional optimal segmentation threshold vector; Based on particle swarm optimization, QPSO uses a Delta potential well as a metric. During the optimization process, it is assumed that the Kth particle is at its optimal position Γ. k =(Γ k1 ,Γ k2 ,...,Γ kD It moves within the Delta potential well centered at 0 (where D is the particle dimension). Γ kD The local optimal position of (d∈[1,D]) can be calculated iteratively using the following formula: In the formula and Γ is any random number in the range [0,1]; best The optimal position among all particles (possessing the best fitness value); Γ kd Γ represents the previous local optimal position of the k-th particle. d This represents the local optimal position after the evolution of the k-th particle.

10. An underwater obstacle segmentation system based on unmanned surface vessel side scanning, characterized in that, Specifically, it includes: The parsing unit is used to acquire underwater side-scan XTF files, independently parse the XTF format, and reconstruct its signal points into a bidirectional waterfall plot. Denoising unit: Used to denoise the underwater side-scan waterfall image using an adaptive BM3D denoising algorithm to obtain a denoised image; The image segmentation unit is used to segment the denoised image using the QPSO algorithm, extract large-area shadow regions, and obtain the final image segmentation contour.

Images