A three-dimensional sonar target detection method and device based on fuzzy decoupling
Through fuzzy decoupling and two-stage prediction methods, the problem of target recognition under the conditions of variable shapes of moving objects and high noise in 3D sonar imaging is solved, and high-precision underwater target detection is achieved.
Patent Information
- Application Number
- CN202411313054.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2044-09-20
AI Technical Summary
Existing 3D sonar imaging systems face the problems of variable shapes of moving objects, high noise intensity and difficulty in identifying weak heterogeneity in underwater target detection. Especially in high-noise environments, it is difficult to effectively extract target feature points and improve imaging quality.
A 3D sonar target detection method based on fuzzy decoupling is adopted. By imitating human observers to perform feature decoupling encoding on point clouds, a two-stage refinement is performed by combining Gaussian heat map prediction and Markov prediction to extract target surrounding information and reduce noise interference.
It effectively solves the challenge of weak heterogeneity in underwater three-dimensional sonar target recognition and improves target detection accuracy and imaging quality in high-noise environments.
Smart Images
Figure CN119439137B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of image processing and underwater detection, and relates to a three-dimensional sonar target detection method and device based on fuzzy decoupling. Background Art
[0002] With the growing demand for marine resource exploration, the importance of underwater surveys is steadily increasing. Against this backdrop, 3D acoustic imaging sonar, one of the most attractive underwater technologies, has become a focus of industry attention due to its ability to generate point-based images beyond the optically visible range. In recent years, the widespread application of 3D imaging sonar systems in underwater surveys has expanded into fields such as environmental research, navigation, underwater robotic vision systems, and marine engineering. In these fields, underwater object detection systems play a crucial role in enabling humans to more clearly identify objects.
[0003] Due to their superior performance, 3D sonar systems based on sparse array imaging have become a leading 3D sonar imaging technology. However, two key challenges have emerged during the development of this technology. First, underwater 3D sonar imaging targets have diverse shapes. Moving objects, such as fish and divers, are often accompanied by numerous bubbles emerging or exhaling underwater. These bubbles contain rich information, which is crucial for object recognition and also provides important clues for human observers to identify objects. Therefore, how to effectively utilize the information surrounding the target has become an urgent problem. Second, speckle noise and sidelobe rise away from the mainlobe have become significant in sparse array 3D sonar imaging. Although sparse array imaging 3D sonar systems can effectively reduce the number of elements in the planar array and improve imaging efficiency, they also lead to the problem of sidelobe rise, which to some extent degrades image quality. Therefore, effectively extracting target feature points and obtaining efficient regression proposals in high-noise environments has become particularly urgent.
[0004] On the other hand, with the introduction of PointNet and the surge in research in autonomous driving and home robotics, more and more researchers are delving into 3D object detection. On land, 3D imaging using LiDAR sensors has become the core of major research, while underwater object imaging relies more heavily on 3D sonar systems. The differences in media and object distribution lead to two significant differences in upper and lower water surface recognition. Therefore, when processing these different point cloud types, directly using relatively mature 3D object algorithms based on optical point clouds is no longer sufficient. Object detection algorithms for 3D sonar images must take into account their specific characteristics, particularly the diverse shapes of objects and their weak uniformity in high-noise environments. Summary of the Invention
[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a three-dimensional sonar target detection method based on fuzzy decoupling.
[0006] The purpose of this invention is to address the challenges posed by the variable shape, high noise intensity and weak heterogeneity of moving objects in 3D sonar imaging. The present invention adaptively improves the network by imitating the idea of human observation of three-dimensional sonar targets.
[0007] The technical solution adopted by the present invention to solve its technical problem is:
[0008] A first aspect of the present invention provides a three-dimensional sonar target detection method based on fuzzy decoupling, comprising the following steps:
[0009] S1 To solve the errors that may be caused by collecting point cloud images with different devices, the formula for normalizing the original point cloud is as follows:
[0010]
[0011] where p i represents the intensity of any point in the point cloud, Indicates the intensity of the preprocessed point cloud, p min and p max They represent the parts with the largest and smallest intensity in the entire point cloud respectively.
[0012] In order to imitate human observers and improve the utilization of information around the target, the present invention first performs fuzzy feature decoupling encoding on the sonar raw point cloud based on prior knowledge. The steps are as follows:
[0013] S21 point cloud data is represented as Γ={(x,y,z) n}, where N∈(1,2,3,...,N), and the voxel resolution is (V ω ,V ζ ,V η ). Here, W, H, and L represent the width, height, and length of the scene, respectively. After voxelizing the entire sonar scene, the first step is to assign all points to their respective voxels. The main steps include establishing fuzzy membership functions, partitioning the original point cloud, and performing parallel fuzzy encoding on the decoupled point cloud Γ0 to Γ1.
[0014] In the parallel decoupling process of S22 point cloud, the decision is mainly based on the echo intensity of the point cloud itself and the information of the K nearest neighbors around it. The comprehensive intensity of a point is defined as follows:
[0015]
[0016] Here, r k represents the k points closest to r.
[0017] S23 object points typically have higher echo intensity, in contrast to the lower echo intensity associated with noise and bubbles. Points above the bright spot threshold are classified as "bright spots", while the likelihood of a point being considered an object is positively correlated with its intensity when it is below the bright spot threshold. The twin point clouds are decoupled through a fuzzy model. Due to the concentration of objects and noise, local dual sample imbalance may occur, which may lead to a mismatch in the number of twin point clouds. To address this issue, a dynamic balancing strategy is implemented based on the fuzzy set designed by Zadeh for object points. The membership function is defined as follows:
[0018]
[0019] Here, a represents the sonar bright spot threshold, while α and β represent the adjustment functions in the Cauchy distribution.
[0020] After S24 classification, the average x, y and z coordinates of the point cloud, the average x, y and z intensities and the offset of the physical center of the three-dimensional space grid are calculated to form two point cloud feature codes, each consisting of 9 indicators.
[0021] S3 extracts the encoded features through the VFE module to obtain BEV features. The process of the VFE module is as follows:
[0022] S31 inputs the point cloud with the above 9 features into the fully connected network module FC (fully connected layer FC+batch normalization layer BN+ReLU activation function) to obtain m-dimensional point-by-point features.
[0023] S32 performs unit-by-unit aggregation on the m-dimensional features obtained in the previous step through maximum pooling to obtain local aggregated features.
[0024] S33 respectively concatenates point-by-point features and local aggregation features to obtain aggregation features
[0025] The final aggregated features of S34 are obtained through a maximum pooling and placed back to the original position according to the position of the original point cloud to obtain multi-map-view features.
[0026] After S4 extracts the complete features, it regresses the heatmap through the backbone network and the projection of the final target's 3D bounding box on the BEV view. The center point of the heatmap is the center point of the target. The length, width, and height of the 3D bounding box and the height of the center point are simultaneously predicted to form a one-stage prediction. The loss function based on the Gaussian heatmap is defined as follows:
[0027] S41 Here, the numerical value of the Gaussian heat map generated for a certain category is displayed using the sigmoid function. In addition, this method sets the Gaussian radius as follows:
[0028]
[0029] Here, R Gaussian is the Gaussian radius of each category in the heat map, is the minimum allowed Gaussian radius. In this study, we set w and l are the width and length of the bounding box, respectively, and Λ is the Gaussian radius function. Finally, the 3D bounding box of the target is obtained by decoding the predicted Gaussian heat map and combining it with the output (w, h, l) of the detection head and the height of the center point.
[0030] S42 The first-stage loss of the present invention is defined as follows:
[0031]
[0032] Here, α and β are hyperparameters, and K is the number of positive targets in the ground truth. In this study, α = 2, β = 4. Factor The weight used to share the total loss by controlling the positive and negative samples. The larger α is, The smaller it is, the more effectively the weight of negative samples is reduced, while the weight of positive samples is relatively increased. For other target attributes, various variables are combined into a feature vector of the ground truth value: GT = (u g ,v g ,d g ,w g ,h g ,l g ) and the detected true value: DT=(u d ,v d ,d d ,w d ,h d ,l d ). Therefore, Loss reg is defined as follows:
[0033]
[0034] The final first-stage loss of S43 is expressed as follows:
[0035]
[0036] Here, l and Loss weights, which are adaptive weights. In this method, l=1,
[0037] Finally, S5 uses Markov prediction in the second stage to extract additional point clouds from the original point cloud, extract features through the S2 step, and obtain the final proposal through a multi-head perceptron regression.
[0038] In the second stage, S51, the score and refinement of the predicted bounding box are performed based on the prediction results of SonarPoint in the first stage. The present invention uses the score Y as shown below:
[0039] Υ=min(1,max(0.2·IOU i -0.5))(8)
[0040] Here, IOU i is the IOU value between the i-th predicted box and the ground truth.
[0041] S52Then the loss function of the second stage is defined as follows:
[0042]
[0043] here, is the prediction score. The final score is defined as follows:
[0044]
[0045] here, is the second stage prediction score, is the first stage prediction score.
[0046] The second aspect of the present invention relates to a three-dimensional sonar target detection method and device based on fuzzy decoupling, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the three-dimensional sonar target detection method based on fuzzy decoupling of the present invention.
[0047] A third aspect of the present invention relates to a computer-readable storage medium, characterized in that a program is stored thereon, and when the program is executed by a processor, a three-dimensional sonar target detection method based on fuzzy decoupling of the present invention is implemented.
[0048] The present invention aims to solve the problem of weak target heterogeneity in the three-dimensional sonar point cloud target recognition task by effectively solving the challenge of three-dimensional sonar weak heterogeneity recognition through fuzzy decoupled feature encoding, point-based Gaussian heat map prediction and two-stage refinement based on Markov prediction.
[0049] The working principle of this invention is to propose a fuzzy parallel decoupling method. This method performs fuzzy correlation analysis on small features in the auxiliary point cloud within the object neighborhood space, especially useful information in underwater environments, such as bubbles and sidelobes, thereby enhancing the network's perception of weakly heterogeneous objects. To address the problem of high noise levels destroying the coherence between points, the invention is based on the Mamba backbone network structure to enhance long-range dependencies while maintaining low complexity. Finally, through Markov refinement, the latent information between points is stimulated while reducing irrelevant interference with the 3D bounding box, achieving efficient recognition of weakly heterogeneous sonar targets.
[0050] The innovation of the present invention is to propose a three-dimensional sonar target detection method based on fuzzy decoupling, which effectively solves the problem of weak heterogeneity target recognition in three-dimensional sonar point cloud images.
[0051] The present invention has the following beneficial effects:
[0052] 1. Currently, processing weakly heterogeneous sonar point clouds caused by high noise presents significant challenges. This paper proposes a novel fuzzy decoupling module that, unlike traditional foreground-background segmentation methods, leverages sonar prior knowledge to simultaneously extract valuable information about the target and its surroundings, mitigating heterogeneity caused by noise and sonar sidelobes.
[0053] 2. Currently, there are major challenges in processing weak heterogeneous sonar point clouds caused by missing and occlusions. This paper proposes a two-stage refinement based on Markov prediction. In the second stage, the original point cloud is additionally extracted to obtain more details through Markov prediction.
[0054] 3. Currently, there are major challenges in processing weakly heterogeneous sonar point clouds caused by low ground resolution. This paper proposes a prediction method based on the Gaussian heat map of sonar point clouds. By using the bird's-eye view Gaussian heat map regression method, the high requirements for the backbone network image resolution are reduced, thereby increasing the perception ability of weakly heterogeneous images. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 A flowchart of target detection for weakly heterogeneous 3D sonar point cloud images.
[0056] Figure 2 Schematic diagram of fuzzy parallel decoupling encoding within a single voxel grid.
[0057] Figure 3 Flowchart of the extraction layer for voxel-wise multiple feature encoding. DETAILED DESCRIPTION
[0058] In order to better understand the above technical solution, the technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.
[0059] Example 1
[0060] like Figure 1 As shown in FIG, a 3D sonar target detection method based on fuzzy decoupling is implemented, and the implementation process specifically includes the following steps:
[0061] S1 To solve the errors that may be caused by collecting point cloud images with different devices, the formula for normalizing the original point cloud is as follows:
[0062]
[0063] where p i represents the intensity of any point in the point cloud, Indicates the intensity of the preprocessed point cloud, p min and p max They represent the parts with the largest and smallest intensity in the entire point cloud respectively.
[0064] In order to imitate human observers and improve the utilization of information around the target, the present invention first performs fuzzy feature decoupling encoding on the sonar raw point cloud based on prior knowledge. The steps are as follows:
[0065] S21 point cloud data is represented as Γ={(x,y,z) n}, where n∈(1,2,3,...,N), and the voxel resolution is (V ω ,V ζ ,V η ). Here, W, H, and L represent the width, height, and length of the scene, respectively. After voxelizing the entire sonar scene, the first step is to assign all points to their respective voxels. The main steps include establishing fuzzy membership functions, partitioning the original point cloud, and performing parallel fuzzy encoding on the decoupled point cloud Γ0 to Γ1.
[0066] In the parallel decoupling process of S22 point cloud, the decision is mainly based on the echo intensity of the point cloud itself and the information of the K nearest neighbors around it. The comprehensive intensity of a point is defined as follows:
[0067]
[0068] Here, r k represents the k points closest to r.
[0069] S23 object points typically have higher echo intensity, in contrast to the lower echo intensity associated with noise and bubbles. Points above the bright spot threshold are classified as "bright spots", while the likelihood of a point being considered an object is positively correlated with its intensity when it is below the bright spot threshold. The twin point clouds are decoupled through a fuzzy model. Due to the concentration of objects and noise, local dual sample imbalance may occur, which may lead to a mismatch in the number of twin point clouds. To address this issue, a dynamic balancing strategy is implemented based on the fuzzy set designed by Zadeh for object points. The membership function is defined as follows:
[0070]
[0071] Here, a represents the sonar bright spot threshold, while α and β represent the adjustment functions in the Cauchy distribution.
[0072] S24 Figure 2 As shown in the figure, after classification, the average x, y and z coordinates of the point cloud, the average x, y and z intensities and the offset of the physical center of the three-dimensional space grid are calculated to form two point cloud feature codes, each consisting of 9 indicators.
[0073] S3 extracts the encoded features through the VFE module to obtain BEV features. The process of the VFE module is as follows:
[0074] S31 Figure 3 As shown, the point cloud with the above 9 features is input into the fully connected network module FC (fully connected layer FC+batch normalization layer BN+ReLU activation function) to obtain m-dimensional point-by-point features.
[0075] S32 performs unit-by-unit aggregation on the m-dimensional features obtained in the previous step through maximum pooling to obtain local aggregated features.
[0076] S33 respectively concatenates point-by-point features and local aggregation features to obtain aggregation features
[0077] The final aggregated features of S34 are obtained through a maximum pooling and placed back to the original position according to the position of the original point cloud to obtain multi-map-view features.
[0078] After S4 extracts the complete features, it regresses the heatmap through the backbone network and the projection of the final target's 3D bounding box on the BEV view. The center point of the heatmap is the center point of the target. The length, width, and height of the 3D bounding box and the height of the center point are simultaneously predicted to form a one-stage prediction. The loss function based on the Gaussian heatmap is defined as follows:
[0079] S41 Here, the numerical value of the Gaussian heat map generated for a certain category is displayed using the sigmoid function. In addition, this method sets the Gaussian radius as follows:
[0080]
[0081] Here, R Gaussian is the Gaussian radius of each category in the heat map, is the minimum allowed Gaussian radius. In this study, we set w and l are the width and length of the bounding box, respectively, and Λ is the Gaussian radius function. Finally, the 3D bounding box of the target is obtained by decoding the predicted Gaussian heat map and combining it with the output (w, h, l) of the detection head and the height of the center point.
[0082] S42 The first-stage loss of the present invention is defined as follows:
[0083]
[0084] Here, α and β are hyperparameters, and K is the number of positive targets in the ground truth. In this study, α = 2, β = 4. Factor The weight used to share the total loss by controlling the positive and negative samples. The larger α is, The smaller it is, the more effectively the weight of negative samples is reduced, while the weight of positive samples is relatively increased. For other target attributes, various variables are combined into a feature vector of the ground truth value: GT = (u g ,v g ,d g ,w g ,h g ,l g ) and the detected true value: DT=(u d ,v d ,d d ,w d ,h d ,l d ). Therefore, Loss reg is defined as follows:
[0085]
[0086] The final first-stage loss of S43 is expressed as follows:
[0087]
[0088] Here, l and Loss weights, which are adaptive weights. In this method, l=1,
[0089] Finally, S5 uses Markov prediction in the second stage to extract additional point clouds from the original point cloud, extract features through the S2 step, and obtain the final proposal through a multi-head perceptron regression.
[0090] In the second stage, S51, the score and refinement of the predicted bounding box are performed based on the prediction results of SonarPoint in the first stage. The present invention uses the score Y as shown below:
[0091] Υ=min(1,max(0.2·IOU i -0.5))(8)
[0092] Here, IOU i is the IOU value between the i-th predicted box and the ground truth.
[0093] S52Then the loss function of the second stage is defined as follows:
[0094]
[0095] here, is the prediction score. The final score is defined as follows:
[0096]
[0097] here, is the second stage prediction score, is the first stage prediction score.
[0098] Example 2
[0099] This embodiment relates to a three-dimensional sonar target detection device based on fuzzy decoupling, including a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, they are used to implement a three-dimensional sonar target detection method based on fuzzy decoupling in Example 1.
[0100] Example 3
[0101] This embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a three-dimensional sonar target detection method based on fuzzy decoupling of the present invention is implemented.
[0102] Finally, it should be noted that the specific scheme and embodiment described above in conjunction with the illustrations are only a preferred embodiment of the present invention. Technical personnel in this profession may make slight changes and modifications to the technical content described without departing from the technical scope of the present invention. However, modifications and deformations made without departing from the spirit and principles of the present invention should fall within the scope of the technical scheme of the present invention.
Claims
1. A three-dimensional sonar target detection method based on fuzzy decoupling, comprising the following steps: S1 normalizes the original point cloud to solve the errors that may be caused by collecting point cloud images with different devices; In order to imitate human observers and improve the utilization of information around the target, S2 first performs fuzzy feature decoupling encoding on the sonar raw point cloud based on prior knowledge; this includes: S21 point cloud data is represented as Γ={(x,y,z) n }, where n∈(1,2,3,...,N), and the voxel resolution is (V ω ,V ζ ,V η ); Here, W, H, and L represent the width, height, and length of the scene, respectively; After voxelizing the entire sonar scene, all points are first assigned to their respective voxels. The steps include establishing a fuzzy membership function, partitioning the original point cloud, and performing parallel fuzzy encoding on the decoupled point cloud Γ0 to Γ1; During the parallel decoupling process of the point cloud, S22 makes decisions based on the echo intensity of the point cloud itself and the information of the K nearest neighbors. The comprehensive intensity of a point is defined as follows: Here, r k represents the k points closest to r; S23 object points typically have high echo intensity, in contrast to the lower echo intensity associated with noise and bubbles. Points above the bright spot threshold are classified as "bright spots," while the likelihood of a point being considered an object is positively correlated with its intensity below the bright spot threshold. Twin point clouds are decoupled using a fuzzy model. Due to the concentration of objects and noise, local dual sample imbalance occurs, resulting in a mismatch in the number of dual point clouds. A dynamic balancing strategy is implemented based on the fuzzy set designed by Zadeh for object points. The membership function is defined as follows: Here, a represents the sonar bright spot threshold, while α and β represent the adjustment functions in the Cauchy distribution; After S24 classification, the average x, y and z coordinates of the point cloud, the average x, y and z intensity and the offset of the physical center of the three-dimensional space grid are calculated to form two point cloud feature codes, each consisting of 9 indicators; S3 extracts the encoded features through the VFE module to obtain BEV features; After S4 extracts the complete features, it regresses the heatmap and the projection of the final target's 3D box on the BEV view through the backbone network. The center point of the heatmap is the center point of the target. At the same time, the length, width, and height of the 3D box and the height of the center point are predicted to form a one-stage prediction. Finally, S5 uses Markov prediction in the second stage to extract additional point clouds from the original point cloud, extract features through the S2 step, and obtain the final proposal through a multi-head perceptron regression.
2. The three-dimensional sonar target detection method based on fuzzy decoupling according to claim 1, characterized in that: The formula for normalizing the original point cloud in step S1 is as follows: where p i represents the intensity of any point in the point cloud, Indicates the intensity of the preprocessed point cloud, p min and p max They represent the parts with the largest and smallest intensity in the entire point cloud respectively.
3. The three-dimensional sonar target detection method based on fuzzy decoupling according to claim 1, characterized in that: Step S3 includes: S31 inputs the point cloud of 9 features into the fully connected network module FC to obtain m-dimensional point-by-point features; S32 aggregates the m-dimensional features obtained in the previous step by unit by unit through the maximum pooling to obtain local aggregated features; S33 respectively concatenates point-by-point features and local aggregation features to obtain aggregation features S34 finally aggregates the features through a maximum pooling to obtain the final features and places them back to their original positions according to the positions of the original point cloud to obtain a multi-voxel feature map.
4. The three-dimensional sonar target detection method based on fuzzy decoupling according to claim 1, characterized in that: In step S4, the loss function based on the Gaussian heat map is defined as follows: S41 Here, the numerical value of the Gaussian heat map generated for a certain category is displayed using the sigmoid function; in addition, this method sets the Gaussian radius as follows: Here, R Gaussian is the Gaussian radius of each category in the heat map, is the minimum allowed Gaussian radius; in this study, setting w and l are the width and length of the bounding box, respectively, and Λ is the Gaussian radius function. Finally, the 3D bounding box of the target is obtained by decoding the predicted Gaussian heat map and combining it with the output (w, h, l) of the detection head and the height of the center point. The S42 first-stage loss is defined as follows: Here, α and β are hyperparameters, K is the number of positive targets in the ground truth; the factor The weight used to share the total loss by controlling the positive and negative samples; the larger α is, The smaller it is, the more effectively the weight of the negative sample is reduced, while the weight of the positive sample is relatively increased; for other target attributes, various variables are merged into the feature vector of the ground truth value: GT = (u g ,v g ,d g ,w g ,h g ,l g ) and the detected true value: DT=(u d ,v d ,d d ,w d ,h d ,l d ); therefore, Loss reg is defined as follows: The final first-stage loss of S43 is expressed as follows: Here, l and Loss weights, which are adaptive weights.
5. The three-dimensional sonar target detection method based on fuzzy decoupling according to claim 1, characterized in that: Step S5 includes: The second stage of S51 predicts the score and refinement of the bounding box based on the prediction results of SonarPoint in the first stage; the score Υ is shown below i : Y i =min(1,max(0.2·IOU i -0.5))(8) Here, IOU i is the IOU value between the i-th predicted box and the ground truth; S52Then the loss function of the second stage is defined as follows: here, is the prediction score; the final score is defined as follows: here, is the second stage prediction score, is the first stage prediction score.
6. The three-dimensional sonar target detection method based on fuzzy decoupling according to claim 1, characterized in that: The fuzzy decoupling coding based on prior knowledge in step S2 first divides the point cloud scene into voxel levels and performs fuzzy division through the fuzzy membership function defined in S23. Second, the decoupled point cloud is encoded to form a unique point cloud coding feature within the grid. In step S4, the first-stage proposal reasoning based on the Gaussian heat map is performed, and the results are displayed in a bird's-eye view of the map perspective. The preliminary prediction results are then refined in step S5; The two-stage network based on Markov prediction and its loss function in step S5 first refine the two-stage point cloud based on the original point cloud through Markov prediction; second, combine the one-stage results in step S4 to give the final three-dimensional marking box score information.
7. A three-dimensional sonar target detection method and device based on fuzzy decoupling, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a three-dimensional sonar target detection method based on fuzzy decoupling according to any one of claims 1 to 6.
8. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, a three-dimensional sonar target detection method based on fuzzy decoupling according to any one of claims 1 to 6 is implemented.