Underwater three-dimensional imaging method based on orthogonal sonar system
By acquiring complementary observation information through orthogonal sonar systems, and utilizing feature extraction, clustering, and matching to achieve three-dimensional reconstruction of underwater targets, the problem of missing elevation angle information from two-dimensional sonar is solved, reducing costs and improving the completeness and stability of reconstruction results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HARBIN ENG UNIV
- Filing Date
- 2026-01-16
- Publication Date
- 2026-05-01
AI Technical Summary
In existing underwater 3D imaging technologies, 2D sonar cannot obtain elevation angle information in the vertical direction, resulting in imaging results that are 2D projections and lack information on the height and three-dimensional structure of the target. Existing 3D reconstruction methods rely on single sonar images, which are easily interfered with or require complex arrays and high-precision carrier movement.
Two-dimensional image sonars with orthogonal deployment are used to acquire complementary observation information. Through feature extraction, clustering, matching and information fusion, three-dimensional point cloud reconstruction of underwater targets is achieved. Two-dimensional imaging information is acquired by horizontal and vertical sonars respectively and matched with feature descriptors.
It eliminates the need for complex array structures and carrier movement, reducing system cost and complexity, compensating for missing elevation angle information, improving the completeness and stability of reconstruction results, and enhancing adaptability to noise and complex environments.
Smart Images
Figure CN121962448A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of underwater acoustic imaging and 3D reconstruction technology, and in particular to an underwater 3D imaging method based on an orthogonal sonar system. Background Technology
[0002] Underwater 3D reconstruction technology, as a crucial component of underwater sensing, is widely applied in underwater structure detection, target recognition, and environmental modeling. Currently, acquiring underwater 3D information primarily relies on 3D imaging sonar or 2D image sonar combined with height estimation methods. However, 3D imaging sonar has complex hardware structures, large array sizes, and high equipment costs, and demands high data processing capabilities and real-time system performance. While 2D image sonar technology is more mature, with simpler structures, lower costs, and greater adaptability, it can only acquire information about the target's distance and azimuth during imaging, failing to directly obtain elevation angle information in the vertical direction. This results in imaging results that are essentially 2D projections, lacking information about the target's height and three-dimensional structure.
[0003] To compensate for the lack of elevation angle information in two-dimensional sonar, existing research has proposed various 3D reconstruction methods based on single sonar images. These include estimating target height using the geometric relationship between target echoes and shadow areas, inferring target surface morphology through brightness distribution and edge features, or achieving 3D reconstruction by fusing data from multiple two-dimensional sonar images. However, these methods all have limitations. Reconstruction methods relying on a single two-dimensional sonar image often infer height information from target shadow information, making them highly sensitive to the accuracy of image segmentation and susceptible to interference. Multi-view reconstruction methods depend on the stability of the sonar carrier's motion, placing high demands on system control precision and environmental adaptability, thus limiting their application in complex sea conditions. Therefore, there is an urgent need for a 3D underwater target reconstruction method that does not require complex array structures, does not rely on carrier motion trajectories, and can effectively compensate for the lack of elevation angle information in two-dimensional sonar. Summary of the Invention
[0004] To address the shortcomings of the existing technologies, the present invention aims to provide an underwater three-dimensional imaging method based on an orthogonal sonar system. This method utilizes orthogonally deployed two-dimensional image sonars to acquire complementary observation information, and through feature extraction, feature clustering, feature matching, and information fusion, achieves three-dimensional point cloud reconstruction of underwater targets.
[0005] The technical solution adopted in this invention is an underwater three-dimensional imaging method based on an orthogonal sonar system, the steps of which are as follows:
[0006] Step S1, Sonar Image Acquisition: Acquire the horizontal sonar image collected by the horizontal sonar and the vertical sonar image collected by the vertical sonar. The horizontal sonar and the vertical sonar are installed in mutually orthogonal directions. The horizontal sonar is used to acquire two-dimensional imaging information of the target in the horizontal direction, and the vertical sonar is used to acquire two-dimensional imaging information of the target in the vertical direction. The detection fields of the two have overlapping areas.
[0007] Step S2, Feature Extraction: Based on the horizontal and vertical sonar images, preprocessing is performed, and feature extraction is performed on the preprocessed sonar images to extract feature points that reflect the echo characteristics of the target surface from the sonar images, forming a set of horizontal feature points and a set of vertical feature points.
[0008] Step S3, Feature Clustering: Based on the horizontal and vertical feature point sets generated in step S2, clustering is performed to divide feature points that are spatially close and have similar echo characteristics into the same feature cluster to form multiple target candidate regions;
[0009] Step S4, Feature point processing: Perform processing operations on the feature points, including cropping the edge regions in the sonar image and adding sonar offset to the sonar data to obtain feature point coordinate information under a unified spatial reference system, thereby eliminating the spatial offset effect caused by sonar installation attitude and imaging geometry.
[0010] Step S5, Feature matching: Construct feature descriptors based on the spatial location information and echo intensity information of feature points, and perform feature matching operations between the horizontal feature point set and the vertical feature point set to determine matching feature point pairs from different sonar images but corresponding to the same physical target point;
[0011] Step S6, 3D Reconstruction: Based on the matched feature point pairs, the distance and azimuth information in the horizontal sonar image and the distance and elevation angle information in the vertical sonar image are fused to calculate the position coordinates of the target feature points in 3D space and generate the 3D point cloud data of the target, thereby realizing the 3D reconstruction of the underwater target.
[0012] Preferably, the feature extraction process in step S2 is as follows:
[0013] Noise is suppressed by smoothing the data using Gaussian filtering, then candidate points are located using maximum value filtering, and finally false alarms are filtered out using dynamic thresholding, resulting in a sparse salient target matrix. Gaussian filtering can perform linear smoothing; its core idea is to introduce a neighborhood weighted average operation for each pixel in the image, with weights calculated based on a two-dimensional Gaussian function distribution. The formula for filtering the raw sonar data using a Gaussian kernel is as follows:
[0014]
[0015] In the formula, The original sonar image in pixel coordinates The intensity value at that location, These are the intensity values of the sonar image after Gaussian filtering. It is the spatial offset relative to the center pixel in the filter window. It is the standard deviation parameter that controls the filtering scale.
[0016] On the denoised image, a maximum value filter is applied to traverse all pixels within a sliding window, as shown in the formula:
[0017]
[0018] In the formula, Therefore The maximum intensity value within the center window. It is the window area. It represents any pixel coordinate within the window;
[0019] The mean and standard deviation of the data are adaptively set based on the overall statistical characteristics of the image, using the following formula:
[0020]
[0021] In the formula, For adaptive detection threshold, The mean of the sonar image, whether global or local. The input threshold factor, This represents the global or local standard deviation of the sonar image.
[0022] Preferably, the feature clustering step in step S3 is as follows:
[0023] By traversing the data, all core points within a neighborhood radius that have more than `min_samples` neighbors are encoded, and these core points, along with their neighbors, form an initial cluster. All other unassigned points within a cluster are assigned to that cluster.
[0024] A point The neighborhood is defined as:
[0025]
[0026] In the formula, These are the feature points to be judged. It is any feature point in the dataset. It is a dataset. It is the neighborhood radius. It is Euclidean distance. It is a point The set of neighboring points.
[0027] The formula for determining the core point is:
[0028]
[0029] In the formula To determine the minimum number of neighbors required for a core point, the data is... The concentration does not belong to any cluster The set of points is defined as noise.
[0030] Due to the characteristics of sound waves, in addition to the direct wave, there are other echoes between the sonar and the object. These echoes travel farther and have longer echo times, and when received by the sonar, they are incorrectly displayed at a farther location than they actually are. The information generated by these sound waves is also displayed in the sonar image. For example, if the cylinder has a large vertical length, the sound wave will hit the front surface first, but it may also bounce back from the top, bottom, or rear, resulting in secondary echoes. These secondary echoes, due to their longer propagation paths, will appear at a farther distance, causing the image to be similar to but not completely identical to the actual target. To eliminate the influence of the sonar imaging boundary on the spatial distribution of feature points, the coordinate processing step in step S4 performs cropping processing on the fan-shaped edge region in the sonar image.
[0031] In the clustered point set, a certain cluster number is The point set is:
[0032]
[0033] In the formula, For the first The point set obtained by clustering For a point in the cluster polar angle, For point The polar radius, For point The strength, For the first The number of cluster points;
[0034] By discretizing the beam, the beam angle direction is discretized into several intervals of fixed width, each interval containing a group of points with approximately equal angles:
[0035]
[0036] In the formula, This indicates the m-th angle group; This represents the starting angle of the m-th discrete beam; The discrete precision of the beam angle; This represents a feature point that satisfies the angle condition.
[0037] Preferably, the feature matching step in step S5 constructs a joint feature descriptor based on the spatial distance information of feature points and the echo intensity information; feature matching is completed by an optimal matching strategy that minimizes the differences of the joint feature descriptor.
[0038] Furthermore, in step S5, the feature matching step, based on the range, intensity, and local average intensity feature descriptor, comprehensively considers the following three differences:
[0039] The coordinate difference that measures the distance deviation between a horizontal point and a vertical point:
[0040]
[0041] In the formula, Indicates distance difference, This represents the distance to a feature point in a horizontal sonar. This represents the distance to a feature point in a vertical sonar.
[0042] This represents the intensity difference between two points in the echo:
[0043]
[0044] In the formula, Indicates differences in intensity. Indicates the intensity of the horizontal sonar point echo. This indicates the intensity of the vertical sonar point echo.
[0045] The intensity gradient is defined as the difference between adjacent points. First, the intensity difference between neighboring points is calculated to capture local edge change information, which is then used to construct the gradient information of the image.
[0046]
[0047] In the formula, Represents the local intensity gradient. This represents the intensity value of adjacent feature points.
[0048] The gradient difference is used to measure the difference in local structural changes between two points:
[0049]
[0050] In the formula, This represents the local intensity gradient.
[0051] Integrating the above three types of differences, we obtain the matching cost:
[0052]
[0053] In the formula, To represent the feature points in the horizontal sonar Feature points in vertical sonar The overall cost of performing the matching;
[0054] Build a Cost matrix:
[0055]
[0056] In the formula, Represents the matching cost matrix; The dimension of the matching cost matrix
[0057] The Hungarian algorithm is used to find the optimal solution to the linear assignment problem, returning a set of matching pairs that minimize the total cost.
[0058]
[0059] In the formula, This is the final set of successfully matched point pairs. Representing horizontal feature points , Representing vertical feature points , Assigning a binary variable, when the feature point in the horizontal sonar Feature points in vertical sonar When selected as a matching pair, ,otherwise .
[0060] Preferably, in step S6, the three-dimensional reconstruction step calculates the coordinate position of the target feature point in three-dimensional space by fusing distance, azimuth and elevation information based on the geometric constraint relationship of the matching feature points in the horizontal and vertical sonar images, and generates a three-dimensional point cloud model of the target.
[0061] Compared with existing technologies, this invention proposes a method for three-dimensional reconstruction of underwater targets based on orthogonal sonar information. The advantages of this method are:
[0062] (1) No complex three-dimensional imaging sonar is required, which significantly reduces system cost and structural complexity;
[0063] This invention acquires complementary observation information by orthogonally deployed two-dimensional image sonar, achieving three-dimensional reconstruction of the target without the need for large-scale arrays or three-dimensional imaging sonar. This effectively reduces hardware costs and system complexity, making it suitable for deployment in small underwater platforms and resource-constrained scenarios.
[0064] (2) To compensate for the lack of elevation angle information in two-dimensional sonar;
[0065] This invention utilizes the azimuth and elevation information contained in orthogonal sonar images to recover the target's position information in three-dimensional space through feature matching and information fusion. This overcomes the problem of missing elevation information in single two-dimensional sonar imaging and improves the completeness and spatial consistency of the target's three-dimensional reconstruction results.
[0066] (3) It does not depend on the trajectory of the carrier;
[0067] This invention can complete the three-dimensional reconstruction process under the condition of fixed orthogonal sonar deployment, without the need to obtain multi-view information through carrier movement, avoiding dependence on motion trajectory planning, attitude estimation and time synchronization, and improving the stability and engineering applicability of the system in complex underwater environments.
[0068] (4) Enhance the ability to adapt to noise and imaging boundaries;
[0069] This invention introduces a joint feature description based on spatial location and echo intensity in the feature matching stage, and processes the edge region of sonar imaging to effectively suppress the influence of noise points and imaging boundaries on the reconstruction results, thereby improving the robustness of 3D reconstruction under complex background and weak echo conditions. Attached Figure Description
[0070] Figure 1 This is an overall flowchart of the method of the present invention.
[0071] Figure 2 This is a frame of raw horizontal sonar data from an example.
[0072] Figure 3 This is a frame of raw data from a vertical sonar system, as shown in the example.
[0073] Figure 4 For example, in Figure 2 The result of feature extraction based on the above;
[0074] Figure 5 For example, in Figure 3 The result of feature extraction based on the above;
[0075] Figure 6 For example, in Figure 4 The result of feature clustering based on this;
[0076] Figure 7 For example, in Figure 5 The result of feature clustering based on this;
[0077] Figure 8 For example, in Figure 6 and Figure 7 The result of reconstruction based on the above. Detailed Implementation
[0078] The technical solutions of the embodiments of this application will be further clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0079] To make the inventive objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings: In order to better understand the above-mentioned objectives, features, and advantages of this invention, the advantages of this invention will be further illustrated below by comparing the embodiments with the accompanying drawings and specific implementation methods.
[0080] This invention proposes an underwater three-dimensional imaging method based on an orthogonal sonar system. The overall flowchart of this method is as follows: Figure 1 As shown.
[0081] To verify the effectiveness of this invention in practical applications, this embodiment uses dual-frequency imaging sonar to collect and process a series of data. The target being detected is a cylinder with a main body length of 1.19 meters. It is pulled by a rope and vertically placed into the test pool at a distance of 4 meters from the sonar. To simplify the experiment, the relative positions of the object and the sonar are adjusted so that the elevation angle of the two sonars is 0°.
[0082] In the experiment, the method of this invention was applied to the original sonar images for target reconstruction, demonstrating the reconstruction effect in the most intuitive way. The results directly stem from the core design goal of this invention: to supplement elevation angle information through orthogonal sonar multi-view data fusion.
[0083] like Figure 4 and Figure 5 As shown, the technical solution of the present invention achieves the feature extraction described in step S2, respectively for... Figure 2 and Figure 3 Preprocessing is performed to dynamically adjust the detection threshold based on the statistical characteristics of local background noise in the sonar image in order to separate the target echo region from the background noise, and local extremum detection is performed within the target echo region to extract feature points that reflect the echo characteristics of the target surface.
[0084] Secondly, based on feature extraction, Figure 6 and Figure 7 This embodies the feature clustering in step S3 of the present invention: clustering is performed based on the spatial location distribution of feature points and the similarity of echo intensity, so that feature points corresponding to the same physical target are divided into the same feature cluster, thereby reducing the interference of isolated noise points on the subsequent feature matching process.
[0085] Furthermore, by performing processing operations on the feature points in step S4 of the present invention, the fan-shaped edge region in the sonar image is cropped to eliminate the influence of the sonar imaging boundary on the spatial distribution of feature points. This further includes converting the polar coordinate data in the sonar image into Cartesian coordinate data in a unified reference coordinate system according to the sonar installation attitude parameters, thereby achieving the alignment of feature points from different sonars in the same spatial coordinate system.
[0086] Then, through feature matching in step S5 of the present invention, a joint feature descriptor is constructed based on the spatial distance information of feature points and the echo intensity information, and cross-sonar feature matching is completed by minimizing the difference of feature descriptors, thus completing the three-dimensional reconstruction in step S6 of the present invention.
[0087] The result of all these processes working together is reflected in Figure 8 The figure shows the point cloud distribution of the target object in three-dimensional space. The length of the point cloud ranges from -0.1 meters to -0.5 meters, with a length of 0.4 meters; the width ranges from 3.8 meters to 4.1 meters, with a width of 0.3 meters; and the height ranges from -2.23 meters to 0.13 meters, with a height of 2.36 meters, exhibiting a vertical columnar distribution.
[0088] In summary, this example verifies that the underwater 3D imaging method based on an orthogonal sonar system proposed in this invention exhibits good stability in noise suppression, feature clustering, and cross-view matching, and can accurately recover the 3D morphology of simple geometric targets.
Claims
1. An underwater three-dimensional imaging method based on an orthogonal sonar system, characterized in that, Includes the following steps: Step S1, sonar image acquisition: Acquire horizontal sonar images acquired by horizontal sonar and vertical sonar images acquired by vertical sonar, wherein the horizontal sonar and vertical sonar are deployed in mutually orthogonal directions and the detection fields of view overlap. Step S2, Feature Extraction: Feature extraction is performed based on horizontal and vertical sonar images to obtain sets of horizontal and vertical feature points. Step S3, Feature Clustering: Based on the horizontal and vertical feature point sets generated in step S2, feature clustering is performed to form multiple target candidate regions; Step S4, Coordinate Processing: The feature points are processed to ensure that feature points from different sonars are located in a unified spatial reference coordinate system; Step S5, Feature Matching: Based on the spatial location information and echo intensity information of feature points, feature matching is performed between the horizontal feature point set and the vertical feature point set to obtain matching feature point pairs; Step S6, 3D Reconstruction: Based on the matched feature point pairs, the distance and azimuth information from the horizontal sonar image and the distance and elevation information from the vertical sonar image are fused to calculate the position coordinates of the target feature points in three-dimensional space and generate the target's three-dimensional point cloud data.
2. The underwater three-dimensional imaging method based on an orthogonal sonar system according to claim 1, characterized in that, The feature extraction process in step S2 is as follows: The data is smoothed by Gaussian filtering to suppress noise, then candidate points are located by maximum value filtering, and finally false alarms are filtered by dynamic thresholding, resulting in a sparse salient target matrix. Gaussian filtering performs linear smoothing filtering by introducing a neighborhood-weighted average operation for each pixel in the image, with the weights calculated based on a two-dimensional Gaussian function distribution. The formula for filtering the raw sonar data using a Gaussian kernel is as follows: ； In the formula, The original sonar image in pixel coordinates The intensity value at that location, These are the intensity values of the sonar image after Gaussian filtering. It is the spatial offset relative to the center pixel in the filter window. It is the standard deviation parameter that controls the filtering scale; On the denoised image, a maximum value filter is applied to traverse all pixels within a sliding window, as shown in the formula: ； In the formula, Therefore The maximum intensity value within the center window. It is the window area. It represents any pixel coordinate within the window; The mean and standard deviation of the data are adaptively set based on the overall statistical characteristics of the image, using the following formula: ； In the formula, For adaptive detection threshold, The mean of the sonar image, whether global or local. The input threshold factor, This represents the global or local standard deviation of the sonar image.
3. The underwater three-dimensional imaging method based on an orthogonal sonar system according to claim 1, characterized in that, The steps for feature clustering in step S3 are as follows: By traversing the data, all core points within a neighborhood radius that have more than min_samples neighbors are encoded, and these core points, together with their neighbors, form an initial cluster; all other unassigned points within a cluster are assigned to that cluster. A point The neighborhood is defined as: ； In the formula, These are the feature points to be judged. It is any feature point in the dataset. It is a dataset. It is the neighborhood radius. It is Euclidean distance. It is a point The set of neighborhood points; The formula for determining the core point is: ； In the formula To determine the minimum number of neighbors required for a core point, the data is... The concentration does not belong to any cluster The set of points is defined as noise.
4. The underwater three-dimensional imaging method based on an orthogonal sonar system according to claim 1, characterized in that, In step S4, the coordinate processing step cropps the fan-shaped edge region in the sonar image: In the clustered point set, a certain cluster number is The point set is: ； In the formula, For the first The point set obtained by clustering For a point in the cluster polar angle, For point The polar radius, For point The strength, For the first The number of cluster points; By discretizing the beam, the beam angle direction is discretized into several intervals of fixed width, each interval containing a group of points with approximately equal angles: ； In the formula, This indicates the m-th angle group; This represents the starting angle of the m-th discrete beam; The discrete precision of the beam angle; This represents a feature point that satisfies the angle condition.
5. The underwater three-dimensional imaging method based on an orthogonal sonar system according to claim 1, characterized in that, In step S5, the feature matching step constructs a joint feature descriptor based on the spatial distance information of feature points and the echo intensity information; feature matching is completed by an optimal matching strategy that minimizes the differences of the joint feature descriptor.
6. The underwater three-dimensional imaging method based on an orthogonal sonar system according to claim 5, characterized in that, The feature matching step, based on the feature descriptor of range, intensity, and local average intensity, comprehensively considers the following three differences: The coordinate difference that measures the distance deviation between a horizontal point and a vertical point: ； In the formula, Indicates distance difference, This represents the distance to a feature point in a horizontal sonar. This represents the distance to a feature point in a vertical sonar; This represents the intensity difference between two points in the echo: ； In the formula, Indicates differences in intensity. Indicates the intensity of the horizontal sonar point echo. Indicates the vertical sonar point echo intensity; The intensity gradient is defined as the difference between adjacent points. First, the intensity difference between neighboring points is calculated to capture local edge change information, which is then used to construct the gradient information of the image. ； In the formula, Represents the local intensity gradient. Indicates the intensity value of adjacent feature points; The gradient difference is used to measure the difference in local structural changes between two points: ； In the formula, Indicates the local intensity gradient; Integrating the above three types of differences, we obtain the matching cost: ； In the formula, To represent the feature points in the horizontal sonar Feature points in vertical sonar The overall cost of performing the matching; Build a Cost matrix: ； In the formula, Represents the matching cost matrix; The dimension of the matching cost matrix The Hungarian algorithm is used to find the optimal solution to the linear assignment problem, returning a set of matching pairs that minimize the total cost. ； In the formula, This is the final set of successfully matched point pairs. Representing horizontal feature points , Representing vertical feature points , Assigning a binary variable, when the feature point in the horizontal sonar Feature points in vertical sonar When selected as a matching pair, ,otherwise .
7. The underwater three-dimensional imaging method based on an orthogonal sonar system according to claim 1, characterized in that, The three-dimensional reconstruction step calculates the three-dimensional coordinates of the target feature points by fusing distance, azimuth, and elevation information based on the geometric constraints of the matching feature points in the horizontal and vertical sonar images.