Underwater scene three-dimensional reconstruction method based on two-dimensional sonar image gray level inversion height

Through the Lambertian reflection model method and combined with geometric size constraints, three-dimensional information is extracted from two-dimensional sonar images, solving the problem that two-dimensional sonar images cannot describe three-dimensional space, and improving the accuracy and processing of underwater obstacle recognition Efficiency, suitable for unmanned underwater vehicles.

CN118915077APending Publication Date: 2024-11-08NORTHWESTERN POLYTECHNICAL UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411024978.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the prior art, two-dimensional sonar images cannot describe three-dimensional spatial information, resulting in insufficient identification and positioning of underwater obstacles, and the existing three-dimensional reconstruction methods have problems such as insufficient practicality, large calculation volume, and high hardware costs.

Method used

Using the method based on Lambertian reflection model, the height information is obtained by dividing the two-dimensional sonar image into multiple areas, inversion calculation is performed, and the real height information is estimated, and finally converted into a three-dimensional point cloud to realize underwater scenes. 3D reconstruction.

Benefits of technology

It realizes the direct extraction of three-dimensional information from two-dimensional sonar images, reduces dependence on equipment, improves processing efficiency and data utilization, and is suitable for obstacle recognition of unmanned underwater vehicles, providing complete three-dimensional spatial information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915077A_ABST
    Figure CN118915077A_ABST
Patent Text Reader

Abstract

The invention relates to an underwater scene three-dimensional reconstruction method based on two-dimensional sonar image gray level inversion height, and belongs to the field of sonar technology and acoustic image processing. Comprising the following steps: acquiring a sonar two-dimensional image, and dividing the sonar two-dimensional image into a plurality of areas according to direction angles; an inversion model is constructed according to a Lambertian reflection model; performing inversion calculation on each divided region based on an inversion model to obtain height information, and performing splicing and normalization processing on results of each region to obtain a normalized height information matrix of a detection region corresponding to the sonar image; adding the normalized height information matrix into the geometric dimension constraint of the sonar detection area, and estimating the real height information in the detection area; and the estimated real height information in the sonar image detection area is converted into a three-dimensional point cloud. The method solves the problem that a two-dimensional sonar image lacks three-dimensional information, provides three-dimensional information of a sound irradiation area for underwater scene detection, and has a good application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for three-dimensional reconstruction of underwater sound scenes of two-dimensional sonar images, which is suitable for obstacle recognition and positioning of underwater vehicles and belongs to the field of sonar technology and sound image processing. Background Art

[0002] Sonar is often used as an environmental information collection device in unmanned underwater vehicles. The echo information of the sonar is processed by acoustic imaging to obtain a two-dimensional sonar image to realize the visualization of underwater information. Common sonar images include side-scan sonar images and forward-looking sonar images. Most of them are two-dimensional plane images, which lose the height value and cannot describe the three-dimensional spatial information. Processing the acoustic echo information into a three-dimensional acoustic image can make the identification and positioning of underwater obstacles more accurate. However, the current mainstream sonar images are two-dimensional images. The two-dimensional acoustic image is a distribution diagram of the acoustic reflection intensity at a certain horizontal angle of the sonar system with distance. Commercial acoustic imaging systems such as forward-looking and side-scanning are limited by the sonar array. Acoustic detection is still mainly based on two-dimensional acoustic images, which lose the height information in the vertical direction and cannot describe the three-dimensional space. The grayscale of the two-dimensional acoustic image is related to the acoustic reflection intensity, and the spatial position of the sonar platform relative to the detection area and the acoustic irradiation angle determine the acoustic reflection intensity. Extracting information from the two-dimensional acoustic image to obtain three-dimensional information can perform three-dimensional reconstruction of underwater scenes without increasing the detection cost, thereby improving data utilization.

[0003] At present, there are several methods for 3D reconstruction of underwater environment based on sonar information: estimating the height of underwater target from the length of target shadow and the position of sonar platform (Wang JY, Han J, Du PF, Jing DX, Chen JW, Qu F Z, Three-Dimensional Reconstruction of Underwater Objects from Side scan Sonar Images[C]. OCEANS'17 MTS / IEEE, Aberdeen, 19-22 June, 2017.), but since the position of sonar platform relative to the target is usually difficult to estimate accurately, this method is not very practical; scanning underwater target from multiple angles by side scan sonar along a fixed circular track, and reconstructing the 3D image of the target through a series of sonar images obtained from different angles (Aykin MD, Negahdaripour S. Three-dimensional target reconstruction from multiple 2-d forward-scan sonar views by space carving[J]. IEEE Journal of Oceanic Engineering, 2017, 42(3): 574-589.). This method is not suitable for underwater vehicle obstacle avoidance and target detection with high real-time requirements; splicing multi-beam bathymetric sonar images together frame by frame to achieve three-dimensional reconstruction (Liu Zhe. Research on pipeline detection and positioning algorithm based on underwater three-dimensional point cloud [D]. Harbin Engineering University, 2020. DOI: 10.27060 / d.cnki.ghbcu.2020.001332.), which requires processing a large number of images and constantly updating the acquired images, which requires a large amount of calculation and the accuracy is also affected by the platform movement trajectory; CodaOctopus company promotes The Echoscope three-dimensional imaging sonar, which uses an acoustic array with three-dimensional spatial acoustic detection capabilities to directly obtain three-dimensional environmental information in real time, is suitable for unmanned underwater vehicles, but its hardware cost is expensive and the amount of calculation is large. In order to improve the accuracy of three-dimensional reconstruction of underwater targets and reduce dependence on equipment, Zhao Jianhu et al. used the minimum value method to invert the height value of the target from the grayscale of the two-dimensional side-scan sonar image (Zhao Jianhu, Shang Xiaodong, Zhang Hongmei. Sonar image seabed topography restoration method under water depth data constraints. Journal of China University of Mining and Technology, 2017, 46(2): 443-448.).

[0004] In general, the current research on underwater 3D reconstruction based on 2D acoustic images is relatively scarce and scattered. In recent years, most of the research has focused on theory. In actual application, there are problems such as lack of universality and long post-processing time. Therefore, it is necessary to further study the underwater environment 3D reconstruction method based on 2D sonar images, increase its universality, reduce dependence on equipment performance, improve processing efficiency and reduce computing time. Summary of the invention

[0005] The technical problems to be solved by the present invention are:

[0006] In order to avoid the shortcomings of the prior art, the present invention provides a method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images, which is used to solve the problem that two-dimensional sonar images lack three-dimensional information.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images, characterized by comprising:

[0009] Acquire a sonar two-dimensional image, and divide the sonar two-dimensional image into multiple areas according to direction angles;

[0010] Construct an inversion model based on the Lambertian reflection model;

[0011] Perform inversion calculation on each divided area based on the inversion model to obtain height information, and splice and normalize the results of each area to obtain the normalized height information matrix of the sonar image corresponding to the detection area;

[0012] The normalized height information matrix is ​​added to the geometric size constraints of the sonar detection area to estimate the real height information in the detection area;

[0013] The estimated true height information within the sonar image detection area is converted into a three-dimensional point cloud.

[0014] A further technical solution of the present invention is as follows: the inversion model is:

[0015]

[0016] Among them, E(x,y) is the gray value of the point corresponding to a certain point underwater in the sonar image, I(x,y) is the reflection intensity of a certain point underwater represented by the Lambertian reflection model, and I n-1 (x,y) is the reflection intensity at the n-1th iteration.

[0017] A further technical solution of the present invention is: adding the normalized height information matrix to the geometric size constraint condition of the sonar detection area to estimate the real height information in the detection area is specifically as follows:

[0018] The maximum vertical distance of the sonar detection area is used as a geometric size constraint;

[0019] The actual height information in the detection area is estimated based on the geometric size constraint and the normalized height information matrix. A further technical solution of the present invention is: the maximum vertical distance of the sonar detection area is calculated by:

[0020] L=r·sinψ

[0021] Among them, L is the required constraint condition, ψ is the vertical opening angle of the sonar beam, and r is the maximum detection distance of the sonar platform along the track direction.

[0022] A further technical solution of the present invention is to estimate the real height information in the detection area based on the geometric size constraint condition and the normalized height information matrix, and the calculation formula used is as follows:

[0023]

[0024] Where ZH(x,y) is the real height information estimated in the detection area after adding geometric size constraints, x∈[0,m], y∈[0,n], Z max is the normalized height information matrix Z m×n The maximum absolute value in , m is the number of pixels of the sonar image on the x-axis, and n is the number of pixels of the sonar image on the y-axis.

[0025] A further technical solution of the present invention is as follows: the estimated real height information in the sonar image detection area is converted into a three-dimensional point cloud, specifically:

[0026] The estimated real height information in the sonar image detection area is matched with the pixel points on the sonar image to generate an m×n row and 3 column matrix. The contents of the 3 columns are the horizontal coordinate x, vertical coordinate y and the estimated real height value ZH(x,y) of the sonar image pixel point respectively; it is converted into a three-dimensional point cloud through a point cloud reading tool to realize the three-dimensional reconstruction and visualization of the underwater environment of the two-dimensional sonar image.

[0027] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0028] A computer-readable storage medium, characterized in that it stores computer-executable instructions, which are used to implement the above method when executed.

[0029] A computer program product, characterized by comprising computer executable instructions, wherein the instructions are used to implement the above method when executed.

[0030] The beneficial effects of the present invention are:

[0031] The present invention provides a method for three-dimensional reconstruction of underwater scenes based on the grayscale inversion height of two-dimensional sonar images. The method establishes an acoustic reflection echo model based on the relative spatial structure of the sonar platform and the underwater scene, and proposes a method for directly inverting the height information of the detection area under the spatial size constraint of the acoustic irradiation area from the grayscale of the two-dimensional sonar image, thereby obtaining the three-dimensional point cloud coordinates of the detection area to achieve three-dimensional reconstruction of the underwater scene. Compared with other methods for three-dimensional reconstruction of underwater scenes, this method only needs to use the grayscale information of the two-dimensional sonar image, and the calculation process is not affected by the acoustic detection angle and model of the sonar equipment, and is applicable to most acoustic images output by sonar. This method solves the problem of the lack of three-dimensional information in two-dimensional sonar images, provides three-dimensional information of the acoustic irradiation area for underwater scene detection, and has good application prospects. The advantages are as follows:

[0032] 1. The three-dimensional space of the underwater environment is reconstructed, providing complete three-dimensional information for unmanned underwater vehicles to avoid obstacles.

[0033] 2. The sonar image is divided into several areas, and the incident sound source in each area is approximated as a plane wave, which reduces the calculation time of each pixel and greatly improves the calculation speed.

[0034] 3. This method only needs to process two-dimensional sonar images, which reduces dependence on equipment and environment, improves data utilization, and has strong universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings are only for the purpose of illustrating specific embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0036] Figure 1 Schematic diagram of acoustic image division.

[0037] Figure 2 This is a schematic diagram of the sonar detection area size.

[0038] Figure 3 It is a schematic diagram of the process of the present invention.

[0039] Figure 4For the underwater forward-looking sonar images of Dalian Jinshitan (39.0904292°N, 122.0071952°E) and Maoming Lake (21.7011602°N, 110.8641811°E) areas, the sonar horizontal opening angle used is 120°, the vertical opening angle is 12°, and the maximum detection distance of the sonar along the track direction is 15m.

[0040] Figure 5 Estimate height values ​​for the three-dimensionally reconstructed region of the invention example.

[0041] Figure 6 The three-dimensional reconstructed point cloud image and the calculation time of the sonar image of the invention example: (a) three-dimensional reconstructed point cloud image; (b) three-dimensional reconstructed point cloud image; (c) three-dimensional reconstruction time. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] like Figure 3 As shown, an embodiment of the present invention provides a method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images, comprising the following steps:

[0044] Step 1: Divide the two-dimensional sonar image by horizontal beam angle;

[0045] The horizontal coordinate of the polar coordinate system image output by the sonar is the angle of the horizontal beam, that is, the direction angle of the incident sound line on the surface of the sonar irradiation area, and the vertical coordinate is the distance from the object to the sonar receiving end along the track direction. The sonar image is divided into several areas according to a certain angle range (such as Figure 1 The direction angle of each area is the average of all direction angles corresponding to the area ρ s Taking into account the accuracy and calculation speed, the directional angle is divided into 10° intervals. The surface incident sound line pitch angle of the sonar illumination area is uniformly used as the pitch angle of the sonar platform relative to the illumination area.

[0046] Step 2: Establish an inversion model based on the Lambertian reflection model to calculate the normalized height;

[0047] The types of acoustic reflection in underwater environments mainly include diffuse reflection and specular reflection. Sonar platforms usually receive reverse acoustic scattering from objects. When the normal vector of the object surface is opposite to the incident sound ray, the echo signal received by the sonar receiver is specular reflection. However, due to the presence of many floating objects in the water, diffuse reflection is the main type of acoustic reflection in underwater environments.

[0048] When diffuse reflection is the main reflection type, the Lambertian reflection model is usually used to represent the reflection intensity I(x,y) of a point underwater:

[0049] I(x,y)=S(x,y)·ρ(x,y)·cos(θ(x,y)) (1)

[0050] Among them, (x, y) is the coordinate of a certain point underwater in the sonar image, S(x, y) is the intensity of the incident sound ray, ρ(x, y) is the surface reflectivity of the point, and θ(x, y) is the angle between the incident direction of the sound ray at the point and the surface normal vector.

[0051] Since the grayscale of the sonar image is related to the reflection intensity, the reflection intensity can be represented by the grayscale of the sonar image:

[0052] I(x,y)=E(x,y) (2)

[0053] Among them, E(x,y) is the grayscale value of the point corresponding to a certain underwater point in the sonar image.

[0054] θ(x,y) can be expressed by the incident direction of the sound ray s(x,y) and the surface normal vector N(x,y) as

[0055]

[0056] The operator “·” represents dot product.

[0057] The incident sound ray is expressed as

[0058] s(x,y)=(I x ,I y ,I z ) (4)

[0059] Among them, ρ s is the direction angle of the incident sound is the pitch angle of the incident sound ray, then

[0060]

[0061] Assume |s(x,y)| = 1, then the direction of the sound source can be written as

[0062]

[0063] make

[0064] N(x,y)=(p,q,-1) (7)

[0065] Among them, p and q are the rates of change of the surface height of the object with respect to x and y, respectively, that is, the surface gradient can be expressed by finite differences:

[0066]

[0067]

[0068] Substituting equations (2) and (3) into equation (1), we can obtain

[0069]

[0070] According to the method proposed by PS Tsai and M Shah,

[0071]

[0072] Substituting equation (11) into equation (10) yields

[0073]

[0074] make

[0075]

[0076] Taylor first-order expansion is performed to obtain

[0077]

[0078] Among them, Z n-1 (x,y) is the 3D surface recovered in the n-1th iteration. Let Z n (x,y)=Z(x,y) and transpose, Z n Move (x,y) to the left of the equal sign, and we get

[0079]

[0080] in

[0081]

[0082] In summary, the iterative formula is

[0083]

[0084] Each area divided in step 1 is inverted and calculated through the above steps, the results of each area are spliced ​​together, and then normalized to obtain the normalized height information matrix Z of the sonar image corresponding to the detection area m×n, m is the number of pixels of the sonar image on the x-axis, and n is the number of pixels of the sonar image on the y-axis.

[0085] Step 3: Estimate the true height information within the detection area according to the geometric size constraints of the sonar detection area;

[0086] The detection area of ​​the multi-beam forward-looking sonar is an oblique fan-shaped column. The maximum vertical distance in the area is calculated as a constraint condition. The calculation method is:

[0087] L=r·sinψ (18)

[0088] Among them, L is the required constraint condition, ψ is the vertical opening angle of the sonar beam, and r is the maximum detection distance of the sonar platform along the track direction (see the schematic diagram Figure 2 ).

[0089] The normalized height information matrix Z is obtained from step 2 m×n , after adding geometric size constraints, the real height information ZH(x,y) in the detection area is estimated

[0090]

[0091] In the formula, x∈[0,m], y∈[0,n], Z max is the normalized height information matrix Z m×n The maximum absolute value in .

[0092] Step 4: Convert the estimated real height information within the sonar image detection area into a three-dimensional point cloud;

[0093] The real height information obtained in step 3 is matched with the pixel points on the sonar image to generate a matrix of m×n rows and 3 columns. The contents of the 3 columns are the horizontal coordinate x, the vertical coordinate y and the estimated real height value ZH(x,y) of the sonar image pixel point respectively; it is converted into a 3D point cloud through the point cloud reading tool to realize the 3D reconstruction and visualization of the underwater environment of the 2D sonar image.

[0094] In order to verify the feasibility of the method of the present invention and illustrate its characteristics, a simulation analysis was carried out. The flow chart is shown in Figure 3 As shown in Figure 1. The underwater forward-looking sonar image of Dalian Jinshitan (39.0904292°N, 122.0071952°E) was selected. The horizontal opening angle of the sonar used was 120°, the vertical opening angle was 12°, and the maximum detection distance of the sonar along the track direction was 15m. Figure 4 As shown, the sonar image has a size of 1024×1468; Figure 5 Estimate the height value of the 3D reconstruction area of ​​the sonar image (the pitch angle of the sonar platform is 20°); Figure 6This is the 3D reconstructed point cloud of the sonar image of the invention example and the running time (9.37s).

[0095] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images, characterized in that: include: Acquire a sonar two-dimensional image, and divide the sonar two-dimensional image into multiple areas according to direction angles; Construct an inversion model based on the Lambertian reflection model; Perform inversion calculation on each divided area based on the inversion model to obtain height information, and splice and normalize the results of each area to obtain the normalized height information matrix of the sonar image corresponding to the detection area; The normalized height information matrix is ​​added to the geometric size constraints of the sonar detection area to estimate the real height information in the detection area; The estimated true height information within the sonar image detection area is converted into a three-dimensional point cloud.

2. The method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images according to claim 1, characterized in that: The inversion model is: Among them, E(x,y) is the gray value of the point corresponding to a certain point underwater in the sonar image, I(x,y) is the reflection intensity of a certain point underwater represented by the Lambertian reflection model, and I n-1 (x,y) is the reflection intensity at the n-1th iteration.

3. The method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images according to claim 1, characterized in that: The normalized height information matrix is ​​added to the geometric size constraint condition of the sonar detection area to estimate the real height information in the detection area, specifically: The maximum vertical distance of the sonar detection area is used as a geometric size constraint; The true height information within the detection area is estimated based on geometric size constraints and the normalized height information matrix.

4. The method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images according to claim 3, characterized in that: The calculation method of the maximum vertical distance of the sonar detection area is: L=r·sinψ Among them, L is the required constraint condition, ψ is the vertical opening angle of the sonar beam, and r is the maximum detection distance of the sonar platform along the track direction.

5. The method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images according to claim 3, characterized in that: The actual height information in the detection area is estimated based on the geometric size constraints and the normalized height information matrix. The calculation formula used is as follows: Where ZH(x,y) is the real height information estimated in the detection area after adding geometric size constraints, x∈[0,m], y∈[0,n], Z max is the normalized height information matrix Z m×n The maximum absolute value in , m is the number of pixels of the sonar image on the x-axis, and n is the number of pixels of the sonar image on the y-axis.

6. The method for three-dimensional reconstruction of underwater scenes based on grayscale inversion height of two-dimensional sonar images according to claim 5, characterized in that: The method of converting the estimated real height information in the sonar image detection area into a three-dimensional point cloud is specifically as follows: The estimated real height information in the sonar image detection area is matched with the pixel points on the sonar image to generate an m×n row and 3 column matrix. The contents of the 3 columns are the horizontal coordinate x, vertical coordinate y and the estimated real height value ZH(x,y) of the sonar image pixel point respectively; it is converted into a three-dimensional point cloud through a point cloud reading tool to realize the three-dimensional reconstruction and visualization of the underwater environment of the two-dimensional sonar image.

7. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

8. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

9. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Citation Information

Cited By

  • Seabed sediment classification method and system based on side-scan sonar image

    CN121353890A

  • Three-dimensional reconstruction method, recording medium and system based on forward-scan sonar image

    CN121999167A