Three-dimensional reconstruction method based on forward-scan sonar images, recording medium and system

Abstract

The present application belongs to the technical field of underwater acoustic imaging and three-dimensional reconstruction, and particularly relates to a three-dimensional reconstruction method based on forward scanning sonar images, which obtains two-dimensional gray scale images collected by a sonar, uses semantic segmentation to analyze the features of the images, obtains light and shade feature channels of highlight areas and shadow areas, and extracts boundary features; extracts gray scale gradients and gray scale ratios in the highlight areas, solves obstacle height profile points by nonlinear fitting and taking obstacle front edge points and shadow boundary points as constraints, and finally realizes three-dimensional topography reconstruction by fusing multi-array features. The present application can effectively suppress underwater artifacts and noise interference, realize high-precision three-dimensional reconstruction, has the advantages of lightweight model, low power consumption, etc., and is suitable for covert detection in complex underwater environments and underwater surveying and mapping scenes. The present application also provides a non-transitory readable recording medium storing a program of the method and a system containing the medium, and the program can be called by a processing circuit to execute the above method.

Description

Technical Field

[0001] This invention belongs to the field of underwater acoustic imaging and three-dimensional reconstruction technology, and discloses a three-dimensional reconstruction method, recording medium and system based on front-scan sonar images. Background Technology

[0002] Underwater obstacle 3D topography reconstruction is a core technology in underwater surveying, autonomous underwater navigation, and underwater target detection, and has significant application value in marine engineering, underwater security, and covert detection scenarios. Front-scan and side-scan sonar, as primary devices for underwater environmental perception, rely on the principle of sound wave reflection to acquire 2D grayscale images of underwater scenes. They offer advantages such as being unaffected by water turbidity, having a long operating range, and a wide applicability, making them crucial technical means for achieving 3D perception of unstructured underwater environments.

[0003] Currently, most 3D reconstruction technologies based on sonar images rely on multi-sensor fusion and multi-viewpoint matching, which suffer from problems such as complex system structure, high computational overhead, and insufficient real-time performance. Meanwhile, the underwater environment is complex and variable, with factors such as inhomogeneous water media, differences in seabed materials, and interference from sound wave scattering and reverberation easily leading to artifacts, noise, and distortion in bright and dark areas in sonar images. Traditional 2D image processing methods struggle to effectively distinguish between bright reflection areas, shadow areas, and the boundaries of real obstacles, resulting in insufficient accuracy in 3D shape calculation and large deviations in contour fitting, failing to meet the requirements for high-precision underwater target reconstruction.

[0004] In practical engineering applications, underwater detection equipment typically requires low power consumption, small size, portability, and low electromagnetic signal emission to meet the needs of long-term underwater operations and covert detection. Existing sonar-based 3D reconstruction systems generally suffer from complex algorithm models, high computational load, and high hardware dependence, making it difficult to achieve lightweight deployment on embedded devices. At the same time, traditional neural network models have shortcomings such as insufficient feature utilization, inadequate fusion of shallow and deep information, and limited receptive field. In complex underwater environments, they are prone to feature loss, insufficient segmentation accuracy, and discontinuous reconstructed contours, further limiting the practical application of underwater 3D reconstruction technology in covert detection and low-power portable devices.

[0005] Therefore, in view of the problems of poor environmental adaptability, serious artifacts and noise interference, insufficient reconstruction accuracy, large model size, high power consumption and weak concealment of existing underwater sonar 3D reconstruction technology, there is an urgent need for a lightweight, high-precision, anti-interference-capable obstacle 3D shape reconstruction method based on sonar images that is suitable for embedded deployment, so as to meet the actual needs of high-precision, low-power and concealed 3D perception in complex underwater environments. Summary of the Invention

[0006] To address the above problems, this invention provides a three-dimensional reconstruction method based on front-scan sonar images, comprising the following steps: S1. Acquire a two-dimensional grayscale image from the ship's forward-scan sonar; S2. The two-dimensional grayscale image is analyzed by a semantic segmentation network to obtain the brightness and darkness feature channels including the bright area and the shadow area, and the corresponding boundary features are extracted. S3. Unsupervised extraction of grayscale gradient and grayscale ratio in the highlighted area; S4. Using the gray-level gradient, gray-level ratio, and physical characteristics of sonar reflection, nonlinear fitting is performed to solve the height profile points of the obstacle along the fore-and-aft direction of the ship. S5. By fusing the boundary features and height contour points of multiple array elements, the three-dimensional shape of obstacles can be reconstructed.

[0007] Preferably, the semantic segmentation model in step S2 is a convolutional neural network, based on the original U-net network architecture, with dilated convolutional kernels and residual connections introduced into its network modules, and a separate channel parallel structure is adopted.

[0008] Preferably, the method further includes the step of using dense connections within the network module and non-dense connections between network modules.

[0009] Preferably, it also includes a step of connecting different layers of the U-Net network architecture to perform feature fusion.

[0010] Preferably, the physical characteristics of sonar reflection include the obstacle distance parameters fed back by the sonar device at the same time for the same sonar element, with the obstacle leading edge point and shadow boundary point as constraints.

[0011] Preferably, the physical characteristics of sonar reflection include the two-dimensional grayscale image reflection features extracted by a semantic segmentation network at the same time for the same sonar element, with the obstacle leading edge point and shadow boundary point as constraints.

[0012] Another aspect of the present invention is to provide a non-transient readable recording medium for storing one or more programs containing multiple instructions, which, when executed, cause the processing circuit to perform the aforementioned three-dimensional reconstruction method based on front-scan sonar images.

[0013] Another aspect of the present invention provides a three-dimensional reconstruction system based on front-scan sonar images, including a processing circuit and a memory electrically coupled thereto. The memory is configured to store at least one program, the program containing multiple instructions. The processing circuit runs the program and can execute the aforementioned three-dimensional reconstruction method based on front-scan sonar images.

[0014] Compared to existing technologies, the present invention provides a 3D reconstruction method, recording medium, and system based on front-scan sonar images. Through the organic combination of a lightweight and innovative design of the network architecture and an unsupervised contour solving strategy, it effectively solves the technical pain points of existing technologies, such as poor environmental adaptability, low reconstruction accuracy, high model power consumption, and weak concealment. Compared to existing technologies, it has the following significant advantages: 1. Significantly improves the anti-interference capability and segmentation accuracy in complex underwater environments. This invention introduces dilated convolution, residual connections, and a parallel structure of separating channels into the U-Net architecture, employing a lightweight strategy of "dense connections within modules and non-dense connections between modules." On one hand, dilated convolution expands the receptive field without increasing the number of parameters, effectively capturing the global contextual relationships between bright and shadow regions in sonar images. On the other hand, residual connections solve the gradient vanishing problem in deep networks, while the parallel structure of separating channels enhances the independent representation of bright and dark features. Combined with multi-scale feature fusion, this invention can accurately suppress interference from underwater artifacts and reverberation noise, achieving accurate semantic segmentation of bright and shadow regions, laying a high-precision feature foundation for subsequent 3D reconstruction.

[0015] 2. Overcoming the bottleneck of high-precision contour calculation in unsupervised scenarios This invention overcomes the limitations of traditional methods that rely on manual annotation or multi-sensor fusion, proposing an unsupervised nonlinear fitting method based on gray-level gradients and gray-level ratios, with "obstacle leading edge points and shadow boundary points" as strong constraints. This design fully utilizes the physical imaging characteristics of sonar images to efficiently calculate the height contour points of obstacles along the ship's fore-and-aft direction. This combination of data-driven and physical constraints not only simplifies the system process but also solves the problems of discontinuous contour reconstruction and high distortion in unannotated scenarios, significantly improving the accuracy and robustness of 3D topography reconstruction.

[0016] 3. Achieve extreme lightweight design to meet low power consumption and embedded deployment requirements. For the application scenarios of portable underwater devices, this invention optimizes network connectivity (non-dense connections between modules) and feature reuse strategies, maximizing the compression of model parameters and computational load while ensuring performance. Compared to traditional heavy-duty deep learning models, the network structure of this invention is more streamlined, with faster inference speed, enabling real-time processing on low-computing-power embedded chips. This advantage directly reduces the device's energy consumption, extends underwater endurance, and provides core algorithmic support for the miniaturization and portability design of the device.

[0017] 4. Enhance electromagnetic concealment This invention, through precise perception and efficient computation at the algorithm level, enables the system to complete 3D reconstruction solely based on the sonar's own 2D grayscale image, without the need to emit complex auxiliary electromagnetic detection signals or rely on signal interaction from external base stations. This fundamentally reduces the outward radiation of electronic signals, lowers the risk of interception, identification, and location by electronic support measures (ESM) systems, and greatly enhances the equipment's stealth and battlefield survivability, making it particularly suitable for fields with stringent requirements for low observability.

[0018] 5. The system has strong compatibility and high engineering implementation value. The technical solution of this invention only requires upgrading the back-end algorithm of existing sonar equipment, without modifying the transmitting and receiving structure of the sonar hardware, resulting in low modification cost and strong compatibility. The proposed method can be seamlessly integrated into the perception system of unmanned surface vessels (USVs) and autonomous underwater vehicles (AUVs), providing high-precision and highly reliable three-dimensional data support for underwater obstacle avoidance, terrain mapping, and target recognition, and has broad engineering application prospects and extremely high promotional value. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the main process flow of the method in an embodiment of the present invention; Figure 2 A schematic diagram for calculating the height profile of obstacles; Figure 3 A schematic diagram of a separate-channel parallel structure is used to introduce dilated convolution kernels and residual connections; Figure 4 This is a schematic diagram of the experimental scenario in the embodiment; Figure 5 This is a comparison chart of the IoU and mIoU score curves of different models in the embodiments; Figure 6 This is a comparison chart showing the changing trends of training loss and validation loss for different models in the embodiments; Figure 7 This is a comparison diagram of ablation experiments in the examples; Figure 8 This is a flowchart of an algorithm for nonlinear fitting based on grayscale value ratios. Figure 9 This is a schematic diagram of the pitch angle distribution of the target to be reconstructed after nonlinear fitting in the embodiment. Figure 10 This is a schematic diagram illustrating the transformation relationship between the sonar coordinate system and the two-dimensional plane coordinate system. Figure 11 A schematic diagram of sonar coordinate system rotation; Figure 12 The height contour information map of the reconstructed target in the embodiment is displayed as a black and white image; Figure 13The image shows the point cloud distribution after nonlinear fitting of the reconstructed target in this embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the implementation methods of the present invention are described in detail below. These described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without innovative effort are within the scope of protection of the present invention.

[0021] An embodiment of a three-dimensional reconstruction method based on front-scan sonar images is as follows: like Figure 1 As shown, the method includes the following steps: S1: Perform semantic segmentation on the forward-looking sonar image, dividing it into target highlight areas and shadow areas; S2: Based on the semantic segmentation results, mark the leading edge points, occlusion contour points, and shadow trailing edge points of the 3D object; S3: Combine the underwater plane normal vector and calibration point information to calculate its pitch angle and obtain the pitch angle distribution map through nonlinear fitting; S4: Combine target information and pitch angle information to achieve three-dimensional reconstruction of the target through coordinate system mapping.

[0022] Further, step S1 includes the following steps: S11: Constructing a semantic segmentation network model: A forward-looking sonar image segmentation model, MCFU-Net (Multi-Scale Cross-Layer Fusion U-Net), based on the U-Net network, was constructed. A DRDS-Block (Diameter-Residual-Separation Channel Module) was designed as the backbone feature extraction network, and an MLFC (Multi-Layer Feature Concatenation) module was added to the skip connections. Figure 3 As shown, joint loss is used to calculate the loss, and the mIoU index is introduced to evaluate the semantic segmentation results.

[0023] Furthermore, step S11 includes the following steps: S111: Modify the backbone feature extraction network: To address the issue that the large number of parameters in the original U-Net network structure leads to slow training speed, and that the traditional convolutions used in it are difficult to simultaneously take into account features at different scales under the limitation of a fixed receptive field at a single scale, this invention replaces them with a lightweight DRDS-Block as the backbone extraction network. By setting different dilation coefficients for the depth-separable convolutions, the receptive field is expanded layer by layer, and finally, residual blocks are added to directly connect the input and output.

[0024] S112: Add a cross-level feature fusion module: Since the fusion between layers in the original U-Net network structure is mostly a layer-by-layer corresponding connection, lacking global feature aggregation across all layers, this invention adds an MLFC module to the skip connections. This module first adjusts the feature maps of different depths of the encoder to the same spatial size, and then concatenates them to construct a multi-scale feature stacking structure. Then, it reconstructs and generates global features through pointwise convolution, and then concatenates them with the original feature maps of the encoder at the corresponding scale. After fusion by depthwise separable convolution, the final output is a feature that fuses local details and cross-layer semantics.

[0025] S113: Calculate the loss using joint loss: To balance region overlap and pixel classification accuracy, a joint loss is employed, consisting of Dice Loss and CrossEntropy loss. The joint loss expression is as follows:

[0026] in, and The contributions of the two losses are used to balance the overall losses, and are empirically set to the same or similar values ​​to take into account the advantages of both losses.

[0027] Dice Loss measures the degree of overlap between the predicted region and the actual region. Its mathematical expression is:

[0028] in, This represents the set of pixels in the region predicted by the model. This represents the actual labeled region. Their intersection. This indicates the number of pixels predicted to be hit, while and These represent the total number of pixels in the predicted region and the real region, respectively.

[0029] The Cross Entropy part employs a variant of Focal Loss to enhance the model's attention to hard-to-classify samples, and its expression is as follows:

[0030] in, The modulation factor represents the model's predicted probability of the true class. This is used to suppress the loss contribution of easily classified samples, while Used to control the intensity of the modulation factor.

[0031] S114: Introducing an evaluation metric for mIoU segmentation results: Mean Intersection over Union (mIoU) is used as the segmentation performance evaluation metric to measure the segmentation accuracy of the model across different semantic categories. Its definition is as follows:

[0032] in, Indicates the number of categories. For the first The number of true instance pixels of a class, representing the number of pixels predicted to be consistent with the true class; A false positive indicates that the model incorrectly predicted the first positive. Pixels of the class; A false negative indicates that the model missed detecting pixels of that class.

[0033] S12: Creating a sonar image dataset: After the model was modified, sonar image data was collected. The experimental scenario was as follows. Figure 4 As shown, a sonar image dataset was created and processed accordingly. The purpose of the sonar image segmentation is to separate the bright target regions formed by active sound wave echoes and the shadow regions formed by object occlusion, without distinguishing the target type represented by each region. Therefore, in the segmentation task, the dataset is labeled with only two types: bright target regions and shadow regions. After experiments, 2070 forward-looking sonar images were collected and organized into a training set, a validation set of 230 images, and a test set of 230 images.

[0034] S13: Experiment Setup: The model of this invention was trained and tested on an NVIDIA GeForce RTX 3080Ti graphics card using the PyTorch deep learning framework. Python was used as the programming language to set parameters such as epochs, batch size, image size, initial learning rate, optimizer, and learning strategy, thereby completing the construction and setup of the overall model.

[0035] S14: Set up a comparative experiment to systematically compare and analyze the semantic segmentation model of this invention with existing mainstream methods, and verify the performance of the model: The training and validation sets are fed into the network to train different semantic segmentation models. The log files with the best training results from each model are selected to perform a multi-faceted evaluation of the semantic segmentation models. The IoU score curves and mIoU score curves for each category of different models are shown below. Figure 5 As shown, the semantic segmentation model of this invention achieves higher IoU and mIoU scores across all categories than the other comparative models, further validating its superior segmentation performance. The trends in training loss and validation loss for different models are shown below. Figure 6 As shown, the model of this invention exhibits excellent optimization characteristics and generalization ability in both the training and validation loss curves.

[0036] S15: Set up ablation experiments to verify the impact of each module on the semantic segmentation performance of the model: Four sets of comparative models were constructed using the controlled variable method: U-Net, U-Net backbone + MLFC, U-Net backbone + DRDS-Block, and U-Net backbone + MLFC + DRDS-Block. The training and validation sets were fed into the network to train the four sets of comparative models, and their respective best mIou segmentation performance and GFLOPs computational complexity were recorded. The results are as follows: Figure 7 As shown, the model of the present invention achieves a reasonable balance between performance improvement and computational overhead, demonstrating that the two modules have good functional complementarity, and overall verifying the rationality of the structural design and the effectiveness of the semantic segmentation model structural design of the present invention.

[0037] S16: Using the trained semantic segmentation model, the sonar image of the target to be reconstructed is segmented into highlight and shadow regions to obtain the final image segmentation result: The weight file in the test code was modified to use the weights that best represented the semantic segmentation model trained according to this invention. The sonar image of the target to be reconstructed was then segmented into highlight and shadow regions to obtain the final segmentation result. After segmentation, the original sonar image was converted into a mask containing only three pixel values, representing the target highlight region, shadow region, and background region, respectively.

[0038] Further, step S2 includes the following steps: S21: In each column of the shaded area, mark the shaded pixel closest to the top of the image as the shadow trailing edge point of that column; S22: Continue searching for the target region in the same column from the shadow trailing edge point in the closer direction. If a pixel belonging to the target region is found, mark the target pixel closest to the shadow region as the target trailing edge point; S23: Search for the connected target region from the target trailing edge point in the near direction, and select the target pixel farthest from the shadow as the target leading edge point of the column; S24: If a column detects three key points simultaneously—the shadow trailing edge, the target trailing edge, and the target leading edge—it is included in the valid data set for subsequent elevation angle interpolation and 3D reconstruction; if any key point is missing, the column is discarded.

[0039] Further, step S3 includes the following steps: S31: Calculate the normal vector of the bottom plane; S32: The leading edge and trailing edge of the 3D object are both located on the bottom plane. Calculate the pitch angle of these two points by combining the normal vector of the bottom plane. S33: According to the forward-looking sonar imaging principle, the pitch angle of the occlusion contour point of a three-dimensional object is the same as that of the shadow trailing edge point. S34: Combining the pitch angle information of the leading edge point and occluded contour point of the three-dimensional object, a nonlinear fitting method is used to obtain the pitch angle distribution map of the front surface of the target.

[0040] Furthermore, step S31 includes the following steps: S311: Measuring the height of the sonar above the seabed and sonar glancing angle ; S312: Calculate the normal vector of the bottom plane And adjust the model to .

[0041] Furthermore, step S32 includes the following steps: S321: Extract the slant distances corresponding to the leading edge point and the trailing edge point of a 3D object. and azimuth ; S322: Combine the underwater plane normal vector to calculate the pitch angle. :

[0042] in, , and These represent the normal vectors of the underwater plane. exist , and Components in three dimensions.

[0043] Furthermore, step S34 includes the following steps: S341: Based on the grayscale information of the target area in the sonar image, initialize the estimated elevation angle of the midpoint of the target's front surface:

[0044] in, For the row number in this column The initial pitch angle estimate of the target point, For the row number in this column The grayscale value of the target point. The grayscale value of the leading edge point of the target in this column. This refers to the grayscale value of the trailing edge point of the target in this column. The pitch angle is formed between the line connecting the target front point and the acoustic center of the sonar array element and the central axis (Y-axis) of the sonar array element; The pitch angle formed between the line connecting the trailing edge of the target and the acoustic center of the sonar array element and the Y-axis (e.g., Figure 2 (as shown) S342: Starting with the maximum number of intermediate points in each column, prioritize using more intermediate points that meet the constraints for nonlinear fitting. If none are found, reduce the number of points and reselect. If no result is found when the number of points is 1, discard the column. S343: Enumerate all qualified intermediate point combinations, calculate the pitch angle of the remaining intermediate points using nonlinear fitting, and determine whether the constraint conditions are met. S344: Select the combination with the smallest fitting error under the current number of points as the optimal fitting result for this column, and stop searching for a smaller number of points.

[0045] The flowchart of the nonlinear fitting algorithm based on grayscale ratio is as follows: Figure 8 As shown. The nonlinear fitting result of the pitch angle of the target to be reconstructed is as follows. Figure 9 As shown, the elevation angle distribution exhibits a more detailed and undulating trend, retaining more features reflecting the undulations of the target surface in the elevation angle curve.

[0046] Furthermore, step S4 includes the following steps: S41: Establish the mapping relationship between the sonar coordinate system and the two-dimensional plane coordinate system: a point in space Projected onto the corresponding point in the forward-looking sonar imaging plane The pitch angle was lost during the projection process. The transformation relationship between the sonar coordinate system and the two-dimensional plane coordinate system is as follows: Figure 10 As shown, the formula is as follows:

[0047] in, It is the azimuth angle.

[0048] S42: Establish the mapping relationship between the underwater coordinate system and the sonar coordinate system: Converting the sonar coordinate system to the underwater coordinate system requires rotation and translation operations.

[0049] The original sonar coordinate system plane around Axis rotation Obtain the rotated sonar coordinate system ,like Figure 11 As shown. Rotated sonar coordinate system. With the original sonar coordinate system The conversion relationship is as follows:

[0050] in, It is the sonar sweep angle.

[0051] Rotated sonar coordinate system translate downwards Then the underwater coordinate system can be obtained. :

[0052] in, It is the height of the sonar above the seabed.

[0053] S43: Establish the mapping relationship between the underwater coordinate system and the two-dimensional plane coordinate system: Sonar coordinate system The underwater coordinate system is obtained after rotation and translation. The combined transformation formula for rotation and translation is:

[0054] The formula for calculating the target height is as follows:

[0055] S44: Combine target information and pitch angle information to reconstruct target height contour information and 3D front surface features. The reconstructed target height contour information is as follows: Figure 12 As shown, the reconstructed contours have a certain degree of similarity to the actual target's shape in overall form, and can basically reflect the target's contour height structure. Furthermore, the reconstructed heights are all within a reasonable range, and can reflect the actual height of the target relatively closely. The point cloud image of the reconstructed target is shown below. Figure 13 As shown, the reconstruction using this method realistically restores the basic outline of the target, and the target surface exhibits a more pronounced convex trend, with details of local shape changes also being well represented. This is consistent with the structural characteristics of actual target surfaces, which have a certain degree of roughness and local convexities, and matches the geometry of the actual target.

[0056] Assembling the above methods and steps into a program and storing it on a hard disk or other non-transitory storage medium constitutes an embodiment of the present invention's "a non-transitory readable recording medium"; while electrically connecting the storage medium to a computer processor and completing three-dimensional reconstruction based on front-scan sonar images through data processing constitutes an embodiment of the present invention's "a three-dimensional reconstruction system based on front-scan sonar images".

[0057] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computers or available storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional reconstruction method based on front-scan sonar images, characterized in that, Includes the following steps: S1. Acquire a two-dimensional grayscale image from the ship's forward-scan sonar; S2. The two-dimensional grayscale image is analyzed by a semantic segmentation network to obtain the brightness and darkness feature channels including the bright area and the shadow area, and the corresponding boundary features are extracted. S3. Unsupervised extraction of grayscale gradient and grayscale ratio in the highlighted area; The gray-level gradient reflects the change in gray-level values ​​of adjacent pixels in the same image, and the gray-level ratio is the ratio of two gray-level values. Together, they reflect the gray-level change information in the image. S4. Using the gray-level gradient, gray-level ratio, and physical characteristics of sonar reflection, nonlinear fitting is performed to solve the height profile points of the obstacle along the fore-and-aft direction of the ship. Specifically, this includes: measuring the height of the sonar above the seabed. and sonar glancing angle ; Calculate the normal vector of the bottom plane And adjust the model to , Based on this, extract the slant distances corresponding to the leading edge point and the trailing edge point of the 3D object. and azimuth ; Calculate the pitch angle by combining the normal vector of the bottom plane. : ;in, , and Represent the normal vectors of the underwater plane, respectively. exist , and The pitch angle of a 3D object is calculated using components in three dimensions, including the leading edge point and the trailing edge point of its shadow. Based on the grayscale variation information of the target area in the sonar image, initialize the estimated elevation angle of the midpoint of the target's front surface: ;in, The row number of the pixel column located in the target shadow area is The initial pitch angle estimate of the target point, For the row number in this column The grayscale value of the target point. This represents the grayscale value of the leading edge point of the target in this column. This refers to the grayscale value of the trailing edge point of the target in this column. The elevation angle formed between the line connecting the target front point and the acoustic center of the sonar array element and the central axis of the sonar array element; The pitch angle formed between the line connecting the trailing edge of the target and the acoustic center of the sonar array element and the central axis of the sonar array element; Starting with the maximum number of intermediate points in each column, prioritize using more intermediate points that meet the constraints for nonlinear fitting. If none are found, reduce the number of points and reselect. If no results are found when the number of points is 1, discard the column. Enumerate all qualified intermediate point combinations, calculate the pitch angle of the remaining intermediate points using nonlinear fitting, and determine whether the constraints are met. Select the combination with the smallest fitting error under the current number of points as the optimal fitting result for that column, and stop searching for a smaller number of points to obtain the height profile points of the obstacle along the front and rear directions of the ship. S5. By fusing the boundary features and height contour points of multiple array elements, the three-dimensional shape of obstacles can be reconstructed; The semantic segmentation model in step S2 is a convolutional neural network, based on the original U-net network architecture. It introduces dilated convolutional kernels and residual connections into the network modules and adopts a separate channel parallel structure.

2. The three-dimensional reconstruction method based on front-scan sonar images according to claim 1, characterized in that, It also includes the steps of using dense connections within the network module and non-dense connections between network modules.

3. The three-dimensional reconstruction method based on front-scan sonar images according to claim 2, characterized in that, It also includes the step of connecting different layers of the U-Net network architecture to perform feature fusion.

4. The three-dimensional reconstruction method based on front-scan sonar images according to claim 3, characterized in that, The physical characteristics of sonar reflection include the obstacle distance parameters fed back by the sonar device at the same time for the same sonar element, with the obstacle's leading edge point and shadow boundary point as constraints.

5. The three-dimensional reconstruction method based on front-scan sonar images according to claim 4, characterized in that, The physical characteristics of sonar reflection include the two-dimensional grayscale image reflection features extracted by a semantic segmentation network at the same time for the same sonar element, with the obstacle leading edge point and shadow boundary point as constraints.

6. A non-transitory readable recording medium for storing one or more programs containing multiple instructions, characterized in that, When the instruction is executed, the processing circuit will perform a three-dimensional reconstruction method based on a front-scan sonar image as described in any one of claims 1-5.

7. A three-dimensional reconstruction system based on front-scan sonar images, comprising processing circuitry and a memory electrically coupled thereto, characterized in that, The memory is configured to store at least one program, the program containing multiple instructions, and the processing circuit runs the program to execute the three-dimensional reconstruction method based on front-scan sonar images according to any one of claims 1-5.

Images