Non-uniform scanning-oriented front-looking sonar image three-dimensional reconstruction method and device

Abstract

This application relates to a method and apparatus for 3D reconstruction of forward-looking sonar images for non-uniform scanning. The method includes: acquiring multiple frames of forward-looking sonar images and sensor pose information; establishing a coordinate mapping between the sonar observation coordinate system and the 3D reconstruction space; obtaining initial occupancy results and generating a reliable envelope region through voxel statistics at a first scale; dividing stable occupancy and voxels to be verified in a high-resolution second-scale voxel space; calculating verification evaluation values ​​based on multi-dimensional indicators and selecting voxels; and extracting the 3D surface after fusion. This method can adapt to underwater non-uniform scanning scenarios, suppress noise and false structures, retain real weakly observed structures that are easily deleted, and improve the integrity, continuity, and noise resistance of the 3D reconstruction model.

Description

Technical Field

[0001] This application relates to the field of forward-looking sonar image processing and 3D reconstruction technology, and in particular to a method and apparatus for 3D reconstruction of forward-looking sonar images for non-uniform scanning. Background Technology

[0002] Forward-looking sonar can acquire target echo images in turbid, low-light, or no-light underwater environments, and is therefore widely used in tasks such as close-range detection, structural inspection, obstacle identification, and underwater target modeling for underwater robots. Compared with optical cameras, forward-looking sonar is less dependent on water visibility, making it suitable for continuous operation in complex underwater environments. However, forward-looking sonar images typically use distance and orientation as the main imaging dimensions, and a single frame image is insufficient to directly determine the complete geometric position of the target in three-dimensional space. Therefore, when performing underwater 3D reconstruction, it is usually necessary to combine multiple frames of sonar images and their corresponding sensor poses to unify acoustic observations from different perspectives into a single three-dimensional space for fusion.

[0003] Existing forward-looking sonar 3D reconstruction methods typically employ multi-frame evidence accumulation, spatial voxel updates, implicit surface modeling, or differentiable geometric constraints to recover the target structure. Among these, voxel-based reconstruction methods have a relatively straightforward computational process, capable of mapping sonar observations to 3D space for occupancy determination; learning- or optimization-based methods estimate the target surface through continuous field functions or differentiable projection processes. However, most of these methods assume that the input observations have sufficient and balanced view coverage, or rely on relatively stable cross-frame geometric constraints. When the sonar acquisition trajectory is irregular, there is insufficient repeated observation in local areas, or there are significant differences in observation density across different regions, the reconstruction stability of existing methods will decrease significantly.

[0004] In actual underwater operations, the movement of underwater robots or platforms is often affected by factors such as operating space, obstacle avoidance requirements, the range of motion of the robotic arm, water flow disturbance, and target occlusion, making it difficult to uniformly scan the target along an ideal trajectory. The resulting forward-looking sonar image sequences often suffer from problems such as sparse local viewpoints, uneven spatial coverage, and significant differences in echo support strength. For voxel update methods that rely solely on a unified discrimination standard, regions that are actually part of the target structure but have been observed less frequently are easily misclassified as non-target regions; while simply lowering the discrimination criteria may introduce a large number of false structures caused by noise, sidelobe echoes, or environmental reflections.

[0005] Therefore, under non-uniform sonar scanning conditions, existing technologies are prone to problems such as local target loss, interruption of slender structures, fragmentation of boundary regions, and residual pseudo-structures, making it difficult to balance structural integrity and noise suppression. How to preserve weak echo regions with actual structural significance while avoiding unconstrained expansion of reconstruction results under uneven observation distribution is a pressing technical problem that needs to be solved in forward-looking sonar 3D reconstruction. Summary of the Invention

[0006] Based on this, it is necessary to provide a method and apparatus for three-dimensional reconstruction of forward-looking sonar images oriented towards non-uniform scanning, which can perform scale fusion and local structural verification on multiple frames of forward-looking sonar images under conditions of uneven distribution of sonar observation viewpoints and insufficient echo support in local areas, thereby improving the structural integrity, spatial continuity and noise resistance stability of the reconstruction results.

[0007] A method for 3D reconstruction of forward-looking sonar images for non-uniform scanning, the method comprising: Acquire multiple frames of forward-looking sonar images and the sensor pose information corresponding to each frame. Based on the multiple frames of forward-looking sonar images and the sensor pose information corresponding to each frame, establish the coordinate mapping relationship from the sonar observation coordinate system to the three-dimensional reconstruction space. Based on the coordinate mapping relationship, the observable state and acoustic echo response of each voxel in the multi-frame forward-looking sonar image are statistically fused in the first scale voxel space to obtain the initial occupancy result characterizing the spatial distribution of the target subject. The initial occupancy result is then subjected to scale mapping and spatial neighborhood expansion processing to generate a reliable envelope region. In the second-scale voxel space with a spatial resolution higher than that of the first-scale voxel space, the cross-frame acoustic response evidence of each voxel is re-statistically analyzed, and the voxels are divided into stable occupied voxels and voxels to be verified according to the preset discrimination rules. Based on the credible envelope region, the spatial correlation between the voxel to be verified and the stable occupied voxel, and the supplementary connection properties of the voxel to be verified to the local fracture structure, the verification evaluation value of the voxel to be verified is calculated. Based on the verification evaluation value, voxels to be verified are selected. The selected voxels to be verified are spatially fused with stable occupying voxels to obtain the target occupying voxel set. Based on the target occupying voxel set, the three-dimensional surface is extracted to complete the three-dimensional reconstruction of the forward-looking sonar image.

[0008] A three-dimensional reconstruction device for forward-looking sonar images oriented to non-uniform scanning, the device comprising: The data acquisition module is used to acquire multiple frames of forward-looking sonar images and the sensor pose information corresponding to each frame. Based on the multiple frames of forward-looking sonar images and the sensor pose information corresponding to each frame, a coordinate mapping relationship is established from the sonar observation coordinate system to the three-dimensional reconstruction space. The coarse-scale occupancy construction module is used to statistically fuse the observable state and acoustic echo response of each voxel in the multi-frame forward-looking sonar image based on the coordinate mapping relationship, in the first-scale voxel space, to obtain the initial occupancy result characterizing the spatial distribution of the target subject. The trusted envelope generation module is used to perform scale mapping and spatial neighborhood expansion on the initial occupancy result to generate a trusted envelope region; The fine-scale voxel discrimination module is used to re-statistically analyze the cross-frame acoustic response evidence of each voxel in the second-scale voxel space with a spatial resolution higher than that of the first-scale voxel space, and classify the voxels into stable occupied voxels and voxels to be verified according to the preset discrimination rules. The local verification scoring module is used to calculate the verification evaluation value of the voxel to be verified based on the credible envelope region, the spatial correlation between the voxel to be verified and the stable occupied voxel, and the supplementary connection attribute of the voxel to be verified to the local fracture structure. The 3D reconstruction output module is used to screen voxels to be screened based on the verification evaluation value, spatially fuse the screened voxels to be screened with stable occupying voxels to obtain the target occupying voxel set, extract the 3D surface based on the target occupying voxel set, and complete the 3D reconstruction of the forward-looking sonar image.

[0009] The aforementioned method and apparatus for 3D reconstruction of forward-looking sonar images under non-uniform scanning first obtains the main spatial distribution of the target through a first-scale voxel space and further forms a reliable envelope region, providing spatial constraints for fine structure recovery at the second scale. Unlike methods that directly output reconstruction results based solely on uniform discrimination conditions, this application directly identifies voxels with stable acoustic responses as reliable structures during the second-scale reconstruction process. Simultaneously, it performs local verification on voxels that do not meet the stable discrimination conditions but still possess effective echo support. This verification process comprehensively considers the consistency between the voxel to be verified and the reliable envelope region, its spatial proximity to the already determined occupied structure, and its supplementary connection effect on local structural breaks. This allows for the suppression of noise voxels and false structures while preserving real structural regions that are easily mistakenly deleted under non-uniform scanning conditions. In summary, this application can improve the problems of structural loss, boundary fragmentation, and slender region breaks caused by uneven viewpoint coverage, sparse local observations, and differences in echo support in forward-looking sonar 3D reconstruction. It improves the integrity and continuity of the 3D model of underwater targets or scenes and maintains high computational efficiency while ensuring reconstruction quality. Attached Figure Description

[0010] Figure 1 This is a rough flowchart illustrating a method for three-dimensional reconstruction of forward-looking sonar images for non-uniform scanning in one embodiment. Figure 2 This is a detailed flowchart illustrating a method for three-dimensional reconstruction of forward-looking sonar images for non-uniform scanning in one embodiment. Figure 3 This is a structural block diagram of a three-dimensional reconstruction device for forward-looking sonar images oriented towards non-uniform scanning, as shown in one embodiment. Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] In one embodiment, such as Figure 1 and Figure 2 As shown, a method for 3D reconstruction of forward-looking sonar images for non-uniform scanning is provided, including the following steps: Step 102: Obtain multiple frames of forward-looking sonar images and the sensor pose information corresponding to each frame. Based on the multiple frames of forward-looking sonar images and the sensor pose information corresponding to each frame, establish the coordinate mapping relationship from the sonar observation coordinate system to the three-dimensional reconstruction space.

[0013] The multi-frame forward-looking sonar images are a sequence of echo images acquired during underwater non-uniform scanning operations. The sensor pose information includes the rotation matrix and translation vector of the sonar sensor at the time of acquisition of each frame. The sonar observation coordinate system is a local observation coordinate system with the sonar probe as the origin, and the three-dimensional reconstruction space is a unified underwater world coordinate system. Establishing a coordinate mapping relationship can accurately project the two-dimensional sonar range-azimuth observation data into three-dimensional space, providing a coordinate basis for subsequent voxel statistics and occupancy judgment, and fundamentally solving the problem that a single-frame forward-looking sonar cannot determine the three-dimensional geometric position of the target.

[0014] Step 104: Based on the coordinate mapping relationship, statistical fusion is performed on the observable state and acoustic echo response of each voxel in the multi-frame forward-looking sonar image within the first scale voxel space to obtain the initial occupancy result characterizing the spatial distribution of the target subject. The initial occupancy result is then subjected to scale mapping and spatial neighborhood expansion processing to generate a reliable envelope region.

[0015] The first-scale voxel space is a low-resolution three-dimensional voxel grid used to quickly characterize the spatial distribution of the target subject. Statistical fusion is to accumulate and calculate the effective observation state and acoustic echo response of each voxel in multiple frames of images to obtain the initial occupancy result. Scale mapping is to transform the low-resolution initial occupancy result to the high-resolution second-scale voxel space. Spatial neighborhood expansion is to expand the mapped voxel set according to a preset radius. The credible envelope region can limit the effective spatial range of subsequent fine reconstruction, block the unconstrained diffusion of false voxels caused by noise, sidelobe echoes, and environmental reflections, and provide structural prior spatial constraints for the second-scale fine reconstruction.

[0016] Step 106: In the second-scale voxel space with a spatial resolution higher than that of the first-scale voxel space, re-statistically analyze the cross-frame acoustic response evidence of each voxel, and divide the voxels into stable occupied voxels and voxels to be verified according to the preset discrimination rules.

[0017] The second-scale voxel space is a high-resolution voxel grid with a higher spatial resolution than the first-scale voxel space, used to reconstruct the fine local structure of the target; cross-frame acoustic response evidence is the cumulative result of the number of effective observations and echo support of voxels in multiple frames of sonar images; the preset discrimination rules include stable occupancy discrimination conditions and voxel discrimination conditions to be verified; dividing voxels into stable occupancy voxels and voxels to be verified can preserve the stable and reliable main structure of the target, while retaining the potential real structure with weak observations but effective echo support, avoiding the problem of false deletion of real structures caused by uniform discrimination standards.

[0018] Step 108: Calculate the verification evaluation value of the voxel to be verified based on the credible envelope region, the spatial correlation between the voxel to be verified and the stable occupied voxel, and the supplementary connection attribute of the voxel to be verified to the local fracture structure.

[0019] Spatial correlation refers to the spatial distance between the voxel to be verified and the voxel that has been identified as a stable occupier. Supplementary connectivity attributes refer to the bridging and repair effect of the voxel to be verified on local fractures and slender structural interruption areas. The verification evaluation value is obtained by weighted calculation of four indicators: acoustic response credibility, envelope consistency, spatial proximity, and local connectivity. This can quantify the structural value of the voxel to be verified and accurately screen weakly observed voxels with actual structural significance.

[0020] Step 110: Based on the verification evaluation value, select voxels to be verified, and spatially fuse the selected voxels to be verified with stable occupying voxels to obtain a target occupying voxel set. Based on the target occupying voxel set, extract the three-dimensional surface to complete the three-dimensional reconstruction of the forward-looking sonar image.

[0021] Voxels with high structural value are selected based on the verification evaluation value. Spatial fusion merges the selected voxels with stable occupying voxels into a complete target occupying voxel set. Based on the target occupying voxel set, the three-dimensional surface is extracted, which can restore the complete three-dimensional geometric model of the target. Finally, the structural integrity, continuity and noise resistance of the forward-looking sonar three-dimensional reconstruction under non-uniform scanning conditions are realized.

[0022] In the aforementioned method for 3D reconstruction of forward-looking sonar images for non-uniform scanning, multiple frames of forward-looking sonar images and corresponding sensor pose information are first acquired. A correspondence between sonar observation data and the 3D reconstruction space is then established based on the sensor pose information. Subsequently, in the first-scale voxel space, the effective observation state and echo support state of each voxel unit in the multiple frames of sonar images are statistically analyzed to obtain an initial occupancy result that reflects the main spatial distribution of the target. To avoid subsequent fine reconstruction relying entirely on weak local observation information, the initial occupancy result undergoes scale mapping and spatial neighborhood expansion to form a reliable envelope region. In the second-scale voxel space, cross-frame acoustic response statistics are re-performed for each voxel unit, and stable occupancy voxels are determined according to preset discrimination conditions. Voxels that do not meet the stable occupancy conditions but still have effective echo support are not discarded directly but are used as voxels to be verified. Then, combining the reliable envelope region, the spatial relationship between the voxels to be verified and the determined occupancy structure, and the supplementary connection effect of the voxels to be verified on local broken regions, the verification evaluation value of the voxels to be verified is calculated. Finally, within the local voxel group corresponding to the sonar range-azimuth observation unit, the voxels that meet the retention conditions are selected according to the verification evaluation value, and they are fused with the stable occupying voxels to obtain the target occupying voxel set; based on the target occupying voxel set, a three-dimensional surface model is extracted to obtain the three-dimensional reconstruction result corresponding to the forward-looking sonar image sequence.

[0023] This embodiment does not simply apply a uniform discrimination condition to all voxels. Instead, after initially obtaining the spatial distribution of the target, a local verification mechanism is introduced to re-evaluate voxels that are weakly observed but structurally significant. This avoids the accidental deletion of true structures due to uneven sonar scanning coverage and suppresses the diffusion of false voxels caused by noise echoes or environmental reflections, thereby improving the completeness and continuity of the reconstruction results.

[0024] In the second-scale processing, this application does not directly delete all weak-response voxels based on a unified discrimination condition. Instead, it first identifies voxel units with stable acoustic responses, and then performs a secondary verification on voxel units that still retain some echo support but do not meet the stability discrimination condition. Specifically, in the second-scale voxel space, the effective observation state and echo support state of each voxel unit in multi-frame sonar images are re-statistically analyzed, and stable occupied voxels are determined according to preset stability discrimination conditions. For voxel units that do not meet the stability discrimination condition but still have effective echo support, they are classified as voxels to be verified. Subsequently, verification evaluation values ​​are calculated based on the credible envelope region where the voxel to be verified is located, its spatial proximity relationship with the determined occupied structure, and its supplementary connection effect on the discontinuous region of the local structure. Finally, within the local voxel group corresponding to the sonar range-azimuth observation unit, the voxels to be verified that meet the retention conditions are selected and fused with the stable occupied voxels to form a target occupied voxel set, thereby extracting the three-dimensional surface model.

[0025] In one embodiment, the observable state and acoustic echo response of each voxel in multiple frames of forward-looking sonar images are statistically fused within a first-scale voxel space to obtain an initial occupancy result characterizing the spatial distribution of the target subject, including: The underwater area to be reconstructed is divided into three-dimensional voxel units of the first scale, and the center position of each voxel unit in the world coordinate system is determined. Based on the sensor pose information, the center position of the voxel unit is transformed to the corresponding sonar coordinate system and projected onto the range-azimuth observation plane of the sonar frame to obtain the range index and azimuth index. Determine whether the distance index and azimuth index are within the effective imaging range of the sonar, and mark the voxel units within the effective imaging range as effective observation voxel units; Based on the difference between the echo intensity of the image position corresponding to the effective observed voxel unit and the background response level of the current frame, it is determined whether occupancy support is obtained, and the number of effective observations and the number of occupancy support are accumulated. The first-scale confidence index is calculated based on the number of valid observations and the number of occupied support, and voxel units are selected according to the preset confidence conditions to form the initial occupation results.

[0026] Specifically, the underwater region to be reconstructed is divided into multiple first-scale voxel units. Let the... The position of the center point of the individual element in the unified reconstructed coordinate system is: , No. The sensor attitude corresponding to the frame-forward sonar is determined by the rotation matrix. Translation vector This indicates that the center point of the voxel unit is at the [missing information]. The relative coordinates in the local coordinate system of a frame sonar can be expressed as: ; in, ; Based on the relative coordinates, calculate the distance and azimuth coordinates of the voxel unit in the sonar observation plane: ; Furthermore, based on the range sampling interval and azimuth sampling interval of the sonar images, continuous range and azimuth coordinates are converted into corresponding image indices: ; ; in, This indicates the minimum effective detection range of the sonar. Indicates the sampling interval in the distance direction. This represents the minimum azimuth angle of the sonar. The sampling interval indicates the azimuth direction. Indicates azimuth coordinates.

[0027] When the image index is within the effective imaging range of the sonar, the voxel unit is determined to have a valid observation in the current frame. Its valid observation flag can be represented as: ; in, and These represent the number of samples in the range and azimuth directions of the sonar image, respectively.

[0028] For a voxel cell with valid observations, it is determined whether the voxel cell receives echo support based on the difference between the echo intensity at the corresponding image location and the background response level of the current frame. Let the voxel cell be... Frame sonar images in index The echo intensity at that location is The estimated background response value is The estimated value of background fluctuation is Then the echo support marker can be represented as: ; in, This is the preset background suppression coefficient.

[0029] Furthermore, the cumulative number of each Effective observation count and echo support count of individual units in multi-frame sonar images: ; ; To reduce the impact of statistical fluctuations caused by too few observations, the first-scale reliability index is constructed as follows: ; in, and This is a smoothing parameter used to adjust the stability of the confidence estimate under low observation conditions. For the first The number of echo support counts for an individual voxel in multi-frame sonar images. For the first The effective number of observations for an individual pixel in multiple frames of sonar images. The first-scale confidence index adjusts the stability of confidence estimation under low observation counts through a smoothing parameter, avoiding statistical fluctuations caused by too few observations, ensuring the accuracy of the initial occupancy results, and adapting to the characteristics of sparse local observations under non-uniform scanning.

[0030] If the first scale confidence index meets the preset conditions, then the corresponding voxel unit is added to the initial occupancy result: ; in, The first-scale credibility threshold, This represents the minimum number of effective observations.

[0031] Through the entire process of voxel division, coordinate transformation, effective observation judgment, occupancy support judgment, and credibility screening, the first-scale coarse occupancy construction is completed, which can quickly lock the spatial range of the target subject, eliminate a large number of invalid spaces, reduce the computational load of subsequent fine reconstruction, and provide basic data for the generation of reliable envelope.

[0032] In one embodiment, scale mapping and spatial neighborhood expansion are performed on the initial occupancy result to generate a reliable envelope region, including: The initial occupancy result is mapped to the second-scale voxel space, and the mapped voxel set is expanded spatially according to a preset neighborhood radius to obtain a reliable envelope region.

[0033] Specifically, the initial occupation result The data is mapped to a second-scale voxel space, and the spatial neighborhood of the mapped voxel set is expanded. Let the index set of the second-scale voxel units be... The set of voxels obtained by mapping the results from the first scale is: Then the reliable envelope region can be represented as: ; in, Represents the neighborhood distance in the voxel index space. The preset neighborhood expansion radius is used to define the spatial range for key verification during the second-scale processing, thereby reducing the diffusion of spurious voxels in irrelevant regions.

[0034] The above-mentioned mapping of the initial occupation result to the second-scale voxel space enables spatial matching from coarse to fine scale. Spatial neighborhood expansion can increase the coverage of the main structure, and the formed credible envelope region can constrain the second-scale reconstruction to be performed only within the effective space, thereby suppressing the generation of noisy voxels and false structures at the spatial level.

[0035] In one embodiment, within a second-scale voxel space with a spatial resolution higher than the first-scale voxel space, the cross-frame acoustic response evidence of each voxel is re-statistically analyzed, and the voxels are divided into stably occupied voxels and voxels to be verified according to a preset discrimination rule, including: The underwater region to be reconstructed is divided into second-scale three-dimensional voxel units; Based on the sensor pose information, the second-scale voxel unit is projected onto the distance-azimuth observation plane of the corresponding sonar image to determine whether it is within the effective imaging range; For the second-scale voxel unit within the effective imaging range, the number of effective observations and echo support times across multiple frames are counted, and the second-scale response confidence is calculated. The second-scale voxel unit that meets the first discrimination condition is identified as a stable occupied voxel; Second-scale voxel units that do not meet the first discrimination condition but meet the second discrimination condition are identified as voxels to be verified.

[0036] Specifically, let the first The effective observation markers and echo support markers of each second-scale voxel unit in multi-frame sonar images are as follows: and .in, and The calculation method is the same as that for effective observation markers and echo support markers in the first-scale voxel space, but with a higher voxel resolution. The cumulative values ​​for the [number]th [item] are [calculated separately]. Effective observation count and echo support count for each second-scale voxel unit: ; ; To obtain the acoustic response confidence level at the second scale, the following metric is constructed: ; in, and This is a second-scale smoothing parameter used to reduce confidence fluctuations caused by insufficient local observations. This index, by reducing confidence fluctuations caused by insufficient local observations through the second-scale smoothing parameter, improves the stability of voxel confidence calculations in weakly observed areas, providing an accurate basis for the stable division of occupied voxels and voxels to be verified.

[0037] If the acoustic response confidence level of a second-scale voxel unit meets a preset stability condition, it is determined as a stable occupied voxel. The set of stable occupied voxels can be represented as: ; in, To stabilize the credibility threshold, The minimum number of effective observations required to maintain a stable position.

[0038] For voxel elements that are not classified as stable occupied voxels but still have some echo support, this application does not directly delete them, but instead treats them as voxels to be verified. The set of voxels to be verified can be represented as: ; in, The minimum number of echo support cycles required for the voxel to be verified. The minimum confidence threshold required for the voxel to be verified, and Less than .

[0039] High-resolution second-scale voxel units can accurately restore local details of the target. Through effective observation judgment, count statistics, and confidence calculation, it can distinguish between stable and reliable occupied voxels and weakly observed voxels to be verified. This ensures the accuracy of strong supporting structures without discarding potential real structures and is suitable for the observation characteristics of non-uniform scanning.

[0040] In one embodiment, calculating the review evaluation value of the voxel to be reviewed includes: The acoustic response confidence term, envelope consistency term, spatial proximity term, and local connectivity term of the voxel to be verified are determined respectively; The acoustic response confidence term, envelope consistency term, spatial proximity term, and local connectivity term are weighted and summed to obtain the review evaluation value. When there are stable occupying voxels in the preset neighborhoods on both sides of the voxel to be verified, it is determined that the voxel to be verified has the function of supplementing and connecting the local broken region, and the corresponding local connection term value is assigned.

[0041] Specifically, to ensure that the preservation process of the voxels to be verified is consistent with the observation geometry of the forward-looking sonar, this application divides the voxels to be verified into multiple local voxel groups based on the distance index and azimuth index of the voxel unit in the sonar observation plane. Let the voxel be... The distance index and orientation index corresponding to each voxel to be verified are respectively and Then its local grouping number can be represented as: ; in, and These represent the grouping step sizes in the range and azimuth directions, respectively. This results in multiple local voxel groups: ; For each voxel to be verified, this application calculates a verification evaluation value based on its acoustic response confidence, confidence envelope consistency, spatial proximity, and local connectivity. Let the voxel to be verified be... The envelope consistency term is : ; in, Less than The constant is used to reduce the retention priority of voxels outside the trusted envelope region.

[0042] Let the voxel to be verified be... To a stable set of occupies The closest distance is: ; Then its spatial neighbor term can be expressed as: ; in, This is a proximity adjustment parameter. This parameter is used to increase the re-re ...

[0043] Furthermore, in order to determine whether the voxel to be examined helps to connect locally fractured structures, local connectivity terms are constructed. If the voxel to be re-examined If stable occupying voxels exist in both adjacent neighborhoods of a given voxel, then the voxel is considered to have a supplementary connecting effect on the local fracture region. This local connectivity term can be expressed as: ; in, and They represent the voxels to be verified. Two local neighborhoods in opposite directions.

[0044] Taking all the above factors into account, the voxels to be re-examined The review evaluation value can be expressed as: ; in, , , and The preset weight parameters are used to adjust the influence of acoustic response confidence, envelope consistency, spatial proximity, and local connectivity on the verification process.

[0045] The four scoring indicators above quantify the value of voxels from four dimensions: voxel self-response, spatial constraint consistency, structural proximity, and fracture repairability. The weighted summation of the review evaluation value can objectively reflect the structural importance of the voxels to be reviewed. Among them, the local connectivity item can specifically repair the problems of slender structure interruption and local fracture, and improve the continuity of the reconstruction model.

[0046] In one embodiment, screening voxels to be reviewed based on review evaluation values ​​includes: The voxels to be verified are divided into several local voxel groups according to the sonar range-azimuth observation unit; Within each local voxel group, voxels are sorted from high to low according to their review evaluation values, and no more than the preset maximum number of voxels to be reviewed are selected.

[0047] Specifically, in each local voxel group Within, according to the review evaluation value Sort the voxels to be retained and select those that meet the retention criteria. The set of retained voxels can be represented as: ; in, This indicates that within the local voxel group, selection should be based on the review evaluation value, not exceeding [a certain value]. Individual factors, This represents the maximum number of voxels retained in this local area.

[0048] Grouping sonar range-azimuth observation units according to the imaging geometry of forward-looking sonar, and sorting within each group by evaluation value with limited selection, can avoid the excessive introduction of pseudo-structures by local voxels, thus maintaining noise suppression while preserving the real weak observation structure.

[0049] In one embodiment, the maximum number of voxels retained in a local voxel set is adaptively determined based on the observation density within that local voxel set; the three-dimensional surface is extracted based on the target-occupied voxel set, and the moving cube algorithm is used to perform isosurface extraction.

[0050] Specifically, It can be adaptively determined based on the observation density within a local voxel set: ; in, The maximum number of voxels allowed to be retained for a single local voxel set. To retain the proportionality coefficient, This indicates the number of voxels to be replicated in the local voxel group.

[0051] Finally, the stable voxel set will be occupied. With preserved voxel sets Fusion yields the target-occupied voxel set: ; Based on the target occupied voxel set A three-dimensional voxel field is constructed, and a three-dimensional surface model is obtained using isosurface extraction. The isosurface extraction method can be the moving cube algorithm or other voxel field-based surface extraction algorithms. This approach preserves weakly observed but structurally significant local voxels under non-uniform sonar scanning conditions, while suppressing isolated noise voxels and unconstrained diffusion of spurious structures.

[0052] The aforementioned maximum retention quantity adaptively determines the retention volume dynamically based on local observation density, taking into account the reconstruction needs of regions with different observation densities. The moving cube algorithm is a mature isosurface extraction algorithm that can quickly and accurately extract three-dimensional surface models based on the target-occupied voxel set, ensuring reconstruction efficiency and model accuracy.

[0053] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0054] In one embodiment, such as Figure 3 As shown, a three-dimensional reconstruction device for forward-looking sonar images with non-uniform scanning is provided, comprising: The data acquisition module 302 is used to acquire multiple frames of forward-looking sonar images and sensor pose information corresponding to each frame of images, and to establish a coordinate mapping relationship from the sonar observation coordinate system to the three-dimensional reconstruction space based on the multiple frames of forward-looking sonar images and sensor pose information corresponding to each frame of images. The coarse-scale occupancy construction module 304 is used to statistically fuse the observable state and acoustic echo response of each voxel in the multi-frame forward-looking sonar image based on the coordinate mapping relationship, in the first-scale voxel space, to obtain the initial occupancy result characterizing the spatial distribution of the target subject. The trusted envelope generation module 306 is used to perform scale mapping and spatial neighborhood expansion on the initial occupancy result to generate a trusted envelope region. The fine-scale voxel discrimination module 308 is used to re-statistically analyze the cross-frame acoustic response evidence of each voxel in a second-scale voxel space with a spatial resolution higher than that of the first-scale voxel space, and divide the voxels into stable occupied voxels and voxels to be verified according to the preset discrimination rules. The local verification scoring module 310 is used to calculate the verification evaluation value of the voxel to be verified based on the credible envelope region, the spatial correlation between the voxel to be verified and the stable occupied voxel, and the supplementary connection attribute of the voxel to be verified to the local fracture structure. The 3D reconstruction output module 312 is used to screen voxels to be verified based on the verification evaluation value, spatially fuse the screened voxels to be verified with stable occupying voxels to obtain a target occupying voxel set, extract the 3D surface based on the target occupying voxel set, and complete the 3D reconstruction of the forward-looking sonar image.

[0055] Specific limitations regarding the device for 3D reconstruction of forward-looking sonar images for non-uniform scanning can be found in the above description of the method for 3D reconstruction of forward-looking sonar images for non-uniform scanning, and will not be repeated here. Each module in the aforementioned device for 3D reconstruction of forward-looking sonar images for non-uniform scanning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0056] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for three-dimensional reconstruction of forward-looking sonar images for non-uniform scanning. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0057] Those skilled in the art will understand that Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0058] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0059] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for 3D reconstruction of forward-looking sonar images for non-uniform scanning, characterized in that, The method includes: Acquire multiple frames of forward-looking sonar images and sensor pose information corresponding to each frame; based on the multiple frames of forward-looking sonar images and sensor pose information corresponding to each frame, establish a coordinate mapping relationship from the sonar observation coordinate system to the three-dimensional reconstruction space. Based on the coordinate mapping relationship, the observable state and acoustic echo response of each voxel in the multi-frame forward-looking sonar image are statistically fused in the first scale voxel space to obtain the initial occupancy result characterizing the spatial distribution of the target subject. The initial occupancy result is then subjected to scale mapping and spatial neighborhood expansion processing to generate a reliable envelope region. In the second-scale voxel space with a spatial resolution higher than that of the first-scale voxel space, the cross-frame acoustic response evidence of each voxel is re-statistically analyzed, and the voxels are divided into stable occupied voxels and voxels to be verified according to the preset discrimination rules. Based on the credible envelope region, the spatial correlation between the voxel to be verified and the stable occupied voxel, and the supplementary connection attribute of the voxel to be verified to the local fracture structure, the verification evaluation value of the voxel to be verified is calculated. Based on the verification evaluation value, voxels to be verified are selected, and the selected voxels to be verified are spatially fused with stable occupying voxels to obtain a target occupying voxel set. Based on the target occupying voxel set, a three-dimensional surface is extracted to complete the three-dimensional reconstruction of the forward-looking sonar image.

2. The method according to claim 1, characterized in that, In the first-scale voxel space, the observable state and acoustic echo response of each voxel in multiple frames of forward-looking sonar images are statistically fused to obtain the initial occupancy result characterizing the spatial distribution of the target subject, including: The underwater area to be reconstructed is divided into three-dimensional voxel units of the first scale, and the center position of each voxel unit in the world coordinate system is determined. Based on the sensor pose information, the center position of the voxel unit is transformed to the corresponding sonar coordinate system and projected onto the range-azimuth observation plane of the sonar frame to obtain the range index and azimuth index. Determine whether the distance index and azimuth index are within the effective imaging range of the sonar, and mark the voxel units within the effective imaging range as effective observation voxel units; Based on the difference between the echo intensity of the image position corresponding to the effective observed voxel unit and the background response level of the current frame, it is determined whether occupancy support is obtained, and the number of effective observations and the number of occupancy support are accumulated. The first-scale confidence index is calculated based on the number of valid observations and the number of occupied support, and voxel units are selected according to the preset confidence conditions to form the initial occupation result.

3. The method according to claim 2, characterized in that, The first scale credibility index is: in, and This is a smoothing parameter used to adjust the stability of the confidence estimate under low observation conditions. For the first The number of echo support times for each first-scale voxel unit in multiple frames of sonar images. For the first The number of effective observations of a first-scale voxel unit in multiple frames of sonar images.

4. The method according to claim 1, characterized in that, The initial occupancy result is subjected to scale mapping and spatial neighborhood expansion processing to generate a reliable envelope region, including: The initial occupancy result is mapped to the second-scale voxel space, and the mapped voxel set is expanded spatially according to a preset neighborhood radius to obtain a reliable envelope region.

5. The method according to claim 1, characterized in that, In the second-scale voxel space, where the spatial resolution is higher than that of the first-scale voxel space, the cross-frame acoustic response evidence of each voxel is re-statistically analyzed. Based on a preset discrimination rule, voxels are divided into stably occupied voxels and voxels to be verified, including: The underwater region to be reconstructed is divided into second-scale three-dimensional voxel units; Based on the sensor pose information, the second-scale voxel unit is projected onto the distance-azimuth observation plane of the corresponding sonar image to determine whether it is within the effective imaging range; For the second-scale voxel unit within the effective imaging range, the number of effective observations and echo support times across multiple frames are counted, and the second-scale response confidence is calculated. The second-scale voxel unit that meets the first discrimination condition is identified as a stable occupied voxel; Second-scale voxel units that do not meet the first discrimination condition but meet the second discrimination condition are identified as voxels to be verified.

6. The method according to claim 5, characterized in that, The confidence level of the second-scale response is: in, and This is the second-scale smoothing parameter, used to reduce confidence fluctuations caused by insufficient local observations. For the first The number of echo support times for each second-scale voxel unit. For the first Effective observation count per second-scale voxel unit.

7. The method according to claim 1, characterized in that, Calculate the review evaluation value of the voxel to be reviewed, including: The acoustic response confidence term, envelope consistency term, spatial proximity term, and local connectivity term of the voxel to be verified are determined respectively; The acoustic response confidence term, envelope consistency term, spatial proximity term, and local connectivity term are weighted and summed to obtain the review evaluation value. When there are stable occupying voxels in the preset neighborhoods on both sides of the voxel to be verified, it is determined that the voxel to be verified has the function of supplementing and connecting the local broken region, and the corresponding local connection term value is assigned.

8. The method according to claim 1, characterized in that, Voxels to be reviewed were selected based on the review evaluation values, including: The voxels to be verified are divided into several local voxel groups according to the sonar range-azimuth observation unit; Within each local voxel group, voxels are sorted from high to low according to their review evaluation values, and no more than the preset maximum number of voxels to be reviewed are selected.

9. The method according to claim 8, characterized in that, The maximum number of voxels to be retained in a local voxel set is adaptively determined based on the observation density within that local voxel set; the extraction of the three-dimensional surface based on the target-occupied voxel set is performed by using the moving cube algorithm to extract isosurfaces.

10. A three-dimensional reconstruction device for forward-looking sonar images oriented towards non-uniform scanning, characterized in that, The device includes: The data acquisition module is used to acquire multiple frames of forward-looking sonar images and sensor pose information corresponding to each frame of images, and to establish a coordinate mapping relationship from the sonar observation coordinate system to the three-dimensional reconstruction space based on the multiple frames of forward-looking sonar images and sensor pose information corresponding to each frame of images. The coarse-scale occupancy construction module is used to statistically fuse the observable state and acoustic echo response of each voxel in the multi-frame forward-looking sonar image in the first-scale voxel space based on the coordinate mapping relationship, so as to obtain the initial occupancy result characterizing the spatial distribution of the target subject. The trusted envelope generation module is used to perform scale mapping and spatial neighborhood expansion on the initial occupancy result to generate a trusted envelope region; The fine-scale voxel discrimination module is used to re-statistically analyze the cross-frame acoustic response evidence of each voxel in the second-scale voxel space with a spatial resolution higher than that of the first-scale voxel space, and classify the voxels into stable occupied voxels and voxels to be verified according to the preset discrimination rules. The local verification scoring module is used to calculate the verification evaluation value of the voxel to be verified based on the credible envelope region, the spatial correlation between the voxel to be verified and the stable occupying voxel, and the supplementary connection attribute of the voxel to be verified to the local fracture structure. The 3D reconstruction output module is used to screen voxels to be screened based on the verification evaluation value, spatially fuse the screened voxels to be screened with stable occupying voxels to obtain a target occupying voxel set, extract the 3D surface based on the target occupying voxel set, and complete the 3D reconstruction of the forward-looking sonar image.

Images