A method, apparatus, device, and storage medium for obstacle detection using forward-looking sonar images.

Abstract

This invention discloses a method, apparatus, device, and storage medium for obstacle detection in forward-looking sonar images, relating to the field of image processing technology. The solution includes: inputting a forward-looking sonar image processed by multi-domain adaptive threshold denoising into an improved YOLOv8n-seg model that replaces the neck network with a polar coordinate feature fusion network; and achieving efficient and high-precision obstacle detection in forward-looking sonar images through the synergistic optimization of multi-domain adaptive threshold denoising and the polar coordinate feature fusion network.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for forward-looking sonar image obstacle detection. Background Technology

[0002] Forward-looking sonar (FLAS) acquires real-time images of the underwater environment through acoustic imaging technology and is widely used in underwater robots, autonomous underwater vehicles (AUVs), and marine exploration equipment. With the widespread application of underwater robots in complex tasks such as marine exploration and pipeline inspection, real-time obstacle avoidance based on FLAS has become a core requirement for ensuring navigational safety. However, due to the complexity of the underwater environment, sonar images often suffer from problems such as high noise levels, low contrast, and blurred details, posing a significant challenge to the accuracy of obstacle detection.

[0003] In related technologies, existing forward-looking sonar (FLS) obstacle detection solutions are limited by the physical characteristics of sonar imaging and the bottleneck of embedded computing power, resulting in problems such as strong speckle noise, obvious fan-shaped distortion, and limited embedded computing power, making it difficult to simultaneously achieve both detection accuracy and efficiency. Summary of the Invention

[0004] This specification provides an embodiment of a forward-looking sonar image obstacle detection method to solve the problem in existing forward-looking sonar obstacle detection schemes that it is difficult to simultaneously achieve both detection accuracy and efficiency.

[0005] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0006] Firstly, the embodiments of this specification provide a forward-looking sonar image obstacle detection method, comprising:

[0007] Acquire the forward-looking sonar image to be detected;

[0008] The forward-looking sonar image to be detected is denoised by multi-domain adaptive threshold denoising.

[0009] The noise-reduced forward-looking sonar image to be detected is input into the improved YOLOv8n-seg model to obtain the target forward-looking sonar image containing the obstacle mask output by the improved YOLOv8n-seg model.

[0010] The improved YOLOv8n-seg model replaces the neck network in the original YOLOv8n-seg model with a polar coordinate feature fusion network, which is used to correct sector distortion and output multi-scale polar coordinate features.

[0011] Secondly, the embodiments of this specification provide a forward-looking sonar image obstacle detection device, comprising:

[0012] The acquisition module is used to acquire the forward-looking sonar image to be detected;

[0013] The processing module is used to perform noise reduction processing on the forward-looking sonar image to be detected through multi-domain adaptive threshold noise reduction;

[0014] The determination module is used to input the noise-reduced forward-looking sonar image to be detected into the improved YOLOv8n-seg model to obtain the target forward-looking sonar image containing the obstacle mask output by the improved YOLOv8n-seg model.

[0015] The improved YOLOv8n-seg model replaces the neck network in the original YOLOv8n-seg model with a polar coordinate feature fusion network, which is used to correct sector distortion and output multi-scale polar coordinate features.

[0016] Thirdly, the embodiments of this specification provide a forward-looking sonar image obstacle detection device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the forward-looking sonar image obstacle detection method in Scheme 1.

[0017] Fourthly, the embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the forward-looking sonar image obstacle detection method in Scheme 1.

[0018] One embodiment of this specification achieves the following beneficial effects: by inputting a forward-looking sonar image after multi-domain adaptive threshold denoising, and replacing the neck network with an improved YOLOv8n-seg model that uses a polar coordinate feature fusion network, the detection of obstacles in the forward-looking sonar image is achieved efficiently and with high accuracy through the synergistic optimization of multi-domain adaptive threshold denoising and polar coordinate feature fusion network. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic flowchart illustrating a forward-looking sonar image obstacle detection method provided in the embodiments of this specification;

[0021] Figure 2This is a schematic diagram of the noise reduction process provided in the embodiments of this specification;

[0022] Figure 3 A schematic diagram of the framework of the polar coordinate feature fusion network provided in the embodiments of this specification;

[0023] Figure 4 A schematic diagram of mask scheduling provided in the embodiments of this specification;

[0024] Figure 5 This is a schematic diagram of the structure of a forward-looking sonar image obstacle detection device provided in the embodiments of this specification;

[0025] Figure 6 This is a schematic diagram of the structure of a forward-looking sonar image obstacle detection device provided in the embodiments of this specification. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0027] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0028] The method for detecting obstacles in forward-looking sonar images provided in the embodiments of the specification will be described in detail with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart illustrating a forward-looking sonar image obstacle detection method provided in an embodiment of this specification.

[0030] like Figure 1 As shown, the process may include the following steps:

[0031] Step 110: Acquire the forward-looking sonar image to be detected.

[0032] In the embodiments described in this specification, a forward-looking sonar image to be detected, such as an 8-bit grayscale forward-looking sonar image, is acquired through a sonar acquisition board. This image can reflect the echo intensity distribution of objects in the underwater environment. The forward-looking sonar image to be detected contains noise, which affects image quality.

[0033] Step 120: Denoise the forward-looking sonar image to be detected by multi-domain adaptive threshold denoising.

[0034] In the embodiments of this specification, the image after multi-domain adaptive threshold denoising processing has significantly reduced speckle noise, while the weak echo edges are completely preserved, providing high-quality input for subsequent feature extraction and segmentation.

[0035] Step 130: Input the noise-reduced forward-looking sonar image to be detected into the improved YOLOv8n-seg model to obtain the target forward-looking sonar image containing the obstacle mask output by the improved YOLOv8n-seg model.

[0036] The improved YOLOv8n-seg model replaces the neck network in the original YOLOv8n-seg model with a polar coordinate feature fusion network, which is used to correct sector distortion and output multi-scale polar coordinate features.

[0037] In the embodiments of this specification, the neck network in the original YOLOv8n-seg model is replaced with a polar coordinate feature fusion network. The polar coordinate feature fusion network may include a polar coordinate remapping module and a multi-scale polar coordinate feature fusion module, which are used to correct sector distortion and output multi-scale polar coordinate features, thereby enhancing the network's ability to perceive the boundaries of arc-shaped and sector-shaped obstacles.

[0038] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0039] In the embodiments of this specification, the neck network of the forward-looking sonar image input after multi-domain adaptive threshold denoising is replaced with an improved YOLOv8n-seg model with a polar coordinate feature fusion network. Through the synergistic optimization of multi-domain adaptive threshold denoising and polar coordinate feature fusion network, efficient and high-precision detection of obstacles in the forward-looking sonar image is achieved.

[0040] based on Figure 1 In addition to the method described in the embodiments of this specification, some specific implementation schemes of the method are also provided, which will be described below.

[0041] Optionally, the denoising process performed on the forward-looking sonar image to be detected using multi-domain adaptive threshold denoising as described in the embodiments of this specification may specifically include:

[0042] The local noise variance is calculated by pixel sliding window, and the directional gradient is calculated by convolution to generate a pixel-by-pixel threshold map;

[0043] The pixel-by-pixel threshold image is subjected to two-level discrete wavelet decomposition, and nonlocal block matching filtering is performed according to the three-dimensional block matching algorithm to obtain the noise-reduced forward-looking sonar image to be detected.

[0044] In the embodiments described in this specification, the mean and variance of a local region are calculated using a pixel sliding window, and the variance is used to measure the noise intensity. The directional gradient is calculated using convolution, and the noise variance and gradient information are fused to generate a pixel-by-pixel threshold map.

[0045] A two-level DWT decomposition is performed on the pixel-by-pixel threshold map, decomposing the image into low-frequency approximation components and high-frequency detail components. The high-frequency components contain noise and edge information, which need to be filtered. The high-frequency components after wavelet decomposition are divided into image blocks, and similar blocks are searched in three-dimensional space and aggregated into block stacks. Noise is suppressed by collaborative filtering to obtain the denoised forward-looking sonar image to be detected.

[0046] Figure 2 This is a schematic diagram of the noise reduction process provided in the embodiments of this specification.

[0047] like Figure 2 As shown, the noise reduction process for the acquired forward-looking sonar image is performed as follows:

[0048] Step 201: Acquire forward-looking sonar images;

[0049] Step 202: Local variance. Use a 32×32 pixel sliding window to slide across the image and calculate the variance σ of the pixel values ​​within each window to determine the local noise intensity. The larger the variance, the stronger the noise in that area.

[0050] Step 203: Orientation gradient, calculate the orientation gradient of the image using a 3×3 Sobel convolution kernel. This is used to identify edge information in images. Regions with high gradient values ​​usually correspond to the edges of objects and need to be specially protected to avoid being weakened by the noise reduction process.

[0051] The calculation of local variance and gradient is parallel and independent, and variance and directional gradient are calculated simultaneously.

[0052] Step 204: Adaptive Thresholding. Based on local variance and directional gradient, a pixel-by-pixel adaptive threshold map is generated. This threshold map can dynamically adjust the noise reduction intensity according to the noise and edge characteristics of different regions of the image.

[0053] Step 205: Wavelet soft thresholding denoising. The image is decomposed into a two-level discrete wavelet decomposition, consisting of a low-frequency sub-band and a high-frequency sub-band. The low-frequency sub-band retains the main structure of the image, while the high-frequency sub-band contains noise and edge information. Adaptive thresholding is applied only to the coefficients of the high-frequency sub-band for soft thresholding. If the absolute value of the coefficients... If, then set to zero (clear noise); if Then it shrinks to (Preserves edges and reduces ringing effects). Soft thresholding provides a smoother transition than hard thresholding, making it less prone to artifacts during reconstruction. Soft thresholding can remove noise while better preserving the edge information of the image, avoiding artifacts that may be caused by hard thresholding.

[0054] Step 206: Adaptive BM3D denoising, utilizing... The mean of σ is used to estimate the global noise level, driving the 3D Block Matching (BM3D) algorithm for nonlocal block matching filtering. Similar 8×8 patches are searched in the image and stacked into a 3D array. A 3D transformation is performed on the 3D array, and a hard threshold is applied to remove noise coefficients. An inverse transformation is performed on the filtered 3D array, and Wiener filtering is used for further optimization. The optimized blocks are weighted and averaged back to their original image positions to generate the final denoising result. BM3D further smooths residual noise by combining 3D transformation of similar image patches with hard thresholding, while preserving image details and edges.

[0055] Step 207: Output the noise-reduced forward-looking sonar image.

[0056] Specifically, local noise variance can represent the degree of local texture variation. The larger the local noise variance, the stronger the noise in that area. The formula for calculating local noise variance is:

[0057]

[0058] in, For pixels The local noise variance at that location For variance calculation, In pixels Centered Image pixel values ​​within a local window.

[0059] The formula for calculating the directional gradient is:

[0060] in, For pixels The gradient magnitude at a given point measures the edge strength. This is the image gradient vector, representing the direction and magnitude of the intensity change of the image at that point; The horizontal image gradient, The vertical image gradient, and The calculation formula is:

[0061] in, The original image matrix, This is a two-dimensional convolution operation.

[0062] The formula for calculating the adaptive threshold is:

[0063] in, For pixels Adaptive threshold at the location, To control the weighting coefficients for noise suppression, To control the weighting coefficients for edge retention, For pixels Local noise standard deviation at the location For pixels The magnitude of the directional gradient at that location. In practice, .

[0064] The wavelet soft thresholding denoising formula is:

[0065] in, These are the coefficients after soft thresholding. These are the original wavelet high-frequency coefficients; An adaptive threshold for the position corresponding to the current coefficient; for The sign function is +1 for positive, -1 for negative, and 0 for zero; For the maximum value operation (if) If the value is zero, then set it to zero.

[0066] The wavelet reconstruction formula is:

[0067] in, The image after wavelet soft thresholding denoising (reconstruction result), IDWT is the two-dimensional wavelet inverse transform. For the low-frequency subband of wavelet transform, For the first The wavelet high-frequency subband of the layer, This represents the wavelet decomposition level.

[0068] Global noise estimation can be used for subsequent BM3D filter intensity adjustment. The formula for global noise estimation is:

[0069] in, For global noise estimation of the image; This represents the average value of all local noise in the entire image; is the scaling factor, which can be set to 1.0 to adjust the estimated intensity; 255 is the maximum pixel value of the grayscale image, scaling the pixel range to [0,1].

[0070] Adaptive BM3D denoising, the BM3D (Block-Matching and 3D Filtering) algorithm includes the following steps:

[0071] Block Matching: Searches for image blocks (e.g., similar to reference blocks) in the entire image and stacks them into a 3D data volume.

[0072] Collaborative HT (Hard Thresholding): Perform 3D transformation (such as DCT) on 3D data volumes, and then perform hard thresholding to remove noise.

[0073] Wiener re-estimation: The second stage uses Wiener filtering to make a more refined estimate of the image patch, improving edge preservation.

[0074] Aggregation: Combines multiple estimated image patches into a complete image using a weighted average, reducing block artifacts.

[0075] The final output is a high signal-to-noise ratio (SNR) and sharp-edge forward-looking sonar image to be detected. The formula for calculating the forward-looking sonar image to be detected is as follows:

[0076] in, The final output is a high signal-to-noise ratio, sharp-edged forward-looking sonar image of the target device; Used to limit pixel values ​​to [0, 255] to prevent BM3D output values ​​from exceeding the image range; BM3D Indicates to Using the BM3D algorithm, based on global noise estimation Noise reduction is performed.

[0077] Wavelet soft thresholding denoising and adaptive BM3D denoising form a two-stage collaborative mechanism in terms of overall structure, proceeding from coarse to fine. Wavelet soft thresholding suppresses high-frequency noise through frequency domain decomposition and local adaptive thresholding, providing a smoother input image for the subsequent refined BM3D denoising. Simultaneously, the mean local standard deviation calculated in the wavelet stage is also used as the global noise estimation input in the BM3D stage, thereby achieving parameter sharing and data coupling, enhancing the robustness and structural preservation capability of the overall denoising framework.

[0078] Optionally, the method described in the embodiments of this specification for inputting the noise-reduced forward-looking sonar image to be detected into an improved YOLOv8n-seg model to obtain a target forward-looking sonar image containing an obstacle mask output by the improved YOLOv8n-seg model may specifically include:

[0079] The noise-reduced forward-looking sonar image to be detected is input into the backbone network of the improved YOLOv8n-seg model, and local feature extraction is performed on the forward-looking sonar image to be detected through convolution operation to generate a feature map.

[0080] The polar coordinate feature fusion network remaps the feature map to a polar coordinate network and performs multi-scale feature fusion in the polar coordinate domain to generate a multi-scale feature map.

[0081] The head network receives the multi-scale feature maps and generates a forward-looking sonar image of the target containing an obstacle mask through linear combination.

[0082] In practice, the main components of the original YOLOv8n-seg model include the backbone, neck, and head. A polar coordinate domain feature fusion network is constructed to replace the neck structure of the original YOLOv8n-seg model.

[0083] Figure 3 This is a schematic diagram of the framework of the polar coordinate feature fusion network provided in the embodiments of this specification.

[0084] like Figure 3 As shown, the forward-looking sonar image (high signal-to-noise ratio, weak edge preservation) after multi-domain adaptive threshold denoising is input into the backbone network of the improved YOLOv8n-seg model. The backbone network extracts local features from the input image through a series of convolutional layers (such as the CSPDarknet structure), generating feature maps at different scales (P3, P4, P5). P3 is at 1 / 8 resolution and contains high-frequency detail information (such as small target edges). P4 is at 1 / 16 resolution, balancing semantics and details. P5 is at 1 / 32 resolution and contains global semantic information (such as obstacle categories). The feature maps extracted by the backbone network provide the foundation for subsequent polar coordinate remapping and multi-scale fusion, ensuring that obstacle features at different scales are effectively captured.

[0085] The Polar-PAN network replaces the neck network of the original YOLOv8n-seg model. The Polar-PAN network includes a polar coordinate remapping module and a multi-scale polar coordinate feature fusion module.

[0086] Polar coordinate remapping module: Performs polar coordinate remapping (ToPolar) on the P5, P4, and P3 features output by Backbone. ToPolar can correct the fan-shaped geometric distortion of the sonar image and convert the feature map in the Cartesian coordinate system to the polar coordinate system.

[0087] Specifically, taking the image vertices as poles, the radius r is divided according to the logarithmic interval (to adapt to the sonar range attenuation characteristics), and the division is equal in angle. (Uniformly cover azimuth angle), construct The size of the sampling grid.

[0088] The grid_sample function is used to project the feature maps P3, P4, and P5 from the Cartesian coordinate system to the (r,θ) polar coordinate grid to generate polar coordinate domain feature maps (Polar-P3, Polar-P4, Polar-P5).

[0089] Multi-scale polar coordinate feature fusion module: bidirectional fusion is performed in the polar coordinate domain. In the top-down path (FPN), the high-level feature (Polar-P5) is upsampled by 2 times and then concatenated with the mid-level feature (Polar-P4), and fused through a C2f convolutional block (512 channels); then upsampled and concatenated with the low-level feature (Polar-P3) to generate a C2f(256) feature map.

[0090] The bottom-up path (PAN) downsamples the low-level detail features (C2f(256)) by 2 times and merges them with the mid-level features (Polar-P4) to generate C2f(512); then it is downsampled again and merged with the high-level features (Polar-P5) to generate C2f(1024).

[0091] The final output consists of three multi-scale polar coordinate feature maps (Polar-P3 (1 / 8), Polar-P4 (1 / 16), and Polar-P5 (1 / 32)), maintaining the geometric consistency of the curved obstacle in the polar coordinate domain. Pixel alignment at the curved obstacle preserves both local texture and global semantics. Only three grid_sample operations and two C2f modules are added, increasing FLOPs by approximately 5%, but improving the mIoU of the curved target by 2 to 3 percentage points. Furthermore, the remapping operation supports backpropagation, and pre-trained weights can be strictly loaded (strict=False), making training convergence effortless.

[0092] The head network receives multi-scale polar coordinate feature maps (Polar-P3, Polar-P4, Polar-P5) output by Polar-PAN. The multi-scale feature maps are linearly combined using 1×1 convolutions to generate instance masks, marking the spatial distribution of obstacles. This generates a forward-looking sonar image of the target containing the obstacle mask, accurately depicting the contour regions of irregular obstacles, which can then guide AUVs in obstacle avoidance.

[0093] In practice, before inputting the forward-looking sonar image to be detected into the improved YOLOv8n-seg model, the improved YOLOv8n-seg model can be pre-trained to improve the accuracy of the model's prediction.

[0094] Optionally, the improved YOLOv8n-seg model described in the embodiments of this specification is run on a platform, and the method may include:

[0095] The GPU utilization of the platform is obtained, and the number of mask prototypes and input resolution of the improved YOLOv8n-seg model are adjusted according to the hysteresis threshold.

[0096] In practice, the improved YOLOv8n-seg model can run on the Jetson Orin NX 15W platform. The frame rate will vary depending on the hardware conditions of the platform. The mask prototype dynamic scheduling can detect the GPU utilization of the platform and ensure that the frame rate can be between 25 and 28 fps.

[0097] Figure 4 This is a logical diagram illustrating mask scheduling provided in the embodiments of this specification.

[0098] like Figure 4 As shown, an independent thread is set to read the GPU utilization U every 0.5 seconds, and an upward and downward threshold is set to avoid performance fluctuations caused by frequent adjustments.

[0099] The core utilization rate U is read via the NVML interface. If the current state is K32 and U exceeds 70%, the mask prototype number is halved to K16. If it is still higher than the threshold, it is further reduced to K8, and the network input resolution is simultaneously reduced from 640px to 480px, thereby directly reducing the number of pixels in the feature extraction layer by 25%. When the utilization rate is lower than the 50% hysteresis threshold, it is restored in reverse order from K8 to K16 to K32, and the resolution is reset to 640px when K16 is about to rise again. Resolution scaling is performed by performing bilinear interpolation on the input frame.

[0100] Since the prototype mask of YOLOv8-Seg already covers 95% of semantic information in the first 16 channels, reducing K from 32 to 8 only results in a 0.4 percentage point loss in mIoU, but reduces mask multiplication and memory usage to one-quarter; combined with resolution downgrading, total FLOPs decrease by about 60%. Running on a 15W Jetson Orin NX platform, under task parallelism, GPU_util can oscillate from 35% to 85%, with the scheduling mechanism ensuring the frame rate remains consistently at 25–28 fps. In contrast, the unscheduled baseline model experiences frame rate drops to 15 fps and misses detections during high-load periods. Furthermore, a threshold design with U_hi / U_lo = 70% / 50% hysteresis avoids fluctuations around 60%.

[0101] By employing multi-domain adaptive threshold denoising, polar coordinate feature fusion networks, and dynamic mask scheduling, this system achieves a balance between high precision, real-time performance, and low maintenance costs that are difficult to achieve with similar sonar obstacle avoidance systems. Multi-domain adaptive threshold denoising reduces speckle noise by an average of 5dB while fully preserving weak echo edges without relying on any prior parameters. Multi-scale feature fusion effectively enhances the network's ability to perceive the boundaries of curved and fan-shaped obstacles. The closed-loop scheduling of dynamically trimmed mask prototypes and resolution ensures a stable frame rate of 25-28 FPS for the detection platform with an accuracy loss of less than 0.4 percentage points, providing highly reliable and low-power real-time obstacle avoidance capabilities for AUV inspection, port security, and emergency search and rescue.

[0102] This invention achieves a real-time three-way balance between noise, computing power, and accuracy, providing AUVs with stable and reliable forward-looking sonar segmentation and obstacle avoidance capabilities in real-world scenarios with long ranges and varying loads.

[0103] Figure 5 This is a schematic diagram of the structure of a forward-looking sonar image obstacle detection device provided in the embodiments of this specification.

[0104] Corresponding to the method embodiment, this embodiment also provides a forward-looking sonar image obstacle detection device, which may include:

[0105] The acquisition module 502 is used to acquire the forward-looking sonar image to be detected;

[0106] Processing module 504 is used to perform noise reduction processing on the forward-looking sonar image to be detected through multi-domain adaptive threshold noise reduction;

[0107] The determination module 506 is used to input the noise-reduced forward-looking sonar image to be detected into the improved YOLOv8n-seg model to obtain the target forward-looking sonar image containing the obstacle mask output by the improved YOLOv8n-seg model.

[0108] The improved YOLOv8n-seg model replaces the neck network in the original YOLOv8n-seg model with a polar coordinate feature fusion network, which is used to correct sector distortion and output multi-scale polar coordinate features.

[0109] Optionally, the denoising process performed on the forward-looking sonar image to be detected using multi-domain adaptive threshold denoising as described in the embodiments of this specification may specifically include:

[0110] The local noise variance is calculated by pixel sliding window, and the directional gradient is calculated by convolution to generate a pixel-by-pixel threshold map;

[0111] The pixel-by-pixel threshold image is subjected to two-level discrete wavelet decomposition, and nonlocal block matching filtering is performed according to the three-dimensional block matching algorithm to obtain the noise-reduced forward-looking sonar image to be detected.

[0112] Optionally, the method described in the embodiments of this specification for inputting the noise-reduced forward-looking sonar image to be detected into an improved YOLOv8n-seg model to obtain a target forward-looking sonar image containing an obstacle mask output by the improved YOLOv8n-seg model may specifically include:

[0113] The noise-reduced forward-looking sonar image to be detected is input into the backbone network of the improved YOLOv8n-seg model, and local feature extraction is performed on the forward-looking sonar image to be detected through convolution operation to generate a feature map.

[0114] The polar coordinate feature fusion network remaps the feature map to a polar coordinate network and performs multi-scale feature fusion in the polar coordinate domain to generate a multi-scale feature map.

[0115] The head network receives the multi-scale feature maps and generates a forward-looking sonar image of the target containing an obstacle mask through linear combination.

[0116] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0117] Figure 6 This is a schematic diagram of the structure of a forward-looking sonar image obstacle detection device provided in an embodiment of this specification. Figure 6 As shown in the embodiments of this specification, a forward-looking sonar image obstacle detection device 600 includes a memory 630, a processor 610, and a computer program 620 stored in the memory. The processor 610 executes the computer program 620 to implement the forward-looking sonar image obstacle detection method described in any of the above embodiments.

[0118] The embodiments of this specification provide a forward-looking sonar image obstacle detection device, which may include a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the forward-looking sonar image obstacle detection method described in any of the above embodiments.

[0119] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the forward-looking sonar image obstacle detection method described in any of the above embodiments.

[0120] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 6 As the device shown is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0121] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0122] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0123] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0124] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0130] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0131] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0133] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0135] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for obstacle detection using forward-looking sonar images, characterized in that, include: Acquire the forward-looking sonar image to be detected; The forward-looking sonar image to be detected is denoised by multi-domain adaptive threshold denoising. The noise-reduced forward-looking sonar image to be detected is input into the improved YOLOv8n-seg model to obtain the target forward-looking sonar image containing the obstacle mask output by the improved YOLOv8n-seg model; specifically including: The noise-reduced forward-looking sonar image to be detected is input into the backbone network of the improved YOLOv8n-seg model, and local feature extraction is performed on the forward-looking sonar image to be detected through convolution operation to generate a feature map. The polar coordinate feature fusion network remaps the feature map to a polar coordinate network and performs multi-scale feature fusion in the polar coordinate domain to generate a multi-scale feature map. The head network receives the multi-scale feature maps and generates a forward-looking sonar image of the target containing an obstacle mask through linear combination. The improved YOLOv8n-seg model replaces the neck network in the original YOLOv8n-seg model with a polar coordinate feature fusion network, which is used to correct sector distortion and output multi-scale polar coordinate features.

2. The method according to claim 1, characterized in that, The noise reduction process for the forward-looking sonar image to be detected by multi-domain adaptive threshold noise reduction specifically includes: The local noise variance is calculated by pixel sliding window, and the directional gradient is calculated by convolution to generate a pixel-by-pixel threshold map; The pixel-by-pixel threshold image is subjected to two-level discrete wavelet decomposition, and nonlocal block matching filtering is performed according to the three-dimensional block matching algorithm to obtain the noise-reduced forward-looking sonar image to be detected.

3. The method according to claim 1, characterized in that, The improved YOLOv8n-seg model is run on the platform, and the method includes: The GPU utilization of the platform is obtained, and the number of mask prototypes and input resolution of the improved YOLOv8n-seg model are adjusted according to the hysteresis threshold.

4. The method according to claim 2, characterized in that, The formula for calculating the adaptive threshold of the pixel-by-pixel threshold map is as follows: in, For pixels Adaptive threshold at the location, To control the weighting coefficients for noise suppression, To control the weighting coefficients for edge retention, For pixels Local noise standard deviation at the location For pixels The magnitude of the directional gradient at that location.

5. A forward-looking sonar image obstacle detection device, characterized in that, include: The acquisition module is used to acquire the forward-looking sonar image to be detected; The processing module is used to perform noise reduction processing on the forward-looking sonar image to be detected through multi-domain adaptive threshold noise reduction; The determination module is used to input the denoised forward-looking sonar image to be detected into the improved YOLOv8n-seg model to obtain the target forward-looking sonar image containing the obstacle mask output by the improved YOLOv8n-seg model; specifically, it includes: The noise-reduced forward-looking sonar image to be detected is input into the backbone network of the improved YOLOv8n-seg model, and local feature extraction is performed on the forward-looking sonar image to be detected through convolution operation to generate a feature map. The polar coordinate feature fusion network remaps the feature map to a polar coordinate network and performs multi-scale feature fusion in the polar coordinate domain to generate a multi-scale feature map. The head network receives the multi-scale feature maps and generates a forward-looking sonar image of the target containing an obstacle mask through linear combination. The improved YOLOv8n-seg model replaces the neck network in the original YOLOv8n-seg model with a polar coordinate feature fusion network, which is used to correct sector distortion and output multi-scale polar coordinate features.

6. The apparatus according to claim 5, characterized in that, The noise reduction process for the forward-looking sonar image to be detected by multi-domain adaptive threshold noise reduction specifically includes: The local noise variance is calculated by pixel sliding window, and the directional gradient is calculated by convolution to generate a pixel-by-pixel threshold map; The pixel-by-pixel threshold image is subjected to two-level discrete wavelet decomposition, and nonlocal block matching filtering is performed according to the three-dimensional block matching algorithm to obtain the noise-reduced forward-looking sonar image to be detected.

7. A forward-looking sonar image obstacle detection device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 4.

Images