An Image Sonar Small Target Detection Method and System Based on Improved Attention Mechanism

Abstract

The present application provides an image sonar small target detection method and system based on an improved attention mechanism, implemented based on YOLOv8. The method includes: inserting the following process at the 10th layer of the backbone network of YOLOv8: upsample the original image, then successively perform 5x5 point convolution, vertical convolution, and horizontal convolution, and finally obtain data F' through an activation function; downsample the original image, then successively perform 1x1 point convolution, horizontal convolution, and vertical convolution, and finally obtain data F'' through an activation function; perform dot multiplication on data F', data F'', and the original image data, and then splice it with the original image data as the final output result. The advantages of the present application are as follows: The method allows the model to learn image features of different dimensions through upsampling and downsampling, then through aggregation in the horizontal and vertical directions, and finally uses residual connection to accelerate convergence, which can output more refined results, reduce background interference, and reduce computational overhead.

Description

Technical Field

[0001] This application belongs to the field of small target detection in image sonar, and specifically relates to a method and system for small target detection in image sonar based on an improved attention mechanism. Background Technique

[0002] Image sonar has important applications in fields such as underwater search and rescue and mine scanning. However, due to factors such as complex water environments, the accuracy of small target detection is affected. Deep learning methods can improve the detection effect, but they increase the computational overhead and lack generalization research. Although the application of deep learning methods in small target detection of image sonar has made certain progress, there are still some defects and challenges. The following are the main aspects:

[0003] 1. Inherent challenges in small target detection:

[0004] Small targets occupy fewer pixels in the image, have limited feature information, and are easily submerged by background noise. Existing deep learning models (such as Faster R-CNN, YOLO, etc.) perform poorly when dealing with small targets and are difficult to capture effective features.

[0005] 2. Model complexity and computational cost:

[0006] Deep learning models usually require a large amount of computing resources, and sonar image processing has high requirements for real-time performance. Complex models may lead to high computational costs and slow inference speeds, making it difficult to meet the needs of practical applications.

[0007] 3. Lack of dedicated models for sonar data:

[0008] Most existing deep learning models are designed for optical images, and the imaging principles and characteristics of sonar images are quite different from those of optical images. Directly applying existing models may result in poor performance, and dedicated models need to be designed according to the characteristics of sonar images. Summary of the Invention

[0009] The purpose of this application is to overcome the defects of low detection accuracy, large computational overhead, and poor generalization of existing technologies.

[0010] To achieve the above purpose, this application proposes a method for small target detection in image sonar based on an improved attention mechanism, implemented based on YOLOv8, including:

[0011] Insert the following process into the 10th layer of the backbone network of YOLOv8:

[0012] Upsample the original image, then perform 5x5 point convolution, vertical convolution, and horizontal convolution in sequence, and finally obtain data F' through an activation function;

[0013] Downsample the original image, then perform 1x1 point convolution, horizontal convolution, and vertical convolution in sequence, and finally obtain the data F” through the activation function;

[0014] Multiply the data F', the data F”, and the original image data element-wise, and then concatenate them with the original image data as the final output result.

[0015] As an improvement to the above method, the upsampling adopts the bilinear interpolation method.

[0016] As an improvement to the above method, the downsampling adopts the average pooling method.

[0017] As an improvement to the above method, the vertical convolution is 1x7 convolution.

[0018] As an improvement to the above method, the horizontal convolution is 7x1 convolution.

[0019] As an improvement to the above method, the activation function is Sigmoid.

[0020] This application also provides an image sonar small target detection system based on an improved attention mechanism, which is implemented based on the above method. The system includes:

[0021] An upsampling module, which is used to upsample the original image, then perform 5x5 point convolution, vertical convolution, and horizontal convolution in sequence, and finally obtain the data F' through the activation function;

[0022] A downsampling module, which is used to downsample the original image, then perform 1x1 point convolution, horizontal convolution, and vertical convolution in sequence, and finally obtain the data F” through the activation function;

[0023] An element-wise multiplication module, which is used to multiply the data F', the data F”, and the original image data element-wise, and then concatenate them with the original image data as the final output result.

[0024] Compared with the prior art, the advantages of this application are:

[0025] The method allows the model to learn image features in different dimensions through upsampling and downsampling, then aggregates in the horizontal and vertical directions, and finally uses residual connections to accelerate convergence, which can output more refined results, reduce background interference, and reduce computational overhead. Brief Description of the Drawings

[0026] Figure 1 Shown is the flowchart of the image sonar small target detection method based on an improved attention mechanism;

[0027] Figure 2(a) shows the schematic diagram of vertical and horizontal decoupled convolution;

[0028] Figure 2(b) shows a schematic diagram of scaled attention aggregation;

[0029] Figure 3 The following shows a diagram of the structure of YOLOv8 and the insertion positions of the attention mechanism. Specific implementation manners

[0030] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings.

[0031] Since small targets in sonar images are sparse and the background occupies most of the image, applying the attention mechanism helps to focus on key regions and reduce background interference.

[0032] Example 1

[0033] As Figure 1 shown, the present application proposes a method for detecting small targets in image sonar (SAMS) based on an improved attention mechanism. This method first infers the upsampled attention map and the downsampled attention map in parallel, and then infers the final attention results F' and F” through vertical convolution and horizontal convolution in different orders. Finally, the residual connection is used to accelerate convergence (as shown in Formula 1, Figure 1 ). The present application uses bilinear interpolation for upsampling and average pooling for downsampling to enhance the model's attention to local and global information. After upsampling, a 5x5 point convolution is first performed, and then vertical convolution and horizontal convolution of 1x7 and 7x1 are used to optimize the computational efficiency and performance. After downsampling, a 1x1 point convolution is first performed, and then horizontal convolution and vertical convolution of 7x1 and 1x7 are used to effectively fuse information of different dimensions, improve robustness, and reduce complexity by adjusting the convolution order to enhance small target perception. As shown in Figure 2(a) and Figure 2(b), Figure 2(a) is a schematic diagram of the structure of depthwise separable convolution, which consists of point convolution, horizontal convolution, and vertical convolution. The upper part of Figure 2(b) represents the convolutional part extracted after upsampling, the lower part represents the convolutional part after downsampling, and the rightmost part represents the multiplication of the upsampling result and the downsampling result, that is, a schematic diagram of aggregating the upsampling and downsampling results.

[0034]

[0035] Among them, represents dot multiplication, and + represents concatenation. The method allows the model to learn image features of different dimensions through upsampling and downsampling, then aggregates them in the horizontal and vertical directions, and finally uses the residual connection to accelerate convergence and output a more refined result.

[0036] As Figure 3 shown, the method of the present application is inserted into the 10th layer of the YOLOv8 backbone network to detect small targets based on the YOLOv8 network.

[0037] The present invention is deployed on a machine equipped with an RTX3090 graphics card, and the proposed attention mechanism is embedded at the tenth layer of the YOLOv8 backbone network (as Figure 3 ), and the UCPR2021 dataset is used to conduct experimental verification respectively.

[0038] The URPC2021 dataset contains 6000 forward-looking sonar images, covering 8 categories of objects. The dataset is divided into a training set, a validation set and a test set, and the division ratio is 7:1:2. Adopting the default training strategy of YOLOv8, the SGD optimizer is used for 350 rounds of training, the batch size is set to 32, the initial learning rate is 0.01, and the experimental input resolution is 640×640 to conduct object detection experiments.

[0039] In the experiment, object detection models such as RetinaNet, SSD, YOLOv3, YOLOv5, YOLOv6 and YOLOv8 are used for comparison. The results show that YOLOv8-SAMS reaches 59.2% mAP under the same conditions, which is 2.9 percentage points higher than the original YOLOv8, proving the effectiveness of SAMS.

[0040] Table 1 Experimental results of URPC2021 under different baseline models

[0041]

[0042] Example 2

[0043] The present application also provides an image sonar small target detection system based on an improved attention mechanism, which is implemented based on the above method. The system includes:

[0044] An upsampling module, which is used to upsample the original image, then perform 5x5 point convolution, vertical convolution and horizontal convolution in sequence, and finally obtain data F' through an activation function;

[0045] A downsampling module, which is used to downsample the original image, then perform 1x1 point convolution, horizontal convolution and vertical convolution in sequence, and finally obtain data F” through an activation function;

[0046] A dot multiplication module, which is used to perform dot multiplication on data F', data F” and the original image data, and then splice it with the original image data as the final output result.

[0047] The present application may also provide a computer device, including: at least one processor, a memory, at least one network interface, and a user interface. Each component in the device is coupled together through a bus system. It can be understood that the bus system is used to implement the connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0048] Among them, the user interface may include a display, a keyboard, or a pointing device. For example, a mouse, a trackball, a touchpad, or a touch screen, etc.

[0049] It can be understood that the memory in the disclosed embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0050] In some embodiments, the memory stores the following elements, executable modules, or data structures, or subsets thereof, or extended sets thereof: an operating system and application programs.

[0051] Among them, the operating system includes various system programs, such as the framework layer, the core library layer, the driver layer, etc., which are used to implement various basic services and handle hardware-based tasks. The application programs include various application programs, such as the Media Player, the Browser, etc., which are used to implement various application services. The program for implementing the method of the embodiments of the present disclosure may be included in the application programs.

[0052] In the above embodiments, the program or instruction stored in the memory can also be called. Specifically, it can be the program or instruction stored in the application program. The processor is used for:

[0053] Executing the steps of the above method.

[0054] The above method can be applied to the processor or implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or the instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed above. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Combining the steps of the above disclosed method can be directly embodied as being completed by the hardware decoding processor, or being completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0055] It can be understood that these embodiments described in the present application can be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0056] For software implementation, the techniques of the present application can be implemented by executing the functional modules of the present application (such as procedures, functions, etc.). The software code can be stored in a memory and executed by a processor. The memory can be implemented inside or outside the processor.

[0057] The present application can also provide a non-volatile storage medium for storing a computer program. When the computer program is executed by a processor, the various steps in the above method embodiments can be implemented.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that any modification or equivalent replacement of the technical solutions of the present application does not depart from the spirit and scope of the technical solutions of the present application, and they should all be covered within the scope of the claims of the present application.

Claims

1. A small target detection method for image sonar based on an improved attention mechanism, implemented based on YOLOv8, including: Insert the following process into the 10th layer of the YOLOv8 backbone network: The original image is upsampled, and then 5x5 point convolution, vertical convolution and horizontal convolution are performed in sequence, and finally the data F' is obtained through the activation function; The original image is downsampled, and then 1x1 point convolution, horizontal convolution and vertical convolution are performed in sequence, and finally the data F" is obtained through the activation function; Perform dot product of data F', data F" and original image data, and then concatenate them with the original image data as the final output result.

2. The image sonar small target detection method based on the improved attention mechanism according to claim 1 is characterized in that: The upsampling adopts a bilinear interpolation method.

3. The image sonar small target detection method based on the improved attention mechanism according to claim 1 is characterized in that The downsampling adopts an average pooling method.

4. The image sonar small target detection method based on the improved attention mechanism according to claim 1 is characterized in that The vertical convolution is a 1x7 convolution.

5. The image sonar small target detection method based on the improved attention mechanism according to claim 1 is characterized in that: The horizontal convolution is a 7x1 convolution.

6. The image sonar small target detection method based on the improved attention mechanism according to claim 1 is characterized in that: The activation function is Sigmoid.

7. An image sonar small target detection system based on an improved attention mechanism, implemented based on the method of any one of claims 1-6, characterized in that: The system comprises: The upsampling module is used to upsample the original image, and then perform 5x5 point convolution, vertical convolution and horizontal convolution in sequence, and finally obtain the data F' through the activation function; The downsampling module is used to downsample the original image, and then perform 1x1 point convolution, horizontal convolution and vertical convolution in sequence, and finally obtain the data F" through the activation function; and The dot product module is used to perform dot product on the data F', data F" and the original image data, and then splice them with the original image data as the final output result.

Images