Dense small target detection method and system based on weighted attention mechanism

Abstract

The invention discloses a dense small target detection method and system based on a weighted attention mechanism, and the method comprises the steps: carrying out the feature extraction of an input image, gradually extracting low-level and high-level features through a plurality of convolution layers (Conv1, Conv2 and the like), and obtaining a group of feature maps; a multi-scale feature fusion strategy is introduced, and dense small targets in the image are accurately positioned; on the basis of a multi-scale feature fusion strategy, a weighted attention mechanism is introduced, and a final weighted feature map is obtained; a loss function of traditional target detection is optimized, and the detection capability of dense small targets is enhanced. According to the invention, through the above improvement strategy, the detection capability of dense small targets is significantly improved. Experimental results on a public data set show that the algorithm provided by the invention is superior to the existing method in precision and recall rate, and especially has outstanding performance in a dense small target detection scene. Meanwhile, the algorithm has high calculation efficiency and is suitable for popularization and application in actual scenes.

Description

Technical Field

[0001] The present invention relates to the field of detection technology, and in particular to a dense small target detection method and system based on a weighted attention mechanism. Background Art

[0002] In the field of object detection, the detection of dense small objects has always been a challenging task. Traditional object detection methods usually focus on the recognition of objects with large size or simple background, but existing algorithms often perform poorly in scenes with dense small objects. Dense small objects are often disturbed by background noise, and the distance between adjacent objects is close, making it difficult for traditional detection frameworks to effectively distinguish and locate these objects. In addition, due to the small size of the object and the limited amount of information, the detection accuracy and recall rate are easily affected.

[0003] In recent years, deep learning, especially convolutional neural networks (CNNs), has made significant progress in object detection. However, CNNs still have limitations when facing small objects. To solve this problem, researchers have proposed a variety of techniques to enhance the expressiveness of models, such as multi-scale feature fusion and context information modeling. However, these methods usually ignore the subtle differences between small objects, resulting in unsatisfactory performance in dense small object detection. Based on this, how to effectively use the attention mechanism to strengthen the model's focus on key information has become an important research direction for improving the accuracy of dense small object detection. Summary of the invention

[0004] In order to solve the above technical problems, a method and system for dense small target detection based on weighted attention mechanism is provided, aiming to solve the problem of insufficient accuracy of existing target detection algorithms in dense small target scenes. By introducing the weighted attention mechanism, the model's ability to distinguish small targets and background features is effectively enhanced, and the detection accuracy and robustness are improved.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] In a first aspect of the present invention, a dense small target detection method based on a weighted attention mechanism is provided, comprising:

[0007] Perform feature extraction on the input image, gradually extract low-level and high-level features through multiple convolutional layers (Conv1, Conv2, etc.) to obtain a set of feature maps;

[0008] Introducing a multi-scale feature fusion strategy to accurately locate dense small targets in images;

[0009] Based on the multi-scale feature fusion strategy, a weighted attention mechanism is introduced to obtain the final weighted feature map;

[0010] The loss function of traditional target detection is optimized to enhance the detection capability of densely packed small targets.

[0011] Preferably, the input image is X∈R H×W×C , where H, W, and C represent the height, width, and number of channels of the image, respectively.

[0012] Preferably, after the feature extraction, a set of feature maps is obtained. in is the i-th layer feature map, and N is the number of feature layers.

[0013] Preferably, the feature after the multi-scale feature fusion strategy is The fusion process is defined as:

[0014]

[0015] Where Resize( i ,H,W) means adjusting the feature map to a uniform resolution of H×W, w i is the fusion weight of the feature map of the i-th layer.

[0016] Preferably, the definitions of channel attention and spatial attention in the weighted attention mechanism are respectively:

[0017]

[0018] where σ(·) is the Sigmoid function, GAP(·) represents global average pooling, W1 and W2 are learnable parameters of channel attention, * represents the convolution operation, and W s is the convolution kernel weight of spatial attention, channel attention A c ∈R 1×1×C Focusing on the importance of features in different channels, spatial attention A s ∈R H×W×1 Focus on the key positions of the feature map.

[0019] Preferably, the final weighted feature map is:

[0020]

[0021] where · represents the element-by-element multiplication operation, and the weighted feature map It effectively enhances the saliency of small objects while suppressing background noise.

[0022] Preferably, the loss function of the traditional target detection is usually composed of the classification loss L cls and the localization loss L IOU composition:

[0023]

[0024] Among them, the target box set is The corresponding prediction box set is is the target box weight. The target box with a smaller area has a higher weight, which strengthens the model to learn small targets.

[0025] Preferably, the optimized loss function is:

[0026] L=λ cls L cls +λ loc L loc +λ IoU L IoU

[0027] Among them, λ IoU is the balance coefficient of the weighted IoU loss.

[0028] In a second aspect of the present invention, a dense small target detection system based on a weighted attention mechanism is proposed, comprising:

[0029] A feature extraction module, which is used to extract features from the input image, and gradually extract low-level and high-level features through multiple convolutional layers (Conv1, Conv2, etc.) to obtain a set of feature maps;

[0030] A feature fusion module, which is used to introduce a multi-scale feature fusion strategy to accurately locate dense small targets in an image;

[0031] A weighting module, which is used to introduce a weighted attention mechanism based on a multi-scale feature fusion strategy to obtain a final weighted feature map;

[0032] The optimization module is used to optimize the loss function of traditional target detection and enhance the detection capability of dense small targets.

[0033] In a third aspect of the present invention, an electronic device is provided. The electronic device has at least one processor; and a memory connected to the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method of the first aspect of the present invention.

[0034] Compared with the prior art, the present invention provides a dense small target detection method and system based on weighted attention mechanism, which has the following beneficial effects:

[0035] The present invention significantly improves the detection capability of dense small targets through the above-mentioned improvement strategy. Experimental results on public data sets show that the algorithm of the present invention is superior to existing methods in both precision and recall, especially in the dense small target detection scenario. At the same time, the algorithm has high computational efficiency and is suitable for promotion and application in practical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of a dense small target detection method based on a weighted attention mechanism in the present invention;

[0037] Figure 2 An example diagram of a data set used by the algorithm of the present invention;

[0038] Figure 3 A scatter matrix diagram of the correlation of the data set used for the present invention;

[0039] Figure 4 This is an indicator diagram of the baseline training process based on YOLOV10 in the present invention;

[0040] Figure 5 This is the experimental result diagram of the present invention based on YOLOV10 as the baseline;

[0041] Figure 6 This is the PR graph of the baseline based on YOLOV10 in this invention;

[0042] Figure 7 This is the F1-Confidence curve of the baseline based on YOLOV10 in the present invention;

[0043] Figure 8 This is the confusion matrix result diagram of the present invention based on YOLOV10 as the baseline;

[0044] Fig. 9 A block diagram of an exemplary electronic device capable of implementing embodiments of the present invention is shown.

[0045] Among them, 800 is an electronic device, 801 is a computing unit, 802 is a ROM, 803 is a RAM, 804 is a bus, 805 is an I / O interface, 806 is an input unit, 807 is an output unit, 808 is a storage unit, and 809 is a communication unit. DETAILED DESCRIPTION

[0046] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0047] Example 1

[0048] Please refer to Figure 1-Figure 8 As shown, a dense small target detection method based on weighted attention mechanism includes:

[0049] S100, extracting features from the input image, gradually extracting low-level and high-level features through multiple convolutional layers (Conv1, Conv2, etc.), and obtaining a set of feature maps;

[0050] S200, introduces a multi-scale feature fusion strategy to accurately locate dense small targets in images;

[0051] S300, based on the multi-scale feature fusion strategy, introduce the weighted attention mechanism to obtain the final weighted feature map;

[0052] S400 optimizes the loss function of traditional target detection and enhances the detection capability of dense small targets.

[0053] The input image is X∈R H×W×C , where H, W, and C represent the height, width, and number of channels of the image, respectively.

[0054] After feature extraction, a set of feature maps are obtained in is the i-th layer feature map, and N is the number of feature layers.

[0055] The features after the multi-scale feature fusion strategy are: The fusion process is defined as:

[0056]

[0057] Where Resize( i ,H,W) means adjusting the feature map to a uniform resolution of H×W, w i is the fusion weight of the feature map of the i-th layer.

[0058] The definitions of channel attention and spatial attention in the weighted attention mechanism are:

[0059]

[0060] where σ(·) is the Sigmoid function, GAP(·) represents global average pooling, W1 and W2 are learnable parameters of channel attention, * represents the convolution operation, and W s is the convolution kernel weight of spatial attention, channel attention A c ∈R 1×1×C Focusing on the importance of features in different channels, spatial attention A s ∈R H×W×1 Focus on the key positions of the feature map.

[0061] The final weighted feature map is:

[0062]

[0063] where · represents the element-by-element multiplication operation, and the weighted feature map It effectively enhances the saliency of small objects while suppressing background noise.

[0064] The loss function of traditional target detection is usually composed of classification loss L cls and the localization loss L IOU composition:

[0065]

[0066] Among them, the target box set is The corresponding prediction box set is is the target box weight. The target box with a smaller area has a higher weight, which strengthens the model to learn small targets.

[0067] The optimized loss function is:

[0068] L=λ cls L cls +λ loc L loc +λ IoU L IoU

[0069] Among them, λ IoU is the balance coefficient of the weighted IoU loss.

[0070] Those skilled in the art can understand that, using a real-life example, the experimental environment is first built on a high-performance server with an NVIDIA 4090D GPU and an operating system of Ubuntu 20.04. The official code base of the PyTorch framework and YOLOv8 is installed to ensure that the pre-trained model weights can be loaded. In the data preparation stage, the small target category in the VisDrone dataset is selected for detection.

[0071] At the beginning of the experiment, modify the configuration file of YOLOv8 and replace the backbone network with the version that introduces the attention mechanism. The attention mechanism module is embedded in the middle and high-level parts of the feature extraction network, focusing on improving the ability to extract small target features. After completing the model configuration, start the training script through the command line and set the training parameters, including learning rate, batch size, and maximum training rounds. During the training process, monitor the training log of the model through TensorBoard, including the change of loss value and the improvement of training accuracy. After the training is completed, the generated weight file is used for subsequent verification.

[0072] In the verification phase, load the improved YOLOv8 model weights and perform inference on the test set images. Use OpenCV to draw the detection box and focus on the detection effect of the dense target area. Save the inference results as images and detection result files in JSON format for subsequent analysis and visualization.

[0073] In this experiment, the environment is built based on Python 3.8 and CUDA 11.3, using the YOLOv5 official repository code. Similarly, in this experiment, the VisDrone dataset is also used for the experiment.

[0074] In the model configuration stage, the detection head of YOLOv5 is improved and a weighted attention module is introduced. The specific implementation is to add a specific attention processing layer after the output layer of CSPNet, focusing on strengthening the extraction of significant features of small targets. The modified model configuration file is saved in the specified directory, and the training script is set to multi-GPU mode to increase the training speed. The dataset is loaded using PyTorch's Dataset class to ensure that data flows into the training model in batches.

[0075] During the training phase, a step-by-step strategy is adopted: in the first phase, the backbone network is frozen and only the newly added attention module is trained; in the second phase, all layers are unfrozen and global fine-tuning is performed using a small learning rate. After the training is completed, the model weights are saved and the training log is recorded. In the inference phase, an inference script is written to call the model, infer each image in the test set, and the detection results are saved in a visual form, while recording the category and location information of each target.

[0076] In the experiment based on YOLOv10, the public VisDrone dataset was selected as the main data source. This dataset contains densely distributed small targets such as pedestrians, vehicles, and cyclists. Before the experiment began, the original dataset was first cleaned and preprocessed, including removing blurry or poor quality images, and checking and correcting the labels of each image. Subsequently, all data was converted to a format supported by YOLO, and the YOLOv10 code library was downloaded, the configuration file was modified, and the experiment was carried out.

[0077] In the model configuration phase, the pre-trained weights of YOLOv10 are loaded, and the network structure is optimized, especially the enhanced small target detection module is added to the feature extraction network. The multi-scale training method is used in the experiment, and the input image size is randomly selected to enhance the adaptability of the model to targets of different sizes. The model training is completed on the NVIDIA 4090D GPU, with the batch size set to 32 and the number of training rounds to 300. The learning rate scheduler is used to dynamically adjust the learning rate. After training, the model is applied to the VisDrone test set, and reasoning is performed on each image. The output detection results are saved as JSON files for subsequent evaluation and visualization.

[0078] In a second aspect of the present invention, a dense small target detection system based on a weighted attention mechanism is proposed, comprising:

[0079] Feature extraction module, the feature extraction module is used to extract features from the input image, and gradually extract low-level and high-level features through multiple convolutional layers (Conv1, Conv2, etc.) to obtain a set of feature maps;

[0080] Feature fusion module: The feature fusion module is used to introduce a multi-scale feature fusion strategy to accurately locate dense small targets in the image;

[0081] Weighted module: The weighted module is used to introduce a weighted attention mechanism based on the multi-scale feature fusion strategy to obtain the final weighted feature map;

[0082] Optimization module,The optimization module is used to optimize the loss function of traditional target detection,,enhancing the detection capability of dense small targets.

[0083] In a third aspect of the present invention, an electronic device is provided. The electronic device has at least one processor; and a memory connected to the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method of the first aspect of the present invention.

[0084] like Fig. 9 A schematic block diagram of an electronic device 800 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0085] The electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0086] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0087] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as methods S100 to S600. For example, in some embodiments, methods S100 to S600 may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the methods S100 to S600 described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute methods S100 to S600 in any other appropriate manner (eg, by means of firmware).

[0088] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0092] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0093] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0094] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0095] In summary, the present invention significantly improves the detection capability of dense small targets through the above-mentioned improvement strategy. The experimental results on the public data set show that the algorithm of the present invention is superior to the existing methods in terms of precision and recall, especially in the dense small target detection scenario. At the same time, the algorithm has high computational efficiency and is suitable for promotion and application in practical scenarios.

[0096] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A dense small target detection method based on weighted attention mechanism, characterized in that: include: Perform feature extraction on the input image, gradually extract low-level and high-level features through multiple convolutional layers to obtain a set of feature maps; Introducing a multi-scale feature fusion strategy to accurately locate dense small targets in images; Based on the multi-scale feature fusion strategy, a weighted attention mechanism is introduced to obtain the final weighted feature map; The loss function of traditional target detection is optimized to enhance the detection capability of densely packed small targets.

2. According to claim 1, a dense small target detection method based on weighted attention mechanism is characterized in that: The input image is ,in and Represent the height, width and number of channels of the image respectively.

3. According to claim 2, a dense small target detection method based on weighted attention mechanism is characterized in that: After feature extraction, a set of feature maps is obtained. ,in It is Layer feature map, is the number of feature layers.

4. According to claim 3, a dense small target detection method based on weighted attention mechanism is characterized in that: The features after the multi-scale feature fusion strategy are: , the fusion process is defined as: in Indicates that the feature map is adjusted to a uniform resolution , For the The fusion weights of the layer feature maps.

5. According to claim 4, a dense small target detection method based on weighted attention mechanism is characterized in that: The definitions of channel attention and spatial attention in the weighted attention mechanism are: in is the Sigmoid function, represents global average pooling, and is a learnable parameter of channel attention, represents the convolution operation, is the convolution kernel weight of spatial attention, channel attention Focus on the importance of features in different channels, spatial attention Focus on the key positions of the feature map.

6. A dense small target detection method based on weighted attention mechanism according to claim 5, characterized in that: The final weighted feature map is: where · represents the element-by-element multiplication operation, and the weighted feature map It effectively enhances the saliency of small objects while suppressing background noise.

7. A dense small target detection method based on weighted attention mechanism according to claim 6, characterized in that: The loss function of traditional target detection is usually composed of classification loss and positioning loss composition: Among them, the target box set is , the corresponding prediction box set is , is the target box weight. The target box with a smaller area has a higher weight, which strengthens the model to learn small targets.

8. The method for dense small target detection based on weighted attention mechanism according to claim 7, characterized in that: The optimized loss function is: in, is the balance coefficient of the weighted IoU loss.

9. A dense small target detection system based on a weighted attention mechanism, used to implement a dense small target detection method based on a weighted attention mechanism as described in any one of claims 1 to 8, characterized in that: include: A feature extraction module, which is used to extract features from an input image, and gradually extract low-level and high-level features through multiple convolutional layers (Conv1, Conv2, etc.) to obtain a set of feature maps; A feature fusion module, which is used to introduce a multi-scale feature fusion strategy to accurately locate dense small targets in an image; A weighting module, which is used to introduce a weighted attention mechanism based on a multi-scale feature fusion strategy to obtain a final weighted feature map; The optimization module is used to optimize the loss function of traditional target detection and enhance the detection capability of dense small targets.

10. An electronic device comprising at least one processor; and a memory connected in communication with the at least one processor; characterized in that: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images