A Weak Target Detection Method and System Based on Hybrid Attention and Harmonization Factor

Abstract

This invention belongs to the field of computer vision technology and discloses a weak target detection method based on hybrid attention and harmonic factors. The method includes: acquiring an image dataset and labeling the target to be detected; dividing the detection network into feature extraction units, feature fusion units, and prediction branch units, and introducing a hybrid attention mechanism to construct the detection network; training the detection network using a data-driven approach to generate a final detection model; and using the final detection model to detect weak targets in real-time sensor images. The invention also discloses a corresponding system. This invention effectively overcomes the challenges of weak targets in typical task scenarios, such as low target pixel ratio, low target signal strength, difficulty in extracting target features, and low distinction between target and background, thereby improving the model's ability to detect weak targets and comprehensively optimizing the model's detection performance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and more specifically, relates to a weak target detection method and system based on hybrid attention and harmonic factors. It can effectively solve the problem of low weak target detection and recognition rate in typical scenarios at present, and accurately identify weak targets appearing in images. Background Technology

[0002] In recent years, with rapid economic development and continuous technological progress, artificial intelligence (AI) technology has received significant attention from various industries. Scenarios that previously required substantial human resources for monitoring can now be gradually replaced by machines. Research indicates that humans acquire over 70% of their information through vision; therefore, computer vision technology is extremely important for the development of the AI ​​industry.

[0003] Specifically, computer vision refers to using electronic imaging devices to replace the human eye in tasks such as object classification and recognition. In recent years, with the widespread availability of high-performance computing resources, deep learning technology has been attempted to be applied to the field of computer vision. In 2012, Hinton and his student Alex proposed the AlexNet convolutional neural network for image classification in the ImageNet competition and won first place in that year's image classification competition. Since then, more and more researchers have tried to apply deep learning technology to computer vision tasks such as object detection, object tracking, semantic segmentation, and scene understanding.

[0004] Studies have shown that some existing target detection algorithms typically perform well in detecting relatively conspicuous targets in images; however, for faint targets in images, due to their small number of pixels and the difficulty in extracting target features, detectors often experience more missed detections and false detections.

[0005] In fact, "small" and "weak" are the two most prominent characteristics of weak targets. "Small" refers to the small number of pixels the target occupies in the image. Currently, the academic definition of a small target is based on its imaging size. Typically, in an N×N image, targets with fewer than 0.12% of pixels (less than N×N) are considered small targets. For general object detection datasets, targets with fewer than 32×32 pixels can be considered small targets. "Weak" means the target has weak signal strength in the image, low contrast with the background, and is easily affected by clutter and noise.

[0006] Accordingly, there is an urgent need for further improvements in this area to better meet the high-precision and high-efficiency detection requirements of weak targets in typical scenarios. Summary of the Invention

[0007] To address the aforementioned deficiencies or needs of existing technologies, the present invention aims to provide a weak target detection method based on hybrid attention and harmonic factors. This method fully considers the relevant characteristics of weak targets in typical task scenarios such as surveillance images and aerial images, selects a specific algorithm as a baseline, and modifies the network structure. As a result, compared with existing technologies, the model's ability to detect weak targets can be further improved, thereby comprehensively optimizing the model's detection performance.

[0008] To achieve the above objectives, according to one aspect of the present invention, a weak target detection method based on hybrid attention and harmonic factors is provided, characterized in that the method includes:

[0009] Step 1: Sample calibration

[0010] A dataset is composed of typical images containing the target to be detected, which are collected in real time using sensors, and then the target to be detected appearing in the typical images is labeled.

[0011] Step 2: Construct the detection network

[0012] The detection network is divided into a feature extraction unit, a feature fusion unit, and a prediction branch unit. The feature extraction unit uses CSPDarkNet53 as the backbone network to extract the location, texture, semantic and other related information of the target to be detected. The feature fusion unit adopts a bidirectional feature fusion method to equally aggregate the deep texture information and shallow location information of the target to be detected, and introduces a harmonic factor to adjust the fusion ratio between adjacent feature layers in the feature fusion unit. The prediction branch unit has multiple branches, and each branch is responsible for detection according to the different scale classifications of the target to be detected.

[0013] Step 3: Training the detection network

[0014] The detection network constructed in step two is trained using the sample data calibrated in step one in a data-driven manner to generate the final detection model.

[0015] Step 4: Weak Target Detection

[0016] Using the trained final detection model, the sensor image is used to detect faint targets in the real-time image and output the results.

[0017] As a further preferred option, in step one, the acquired images are first selected based on principles such as image quality, the clarity of the target to be detected, and whether the target is occluded, to prepare an initial dataset; then, the dataset is preferably standardized using the annotation format of the PASCAL VOC dataset.

[0018] As a further preferred embodiment, in step two, for the feature fusion unit, the bidirectional feature fusion method preferably includes the following process: First, the high-level features with low resolution and high semantic information are aggregated from top to bottom with the low-level features with high resolution and low semantic information through upsampling, so that the features at all scales contain rich target semantic information; then, a bottom-up feature aggregation path is added on the basis of the top-down aggregation path, thereby transferring the position information of the lower level to the high-level features, and completing the fusion of target position information.

[0019] As a further preferred embodiment, in step two, the feature fusion unit is preferably also equipped with a hybrid attention unit, which includes channel attention subunits and spatial attention subunits connected in series, wherein:

[0020] For the channel attention subunit, its feature vector is averaged across all pixels in the feature map of each channel using global average pooling. Then, a one-dimensional convolution with a kernel of k×k is used to generate channel weights between 0 and 1. Finally, the generated channel weights are multiplied element-wise with the feature map to produce a refined feature map.

[0021] For the spatial attention subunit, firstly, average pooling and max pooling operations are performed on the input feature map, and then the two resulting feature vectors are concatenated. Next, the channel dimension is compressed using a convolutional layer to generate spatial weights, which are then multiplied element-wise with the input feature vector to produce a refined feature map.

[0022] As a further preferred embodiment, in step two, the harmonic factor is preferably set to a hyperparameter that adapts to the training process and is continuously updated iteratively along with the loss function during network training iterations.

[0023] As a further preferred embodiment, in step two, the prediction branch unit is preferably a four-branch structure, wherein the first prediction branch is generated from a low-level, high-resolution feature map and is responsible for predicting small targets at the first scale; the second prediction branch is obtained by downsampling the first prediction branch and is responsible for predicting small targets at the second scale; the third prediction branch is obtained by downsampling the second prediction branch and is responsible for predicting small targets at the third scale; and the fourth prediction branch is generated from a high-level, low-resolution feature map and is responsible for predicting small targets at the fourth scale.

[0024] As a further preferred option, in step three, the standard Adam optimizer is preferably used for multiple rounds of training. After the training is completed, the final detection model can be obtained.

[0025] As a further preferred embodiment, in step four, it is preferable to use colored boxes to mark the faint targets present in the image, with different types of targets represented by rectangular boxes of different colors, and the accurate position coordinates of the faint targets are output in real time.

[0026] According to another aspect of the present invention, a corresponding weak target detection system based on hybrid attention and harmonic factors is also provided, characterized in that the system comprises:

[0027] The sample labeling module is used to collect a dataset of typical images containing the target to be detected in real time using sensors, and then label the target to be detected appearing in the typical images.

[0028] The detection network module is divided into a feature extraction unit, a feature fusion unit, and a prediction branch unit. The feature extraction unit uses CSPDarkNet53 as its backbone to extract the location, texture, and semantic information of the target object. The feature fusion unit employs a bidirectional feature fusion method to equally aggregate deep texture information and shallow location information of the target object, and introduces a harmonic factor to adjust the fusion ratio between adjacent feature layers. The prediction branch unit has multiple branches, and each branch is responsible for detection according to the different scales of the target object.

[0029] The training detection network module is used to train the constructed detection network using the sample data calibrated in step one in a data-driven manner, thereby generating the final detection model.

[0030] The weak target detection module is used to detect weak targets in the real-time image captured by the sensor using the trained final detection model, and outputs the results.

[0031] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0032] (1) This invention fully considers the relevant characteristics of weak targets in typical task scenarios such as surveillance images and aerial images, and makes targeted improvements to key operation steps and algorithm mechanisms. The selected backbone network includes novel Focus and CSP structures, which can further enhance the ability of convolutional neural networks to extract target features and facilitate the extraction of detailed information of weak targets.

[0033] (2) By adopting a bidirectional feature fusion network and introducing a harmonic factor, this invention can adaptively control the feature fusion relationship between adjacent layers of the network. During the training process, the harmonic factor changes dynamically with the loss function, thereby further promoting the learning of subtle target details and thus enabling more stable and accurate identification of small targets.

[0034] (3) The present invention further incorporates a hybrid attention mechanism into the network upsampling and feature splicing operations. The channel attention subunit contains only global average pooling and convolution operations, which can generate accurate channel weights while ensuring the module is lightweight. The spatial attention subunit contains average pooling, max pooling and convolution operations, which can generate accurate spatial weight information. Accordingly, the interference of complex backgrounds during the network training process is further suppressed, and the overall performance of the detection model is improved.

[0035] (4) The weak target detection method based on hybrid attention and harmonic factor according to the present invention can effectively overcome the difficulties of weak targets in typical task scenarios such as surveillance images and aerial images, such as low target pixel ratio, low target signal strength, difficulty in extracting target features, and low target-background distinction. It also significantly improves the detection accuracy of weak targets, optimizes the overall performance of the detection network, and enhances the robustness of the model. Attached Figure Description

[0036] Figure 1 This is a flowchart of the weak target detection method based on hybrid attention and harmonic factors according to the present invention;

[0037] Figure 2 This is a schematic diagram illustrating the entire weak target detection process according to the present invention;

[0038] Figure 3 This is a preferred embodiment of the present invention, showing an overall structural diagram of a hybrid attention unit for demonstration purposes. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Figure 1 This is an overall flowchart of the weak target detection method based on hybrid attention and harmonic factors according to the present invention. The following will combine... Figure 1 To explain the invention in more detail.

[0041] First, there is the step of sample calibration.

[0042] In this step, a dataset consisting of typical images containing the target to be detected is acquired in real time using sensors, and then the target to be detected appearing in the typical images is labeled.

[0043] More specifically, firstly, on-site images are collected by sensors according to the specific task scenario, and the collected images are filtered according to principles such as image quality, clarity of the target to be detected, and whether the target is occluded, to prepare an initial, unlabeled dataset;

[0044] Next, the prepared dataset is labeled. For example, the labeling format of the PASCAL VOC dataset, which is widely used in the field of object detection, can be used to label the self-made dataset. The labeling tool is Label Image. During the labeling process, the coordinates and categories of the target to be identified need to be labeled at the same time. After each image is labeled, an XML file will be generated.

[0045] Next is the step of building the detection network.

[0046] In this step, the detection network is divided into a feature extraction unit, a feature fusion unit, and a prediction branch unit. The feature extraction unit uses CSPDarkNet53 as the backbone network to extract the location, texture, semantic and other related information of the target to be detected. The feature fusion unit adopts a bidirectional feature fusion method to equally aggregate the deep texture information and shallow location information of the target to be detected, and introduces a harmonic factor to adjust the fusion ratio between adjacent feature layers in the feature fusion unit. The prediction branch unit has multiple branches, and each branch is responsible for detection according to the different scale classifications of the target to be detected.

[0047] More specifically, this invention preferably uses CSPDarkNet53 as the backbone network for feature extraction to extract information such as the location, texture, and semantics of the target to be detected. Compared to the traditional DarkNet53 network, the new backbone network includes novel Focus and CSP structures, which can further enhance the ability of convolutional neural networks to extract target features. The Focus structure refers to taking a value for every other pixel in the image before it enters the backbone network, similar to the neighbor downsampling operation, thus obtaining four sets of images. The Focus operation concentrates the width and height information of the image into the channel space, expanding the channel dimension to four times the original, meaning the stitched result is equivalent to turning the original three-channel mode into 12 channels. The advantage of the Focus structure is that it reduces the loss of target detail information during downsampling, which is beneficial for the detection of small targets. The CSP structure splits the feature map into two parts by channel. One part is processed by convolutional layers and residual modules for feature extraction, while the other part is combined with the transmitted feature map. Then, the two parts of the output feature map are concatenated, which solves the problem of gradient information duplication in network optimization. While reducing the computational load of the network, it forms a new fusion method, promotes the extraction of target detail information in the image, improves the efficiency of the backbone network, and enhances the overall performance of the feature extraction network.

[0048] Convolutional neural networks contain more semantic information about the target in their high-level features and more location and texture information in their low-level features. For weak target detection tasks, prediction networks need both location and semantic information about the target to achieve stable and accurate identification of small targets.

[0049] Against this backdrop, the present invention specifically employs a bidirectional feature fusion network. First, high-level features with low resolution and high semantic information are aggregated from top to bottom with low-level features with high resolution and low semantic information through upsampling, so that features at all scales contain rich target semantic information. Then, a bottom-up feature aggregation path is added on the basis of the top-down aggregation path, thereby transferring the positional information of the lower level to the higher level features and completing the fusion of target positional information.

[0050] Furthermore, since this invention focuses on the detection of weak targets, and the detailed features of small targets are more difficult to learn than those of medium and large targets, this invention proposes adding a harmonic factor between adjacent feature layers in the feature fusion module to control the feature fusion relationship between adjacent layers, in order to ensure that the trained model has good detection accuracy. This harmonic factor determines the degree of coupling between adjacent layers in the feature fusion module by reweighting the loss in gradient backpropagation. This harmonic factor is set as a hyperparameter that adapts to the training process; during network training iterations, it is continuously updated along with the loss function, and its value changes to reflect the changes in the learning difficulty of each layer during feature fusion.

[0051] According to a preferred embodiment of the present invention, considering that the target to be detected in the present invention is a weak target, such a target often occupies only a few pixels in the image and has low distinguishability from the background, making it susceptible to interference from complex backgrounds in the scene, thus affecting the detection performance of the algorithm. To address the above problems, the present invention further proposes a novel and efficient hybrid attention mechanism, comprising two sub-units: channel attention and spatial attention, with the two attention sub-units connected in a serial data flow manner. This mechanism is then integrated into the network's upsampling and feature concatenation operations.

[0052] According to another preferred embodiment of the present invention, the structure of the channel attention subunit can be described as follows. First, for the feature vector (H×W×C) of the input attention unit, global average pooling (GAP) is used to calculate the average value of all pixels in the feature map of each channel to reduce the number of parameters and computational cost. Then, a one-dimensional convolution with a kernel of k×k is used to achieve cross-channel interaction. The kernel k of the one-dimensional convolution determines the range of channel interaction, and k is an adjustable hyperparameter. In this invention, the initial value of k is set to 3. The feature vector output by the one-dimensional convolution is mapped by the sigmoid function to generate channel weights between 0 and 1. Finally, the generated channel weights are multiplied element-wise with the feature map of the input attention module to produce a refined feature map.

[0053] According to another preferred embodiment of the present invention, the structure of the spatial attention subunit can be described as follows. First, average pooling and max pooling operations are performed on the input feature map. Then, the two resulting feature vectors are concatenated. Next, the channel dimension is compressed using a convolutional layer. Finally, spatial weights are generated, for example, using a sigmoid function, and multiplied element-wise with the input feature vector to produce a refined feature map.

[0054] See also Figure 3Let's understand the combination of the two attention subunits. The input feature vector first passes through the channel attention module to generate channel weights, highlighting the category information of the target to be detected. Then, the channel weights are multiplied element-wise with the feature vector, further emphasizing the importance of different channels within the feature vector. After obtaining channel attention, the vector is fed into the spatial attention module to obtain spatial weight information, allowing the network to focus on learning the target's location information and enhancing its ability to locate the target. By introducing the attention module, the network is prompted to consistently focus on learning the detailed features of the target during training, suppressing interference from complex backgrounds in the image and improving the overall performance of the detection network.

[0055] According to another preferred embodiment of the present invention, after fully considering the difficulty of detecting small targets, a four-branch prediction network is designed to mitigate the negative impact of drastic changes in target scale on the detection results. The four branches are numbered 1, 2, 3, and 4 from top to bottom. The first prediction branch is generated from a low-level, high-resolution feature map. High-resolution feature maps are more sensitive to target location information, therefore this level is responsible for predicting small targets. The second prediction branch is obtained by downsampling from branch 1, halving the feature map size, and is responsible for predicting ordinary small targets. The third prediction branch is obtained by downsampling from branch 2 and is responsible for predicting medium-sized targets. The fourth branch is generated from a high-level, low-resolution feature map, containing rich target semantic information, and is responsible for predicting large-scale targets. In summary, the multi-scale prediction network can improve the detection accuracy of small targets while ensuring that targets of other scales can be detected stably, thereby preventing performance fluctuations in the detection model during dynamic changes in target scale.

[0056] According to another preferred embodiment of the present invention, the loss function of the weak target detection algorithm designed in this invention preferably consists of two parts: bounding box regression loss and classification loss. Bounding box regression uses GIoU as the loss function, and the calculation formula for GIoU is shown in equation (1) below, where A is the predicted box, B is the ground truth box, and C represents the smallest convex closed box containing A and B. The loss function for bounding box regression is shown in equation (2).

[0057]

[0058] L GIoULoss =1-GIoU (2)

[0059] In addition, the classification loss can be achieved by using binary cross-entropy as the loss function, and the overall loss function of the network is shown in equation (3).

[0060]

[0061] Next comes the step of training the detection network.

[0062] In this step, the detection network constructed above is trained using calibrated sample data in a data-driven manner to generate the final detection model.

[0063] More specifically, after building the weak object detection network, it is trained end-to-end using labeled data in a data-driven manner. For example, the standard Adam optimizer can be used to train the network for 100 epochs with an initial learning rate of 1e-4. In the 60th epoch, the learning rate is reduced to 1e-5 to further adjust the network parameters. The batch size during training is 16. After training, the final weak object detection model is obtained.

[0064] Finally, there is the weak target detection step.

[0065] In this step, the trained final detection model is used to detect the real-time image captured by the sensor, obtain the faint targets in the image, and output the results.

[0066] More specifically, in practical applications, when the target to be detected appears in an image, the detection network can accurately detect the target in the image and mark it with a colored box. Different types of targets are represented by rectangles of different colors, and the network outputs the accurate position coordinates of faint targets in real time.

[0067] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A weak target detection method based on hybrid attention and harmonic factors, characterized in that, The method includes: Step 1: Sample calibration A dataset is composed of typical images containing the target to be detected, which are collected in real time using sensors, and then the target to be detected appearing in the typical images is labeled. Step 2: Construct the detection network The detection network is divided into a feature extraction unit, a feature fusion unit, and a prediction branch unit. The feature extraction unit uses CSPDarkNet53 as the backbone network to extract relevant information such as the location, texture, and semantics of the target to be detected. The feature fusion unit adopts a bidirectional feature fusion method to equally aggregate the deep texture information and shallow location information of the target to be detected, and introduces a harmonic factor to adjust the fusion ratio between adjacent feature layers in the feature fusion unit. The prediction branch unit has multiple branches, and each branch is responsible for detection according to the different scales of the target to be detected. For the feature fusion unit, the bidirectional feature fusion method includes the following process: First, the high-level features with low resolution and high semantic information are aggregated from top to bottom with the low-level features with high resolution and low semantic information through upsampling, so that the features at all scales contain rich target semantic information; then, a bottom-up feature aggregation path is added on the basis of the top-down aggregation path, thereby transferring the position information of the lower level to the high-level features, and completing the fusion of target position information; the harmonic factor is set as a hyperparameter that adapts to the training process, and it is continuously updated iteratively along with the loss function during the network training iteration process; Furthermore, the feature fusion unit is also equipped with a hybrid attention unit, which includes channel attention sub-units and spatial attention sub-units connected in series. Specifically: for the channel attention sub-unit, its feature vector is subjected to global average pooling to calculate the average value of all pixels in the feature map of each channel, and then a one-dimensional convolution with a k×k kernel is used to generate channel weights between 0 and 1; finally, the generated channel weights are multiplied element-wise with the feature map to produce a refined feature map. For the spatial attention sub-unit, first, average pooling and max pooling operations are performed on the input feature map, and then the two generated feature vectors are concatenated; next, a convolutional layer is used to compress the channel dimension and generate spatial weights, which are then multiplied element-wise with the input feature vector to produce a refined feature map. Step 3: Training the detection network The detection network constructed in step two is trained using the sample data calibrated in step one in a data-driven manner to generate the final detection model. Step 4: Weak Target Detection Using the trained final detection model, the sensor image is used to detect faint targets in the real-time image and output the results.

2. The weak target detection method as described in claim 1, characterized in that, In step one, the collected images are first screened based on principles such as image quality, the clarity of the target to be detected, and whether the target is occluded, to prepare an initial dataset; then, the dataset is standardized using the annotation format of the PASCAL VOC dataset.

3. The weak target detection method as described in claim 2, characterized in that, In step two, the prediction branch unit has a four-branch structure. The first prediction branch is generated from a low-level, high-resolution feature map and is responsible for predicting small targets at the first scale. The second prediction branch is obtained by downsampling the first prediction branch and is responsible for predicting small targets at the second scale. The third prediction branch is obtained by downsampling the second prediction branch and is responsible for predicting small targets at the third scale. The fourth prediction branch is generated from a high-level, low-resolution feature map and is responsible for predicting small targets at the fourth scale.

4. The weak target detection method as described in claim 3, characterized in that, In step three, the standard Adam optimizer is used for multiple rounds of training. After training, the final detection model can be obtained.

5. The weak target detection method as described in claim 4, characterized in that, In step four, colored boxes are used to mark faint targets in the image. Different types of targets are represented by rectangles of different colors, and the accurate position coordinates of the faint targets are output in real time.

6. A weak target detection system based on hybrid attention and harmonic factors, characterized in that, The system includes: The sample labeling module is used to collect a dataset of typical images containing the target to be detected in real time using sensors, and then label the target to be detected appearing in the typical images. The detection network module is divided into a feature extraction unit, a feature fusion unit, and a prediction branch unit. The feature extraction unit uses CSPDarkNet53 as the backbone network to extract the location, texture, and semantic information of the target to be detected. The feature fusion unit employs a bidirectional feature fusion method to equally aggregate deep texture information and shallow location information of the target, and introduces a harmonic factor to adjust the fusion ratio between adjacent feature layers. The prediction branch unit has multiple branches, each responsible for detection according to different scales of the target. For the feature fusion unit, the bidirectional feature fusion method includes the following process: first, high-level features with low resolution and high semantic information are upsampled and aggregated from top to bottom with low-level features with high resolution and low semantic information, ensuring that features at all scales contain rich target semantic information; then, a bottom-up feature aggregation path is added to the top-down aggregation path. This process transfers the location information from the lower layers to the higher-level features, thus completing the fusion of target location information. The harmonic factor is set as a hyperparameter that adapts to the training process and is continuously updated along with the loss function during network training iterations. Furthermore, the feature fusion unit is equipped with a hybrid attention unit, which includes channel attention subunits and spatial attention subunits connected in series. Specifically: for the channel attention subunit, its feature vector is subjected to global average pooling to calculate the average value of all pixels in the feature map of each channel, and then a one-dimensional convolution with a k×k kernel is used to generate channel weights between 0 and 1. Finally, the generated channel weights are multiplied element-wise with the feature map to produce a refined feature map. For the spatial attention subunit, average pooling and max pooling operations are first performed on the input feature map, and then the two generated feature vectors are concatenated. Next, a convolutional layer is used to compress the channel dimension and generate spatial weights, which are then multiplied element-wise with the input feature vector to produce a refined feature map. The training detection network module is used to train the constructed detection network using calibrated sample data in a data-driven manner, thereby generating the final detection model. The weak target detection module is used to detect weak targets in the real-time image captured by the sensor using the trained final detection model, and outputs the results.

Images