Highway dangerous object small target detection method and device based on feature fusion and self-attention mechanism residual block

Abstract

The application discloses a highway dangerous small target detection method and device based on feature fusion and a self-attention mechanism residual block, and the method comprises the following steps: S1, a highway image data set with a dangerous small target is constructed, and data preprocessing is performed on the data set pictures; S2, a feature fusion strategy is constructed, and a feature map layer is extracted by a backbone network in a feature extraction stage, and a sequential fusion strategy and a separate fusion strategy are adopted; S3, a self-attention mechanism residual block is constructed, and the self-attention mechanism residual block is introduced into a feature extraction network structure; S4, the extracted features of the self-attention mechanism residual block and the fused images after feature fusion are sent into a detection head, total image loss is calculated, an optimization algorithm of gradient descent is used to adjust the weight of the network to minimize the loss, and the model is repeatedly optimized for multiple times to gradually optimize the model. The model is enhanced in the perception and extraction capacity for small target features by introducing the self-attention mechanism, and more accurate and reliable results are provided for the highway dangerous small target detection task.

Description

Technical Field

[0001] This invention relates to the field of computer vision image processing, specifically to a method and apparatus for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks. Background Technology

[0002] As a crucial component of the transportation network, highways play an irreplaceable role in promoting regional economic development and strengthening urban-rural connections. However, with increasing traffic volume, highway safety issues are becoming increasingly prominent, particularly incidents such as slow-moving traffic / congestion, littering, and vehicles leaving the road unattended. Accidents on highways can have severe consequences, causing not only casualties but also potentially long-term environmental impacts. Therefore, effectively detecting small hazardous targets on highways has become a vital task for ensuring road safety and protecting people's lives and property. However, existing methods for feature extraction of small hazardous targets still have many shortcomings and are easily affected by lighting conditions, weather conditions, and traffic density, leading to reduced detection accuracy.

[0003] To address these issues, some methods utilize YOLOv5 for small target detection of hazardous objects on highways. YOLOv5, as an innovative technology in the field of target detection, possesses numerous advantages. Its lightweight network structure and optimized model design enable it to achieve high-precision target detection while maintaining faster processing speeds and excellent real-time performance. Furthermore, its adaptive training strategy endows the model with better generalization capabilities, adapting to different data distributions. Leveraging cutting-edge deep learning techniques, such as feature pyramid networks and attention mechanisms, YOLOv5 improves detection accuracy and robustness in complex scenes, significantly enhancing the performance of target detection tasks. However, it is worth noting that due to its lightweight design, YOLOv5 may be slightly less effective in handling complex scenes, small targets, or extreme cases. Its adaptability under extreme data distributions still needs improvement, and its stability under complex conditions requires further enhancement.

[0004] Given the overall advantages of YOLOv5, methods based on improved YOLOv5 can overcome the aforementioned problems to some extent. However, these feature extraction methods still suffer from information loss. While existing methods aim to minimize information loss, they often lack consideration for innovation in feature fusion methods and further enhancement of feature extraction capabilities. Summary of the Invention

[0005] This invention aims to overcome the problems of insufficient feature extraction capability and inadequate feature fusion in existing technologies, and provides a method and device for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks includes:

[0008] S1. Construct a highway image dataset containing small hazardous targets, and divide it into training and testing sets. Perform data preprocessing on the dataset images, including image resizing and image enhancement techniques.

[0009] S2. Construct a feature fusion strategy. The feature fusion strategy is to extract three feature layers of different sizes from the backbone network during the feature extraction stage and simultaneously adopt a strategy of sequential fusion and separate fusion. The shallow P2 feature layer is fully utilized to make full use of its rich detailed information to achieve effective fusion of features at different scales. At the same time, the advantages of deep feature maps and shallow feature maps are combined to capture the feature information of the target at different scales, improve the network's utilization of image information, and improve the accuracy and efficiency of small target detection.

[0010] S3. Construct a self-attention mechanism residual block. The self-attention mechanism residual block is introduced into the feature extraction network structure. The self-attention mechanism can help the network better understand the semantic relationship between different targets, enabling the network to automatically focus on relevant regions in the image and improve the accuracy of small target detection of dangerous objects on highways. The residual connection helps to avoid the gradient vanishing problem.

[0011] S4. The fused image after feature extraction and feature fusion of the residual block through the self-attention mechanism is fed into the detection head. The total image loss is calculated based on the difference between the small target label in the source image and the model prediction box. The total loss is a linear combination of coordinate loss, confidence loss and class loss. Each part of the loss is weighted according to its importance. The loss is passed from the output layer to the input layer through backpropagation. Optimization algorithms such as gradient descent are used to update the network parameters and adjust the network weights to minimize the loss. The whole process is repeated on multiple batches of training data to gradually optimize the model so that it can more accurately predict objects in the image.

[0012] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0013] Preferably, the feature fusion strategy described in step S2 is formed by fusing the small, medium and large feature layers extracted from the backbone network with the shallow feature layer P2.

[0014] Preferably, the self-attention mechanism residual block described in step S3 includes a Convolution Module and a Res2Attention module.

[0015] The Convolution Module consists of a Conv2d convolutional layer, a batch normalization layer, and a SiLU activation function.

[0016] The Res2Attention module uses 1×1 convolutions to uniformly split the input feature map into S subsets according to the channel dimension, using x... i Let x represent the expression, where i ∈ {1, 2, ..., S}. Except for x1, each x... i By 3×3 convolution filter K i (·) Processing. Starting from i=3, x i After being sent to K i (·) before K i-1 The outputs of (·) are added together, and the result is fed into the CBAM attention mechanism. All feature mapping results are concatenated and passed through a 1×1 convolutional filter to maintain the channel size of the residual block. Finally, the self-attention mechanism residual block is constructed through residual connections.

[0017] Preferably, the loss function measures the difference between the model's predicted bounding box and the ground truth bounding box. The total loss of the network model is a linear combination of coordinate loss, confidence loss, and class loss.

[0018] Preferably, the total loss of the network model includes:

[0019] The total loss of the network model is composed of the target confidence loss L. obj Classification loss L cls And coordinate loss L loc composition:

[0020] Loss=λ1L obj +λ2L cls +λ3L loc

[0021] In the formula, λ1, λ2, and λ3 are balancing coefficients used to balance the contribution of the weights of different loss terms to the overall loss.

[0022] This is a balancing coefficient used to balance the contribution of the weights of different loss terms to the overall loss.

[0023] The target confidence loss L obj Its mathematical expression is:

[0024]

[0025] In the formula, S 2 B represents the number of grid cells, and B represents the number of anchor frames in each grid cell. The value indicates whether the target prediction box contains a target. It can be either 0 or 1, where 0 indicates the target is present and 1 indicates the target is not present. C represents the true confidence level of the target. i λ represents the confidence level of the target prediction. noobj Penalty weighting coefficient.

[0026] The classification loss L cls Its mathematical expression is:

[0027]

[0028] In the formula, c represents the category of the detected target, and P i (c) is the probability of predicting the target category. This refers to the actual probability that an object in the grid belongs to a category.

[0029] The coordinate loss L loc Its mathematical expression is:

[0030]

[0031]

[0032]

[0033] In the formula, IoU represents the area of ​​the intersection of the target prediction bounding box and the image label bounding box divided by the area of ​​their union, ρ 2 (b,b gt α represents the Euclidean distance between the center points of the target bounding box and the image label box, m represents the diagonal distance of the smallest bounding rectangle containing both the target bounding box and the image label box, α is the weighting coefficient, v represents the parameter used to measure the aspect ratio consistency between the target bounding box and the image label box, and w gt and h gt represents the width and height of the image label box; w and h represent the width and height of the target prediction box.

[0034] A second aspect of the present invention relates to a small target detection device for hazardous objects on highways based on feature fusion and self-attention mechanism residual blocks, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the method for detecting small targets for hazardous objects on highways based on feature fusion and self-attention mechanism residual blocks of the present invention.

[0035] A third aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks according to the invention.

[0036] The method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks provided by this invention has the following advantages compared with the prior art:

[0037] This invention proposes a feature fusion method that fully leverages the advantages of two fusion approaches, innovatively constructing a more effective fusion strategy. Compared to traditional model improvement methods, this invention focuses on optimizing the feature fusion method, thereby further enhancing detection accuracy and localization capabilities. Traditional model improvement methods emphasize improving feature extraction and feature fusion techniques. The method of this invention has been validated on multiple real-world hazardous small target detection datasets in highway scenarios, demonstrating its reliability and generalization ability. Therefore, this method is more suitable for highway hazardous small target detection tasks. Furthermore, this invention introduces an attention mechanism based on the residual block design, enhancing the model's perception and extraction capabilities for small target features by improving the feature extraction method, further improving the model's detection accuracy and localization capabilities. In summary, the highway hazardous small target detection method based on feature fusion and self-attention mechanism residual blocks, while improving the feature fusion method, further enhances the model's small target feature perception and extraction capabilities by introducing a self-attention mechanism, bringing more accurate and reliable results to highway hazardous small target detection tasks. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0039] Figure 1 This is a flowchart of the method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks according to the present invention.

[0040] Figure 2 This is a structural diagram of the self-attention residual block of the present invention;

[0041] Figure 3 This is a structural diagram of the feature fusion strategy of the present invention;

[0042] Figure 4 This is a structural diagram of the CBAM attention mechanism of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be described in detail and comprehensively below with reference to the accompanying drawings. It should be noted that the embodiments described below represent only some examples of the present invention, and not all possible embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are subject to the protection scope of the present invention.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the meanings commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention.

[0045] Example 1

[0046] like Figure 1 As shown in the embodiments of the present invention, the following is a clear and complete description of the technical solution:

[0047] A method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks includes the following steps:

[0048] Step S1: Data Preprocessing: For images of small hazardous targets on highways, data preprocessing is performed first. This includes resizing the images to fit the model's input requirements. Additionally, image enhancement techniques, such as contrast enhancement and histogram equalization, can be used to improve image quality. Images of small hazardous targets on highways are used to construct the model's training and testing sets.

[0049] Step S2: Construct an improved YOLOv5 model. This model includes a self-attention mechanism residual block convolutional feature extraction network, a feature fusion strategy, and bounding box prediction. In this embodiment, a convolutional neural network (CNN) is used as the feature extraction network. This network extracts rich feature representations from the original image through a series of convolutional and pooling layers for subsequent small object detection tasks. To enhance the model's ability to perceive and extract small object features, this embodiment introduces self-attention mechanism residual blocks. These blocks are inserted at appropriate positions in the network to enable the network to focus more on important feature regions and maintain residual connections during feature propagation to prevent the gradient vanishing problem.

[0050] like Figure 2 As shown, the residual block structure of the self-attention mechanism implemented in this example has a mapping dimension of 4.

[0051] The self-attention mechanism residual block consists of a Convolution Module and a Res2Attention module.

[0052] The Convolution Module consists of a Conv2d convolutional layer, a batch normalization layer, and a SiLU activation function. The normalization layer normalizes the results from the previous layer.

[0053]

[0054] Where, x (k) Refers to the output of the k-th layer, E(x) (k) ) refers to the expected output, Var(x) (k) This refers to the variance of the output. This refers to the result of normalization.

[0055] The Res2Attention module uses 1×1 convolutions to uniformly split the input feature map into S subsets according to the channel dimension, using x... i Let x represent the expression, where i ∈ {1, 2, ..., S}. Except for x1, each x... i By 3×3 convolution filter K i (·) Processing. Starting from i=3, x i After being sent to K i (·) before K i-1 The outputs of (·) are summed, and the result is fed into the CBAM attention mechanism. All feature mapping results are concatenated and passed through a 1×1 convolutional filter to maintain the channel size of the residual block. Finally, a self-attention mechanism residual block is constructed through residual concatenation. That is:

[0056]

[0057] Among them, y i This represents the output obtained from each group, where S represents the controllable scaling dimension; K i (·) indicates a convolution operation.

[0058] Feature fusion strategies are employed to fully utilize feature information at different levels. The deep feature maps extracted by the backbone network are merged with the shallow feature maps using different fusion methods to obtain a more comprehensive and richer feature representation.

[0059] like Figure 3 As shown, this example implements a feature fusion strategy.

[0060] Small, Medium, and Large correspond to the feature maps to be detected, which are extracted by the backbone network and fused by PANet, with sizes of 20×20, 40×40, and 80×80, respectively. Both fusion strategies a and b can improve the performance of the model by fusing the P2 layer feature maps with Small, Medium, and Large. This example uses a strategy that performs both methods simultaneously.

[0061] The network structures with the same name in this application are the same, but the network parameters are not necessarily the same. For example, the convolutional module, as well as the BN layer and the concatenation of Concat, are also conventional unified structures, which will not be described in detail in this embodiment.

[0062] Step S3: After constructing the improved YOLOv5 model, set the loss functions for the network model. These loss functions are the target confidence loss, localization loss, and classification loss. The localization loss uses the ground truth bounding box and the predicted bounding box to calculate the CloU loss function. The target confidence loss calculates the cross-entropy loss based on the prediction results of positive and negative samples and whether the feature point contains an object. The classification loss calculates the cross-entropy loss based on the prediction results of the ground truth bounding box type and the prior bounding box type. That is:

[0063] The CloU loss function is:

[0064]

[0065]

[0066]

[0067] Where IoU represents the area of ​​the intersection of the target prediction bounding box and the image label bounding box divided by the area of ​​their union, ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the target prediction bounding box and the image label bounding box, m represents the diagonal distance of the smallest bounding rectangle containing both the target prediction bounding box and the image label bounding box, α is a weighting coefficient, υ represents a parameter used to measure the consistency of the aspect ratio between the target prediction bounding box and the image label bounding box, and w gt and h gt represents the width and height of the image label box; w and h represent the width and height of the target prediction box.

[0068] The target confidence loss is defined as follows:

[0069]

[0070] Among them, S 2 B represents the number of grid cells, and B represents the number of anchor frames in each grid cell. The value indicates whether the target prediction box contains a target. It can be either 0 or 1, where 0 indicates the target is present and 1 indicates the target is not present. C represents the true confidence level of the target. i λ represents the confidence level of the target prediction. noobj Penalty weighting coefficient.

[0071] Classification loss L cls Its mathematical expression is:

[0072]

[0073] Where c represents the category of the detected target, P i (c) is the probability of predicting the target category. This refers to the actual probability that an object in the grid belongs to a category.

[0074] Step s4: After constructing the neural network model, iterative training is performed to optimize model performance. During training, subsequent steps are often required, such as non-maximum suppression (NMS) and post-processing. The NMS algorithm is used to process the classification results, eliminating multiple overlapping detection boxes and retaining the detection results with the highest confidence, thereby improving detection accuracy. Post-processing is then performed on the remaining detection boxes to obtain more refined detection results.

[0075] Step S5, Result Visualization Stage. In this stage, detected small hazardous targets are labeled on the original image, allowing users to easily view and analyze the detection results, thereby gaining a better understanding of the model's performance and detection status. By visually presenting the detection results on the original image, users can more accurately evaluate the model's performance and take further actions as needed, such as adjusting parameters, improving model structure, or optimizing training strategies.

[0076] This invention creatively develops a more effective fusion strategy, fully leveraging the advantages of two fusion methods. Unlike traditional model improvement methods, this invention focuses on optimizing the feature fusion method, thereby further enhancing detection accuracy and localization capabilities. Traditional model improvement methods mainly concentrate on improving steps such as feature extraction and fusion. Furthermore, an attention mechanism based on residual blocks is introduced into the design of the residual blocks. By improving the feature extraction method, the model's ability to perceive and extract small target features is enhanced, thereby further improving detection accuracy and localization capabilities. To verify the reliability and generalization ability of the method, it was validated on multiple real-world hazardous small target detection datasets in highway environments, demonstrating its superior performance. In summary, this invention's highway hazardous small target method based on feature fusion and self-attention mechanism residual blocks, based on improved feature fusion methods, further enhances the model's ability to perceive and extract small target features by introducing a self-attention mechanism, providing more accurate and reliable results for highway hazardous small target detection tasks.

[0077] Example 2

[0078] This embodiment relates to a small target detection device for hazardous objects on highways based on feature fusion and self-attention mechanism residual blocks, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the small target detection method for hazardous objects on highways based on feature fusion and self-attention mechanism residual blocks of Embodiment 1.

[0079] Example 3

[0080] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks as described in Embodiment 1.

[0081] Based on the embodiments of the present invention, further innovative explorations and applications can be made to obtain more different embodiments. In summary, the scope of protection of the present invention applies to various embodiments based on the described technical solutions, as well as all variations obtained by those skilled in the art according to the guidance of the present invention.

Claims

1. A method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks, characterized in that, Includes the following steps: S1. Construct a highway image dataset containing small hazardous targets, dividing it into a training set and a test set. Perform data preprocessing on the dataset images, including image resizing and image enhancement techniques. S2. Construct a feature fusion strategy. In the feature extraction stage, the backbone network extracts three feature layers of different sizes. At the same time, the strategy of sequential fusion and separate fusion is adopted to make full use of the shallow P2 feature layer, which has rich detailed information, to achieve effective fusion of features at different scales. It has the advantages of both deep feature maps and shallow feature maps to capture the feature information of the target at different scales, improve the network's utilization of image information, and improve the accuracy and efficiency of small target detection. S3. Construct a self-attention mechanism residual block. Introduce a self-attention mechanism residual block into the feature extraction network structure. The self-attention mechanism helps the network better understand the semantic relationship between different targets, enabling the network to automatically focus on relevant regions in the image and improve the accuracy of small target detection. The residual connection helps to avoid the gradient vanishing problem. The self-attention mechanism residual block includes a Convolution Module and a Res2Attention module. The Convolution Module consists of a Conv2d convolutional layer, a batch normalization layer, and a SiLU activation function. The normalization layer normalizes the results from the previous layer. in, Refers to the output of the k-th layer. This refers to the expected output. This refers to the variance of the output. This refers to the result of normalization; The Res2Attention module uses 1×1 convolutions to uniformly split the input feature map according to the channel dimension. A subset, using It means that, among them ;remove In addition, each Convolutional filter with 3×3 Processing; from =Starting with 3, After being sent Previously with The outputs are summed, and the result is fed into the CBAM attention mechanism. All feature mapping results are concatenated and passed through a 1×1 convolutional filter to maintain the channel size of the residual block. Finally, a self-attention mechanism residual block is constructed through residual concatenation; that is: in, This represents the output obtained from each group. Indicates the controllable scaling dimension; Indicates the convolution operation; S4. The fused image after feature extraction and feature fusion of the residual blocks through the self-attention mechanism is fed into the detection head. The total image loss is calculated based on the difference between the small target labels in the source image and the model's predicted bounding boxes. The total image loss is a linear combination of coordinate loss, confidence loss, and class loss. Each part of the loss is weighted according to its importance. The loss is passed from the output layer to the input layer through backpropagation. The optimization algorithm of gradient descent is used to update the network parameters and adjust the network weights to minimize the loss. The entire process is repeated on multiple batches of training data to gradually optimize the model so that it can more accurately predict objects in the image.

2. The method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks as described in claim 1, characterized in that, The feature fusion strategy involves the backbone network extracting feature layers of three different sizes (small, medium, and large) and a shallow feature layer. Combination and formation.

3. The method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks as described in claim 1, characterized in that, The loss function measures the difference between the model's predicted bounding box and the ground truth bounding box. The total loss of the network model is a linear combination of coordinate loss, confidence loss, and class loss. ; The total loss of the network model Loss due to target confidence Classification loss and coordinate loss composition: In the formula, This is a balancing coefficient used to balance the contribution of the weights of different loss terms to the overall loss; This is a balancing coefficient used to balance the contribution of the weights of different loss terms to the overall loss; The target confidence loss Its mathematical expression is: In the formula, Indicates the number of grids. This indicates the number of anchor frames in each grid. The value indicates whether the target prediction box contains a target. It can be either 0 or 1, where 0 indicates the target is present and 1 indicates the target is not present. Indicates the true confidence level of the target. Indicates the confidence level of the target prediction; Penalty weighting coefficient; The classification loss Its mathematical expression is: In the formula, To detect the category of the target, To predict the probability of the target category, This refers to the actual probability that an object in the grid belongs to a category; The coordinate loss Its mathematical expression is: In the formula, This represents the area of ​​the intersection of the predicted target bounding box and the image label bounding box divided by the area of ​​their union. This represents the Euclidean distance between the center points of the predicted target bounding box and the image label bounding box. This represents the diagonal distance of the smallest bounding rectangle containing the target prediction box and the image label box. These are the weighting coefficients. This represents a parameter used to measure the aspect ratio consistency between the predicted target bounding box and the image label bounding box. and Define the width and height of the image label box; and Define the width and height of the target bounding box.

4. A small target detection device for hazardous materials on highways based on feature fusion and self-attention mechanism residual blocks, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks as described in any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the method for detecting small hazardous targets on highways based on feature fusion and self-attention mechanism residual blocks as described in any one of claims 1-3.

Images