An object detection method based on hybrid attention mechanism

Abstract

The present invention relates to a target detection method based on a hybrid attention mechanism, including obtaining an existing data set, constructing a multi-scale road scene target detection model, which is an improvement on the small target detection model MS-YOLOv5; MS-YOLOv5 uses the backbone feature extraction network of YOLOv5 as the basic model, and replaces several modules therein; MS-YOLOv5 is used as the basic model, and the modules therein are replaced and a hybrid attention mechanism MSCA is added; data resampling operation is introduced to the input data set data, and training is performed, and after optimal convergence, the model parameters are saved to obtain an optimal small target detection model; the optimal small target detection model is called, and a test picture is input for detection, and the output is the recognition result. The experimental verification results show that the method improves the detection accuracy and improves the situation of missed detection in congested scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of road scene target detection, and in particular to a target detection method based on a hybrid attention mechanism. Background Art

[0002] Road scene object detection refers to the use of computer vision technology to automatically identify and locate various types of target objects in road traffic scenes, such as vehicles, pedestrians, traffic signs, etc. At present, road scene object detection has been widely used in intelligent driving, traffic safety monitoring, urban management and other fields. It requires the use of computer vision methods to process images or video streams to achieve accurate and real-time detection of various target objects. Object detection methods involve multiple technical fields such as deep learning, machine learning, and computer vision, and have high technical complexity.

[0003] Scale inconsistency in road scene object detection has a negative impact on detection performance. If the model can only adapt to objects of a certain scale, then objects at different scales will not be correctly identified. At the same time, due to the changes in the scales of vehicles, pedestrians and other objects, there are a large number of objects with inconsistent scales in road scenes, so scale inconsistency is a serious problem in road scene object detection. To solve this problem, in 2015, He et al. designed a new module, namely Spatial Pyramid Pooling (SPP), which divides the input data into different scales, performs a maximum pooling operation on each block from all the blocks obtained, and finally merges the results to obtain a fixed input size for the fully connected layer. This solves the problem of requiring fixed-size input data.

[0004] The YOLOv5 method framework uses spatial pyramid pooling technology, which aims to fuse multiple-size maximum pooling operations together to enhance the association between local features and overall features, further expand the representation capability of the final feature map, and obtain more useful semantic information. At the same time, it can also enhance the robustness of the method in terms of spatial layout and object variability, thereby improving the detection accuracy of the method. The YOLOv5 method adds an SPP structure before the Neck network and at the back of the backbone feature extraction network backbone.

[0005] Although the YOLOv5 method uses the SPP module to extract features using pooling layers of different sizes, it can fuse features of different scales and enhance the expressiveness of the final features. However, there are still the following shortcomings:

[0006] 1. The pooling layer increases the receptive field while reducing the resolution, which leads to the loss of some detail information in the image and the inability to restore all the details, resulting in poor detection of some small targets.

[0007] 2. When traffic congestion occurs on the road, there are occlusions between objects, resulting in missed detections. Summary of the invention

[0008] In view of the above problems existing in the prior art, the technical problem to be solved by the present invention is: in the case of traffic congestion on the road, how to ensure the accuracy of inspection and avoid missed inspections as much as possible.

[0009] To solve the above technical problems, the present invention adopts the following technical solution: a target detection method based on a hybrid attention mechanism, comprising the following steps:

[0010] S1: Obtain an existing data set and preprocess the data set. All preprocessed images constitute a training set.

[0011] S2: Construct a multi-scale road scene target detection model MSAT-YOLOv5, which is an improvement on the small target detection model MS-YOLOv5;

[0012] S21: The MS-YOLOv5 replaces the channel-reorganized lightweight inverse bottleneck residual module shuffle_res2net_s with the residual structure in CSP_BottleNeck in the YOLOv5 backbone feature extraction network, and is defined as the MSL_backbone model;

[0013] Replace the improved multi-scale atrous spatial pyramid pooling module Imp_ASPP with the spatial pyramid pooling module SPP in YOLOv5;

[0014] The improved module CONV_NS for improving small target detection performance is used to replace the downsampling module in the YOLOv5 network;

[0015] S22: Take MS-YOLOv5 as the basic model, and replace the shuffle_res2net_s structure in each CSP_Bottle2Neck in the MS-YOLOv5 backbone network with the ATTShuffle_res2net_s structure, which is defined as CSPAT_Bottle2Neck;

[0016] The hybrid attention mechanism MSCA is integrated with the path aggregation network PANet in MS-YOLOv5.

[0017] S3: Introduce data resampling operation on the input data set and perform training. After optimal convergence, save the model parameters to obtain the optimal small target detection model;

[0018] S4: Call the optimal small target detection model, input the test image for detection, and the output is the recognition result.

[0019] Preferably, the preprocessing in S1 refers to: removing the Misc and DontCare categories in the existing data, merging the three categories of Van, Truck and Tram with very small sample numbers into the large category of Car, merging the Person_sitting category into the Pedestrain category, and standardizing all the images in the data set to finally obtain the images of the three large categories of Car, Cyclist and Pedestrain.

[0020] Preferably, ATTShuffle_res2net_s in S22 includes:

[0021] The input feature map of ATTShuffle_res2net_s is X∈R C×H×W , after extracting the hierarchical feature information, the output feature map y is finally obtained i , where i∈(1,2,3,4).

[0022] The output feature map is concatenated on the channel, and the feature information is fused and the channel dimension is reduced through a convolution operation with a convolution kernel size of 1×1. Then, the ATTention(.) attention mechanism is used [ATTention(.) uses CA(.)]. Finally, the feature map enhanced by attention is added to the initial input feature map at the element level.

[0023] Preferably, the hybrid attention mechanism MSCA in S22 includes:

[0024] The input feature map is X∈R C×H×W ;

[0025] After multiple convolution kernels with a size of 3×3, the depth of the separable dilated convolution DWATCon3v ×3 (.) Multi-scale feature extraction is performed, and the sampling rates are 1, 2, 3, and 4 respectively. The feature map after multi-scale feature extraction is X i ∈R C×H×W , where i∈(1,2,3,4);

[0026] Then, each feature map is added and fused at the element level;

[0027] The fused feature map is enhanced by CA attention mechanism, which is represented by CA(.);

[0028] The weight coefficient g of the height and width will be obtained h ,g w Multiply and send to the Softmax layer for processing;

[0029] The output obtained after Softmax processing is multiplied by the output feature maps of the previous four depth-separable dilated convolutions to obtain four output feature maps;

[0030] Finally, the four output feature maps are added and fused at the element level, and then added to the initial input feature map to implement the attention weighting operation on the original feature map, and the output feature map Y∈R is obtained. C×H×W .

[0031] Preferably, the improved multi-scale atrous spatial pyramid pooling module in S21 includes:

[0032] The input feature map of Imp_ASPP is X∈R C×H×W ;

[0033] The input feature map is divided into 5 branches. The first branch uses a convolution kernel size of 1×1 for convolution operation;

[0034] For the 2nd to 4th branches, depth-wise separable dilated convolutions with different sampling rates are used to obtain image context information of different scales in parallel, and a sampling channel shuffle operation is performed after each depth-wise separable dilated convolution. For the input of the branch, the output of the previous branch is concatenated with the input of the current branch as the input of the current branch. At the same time, for the output of the branch, the output of the previous branch is concatenated with the output of the current branch as the output of the current branch;

[0035] The fifth branch first passes through an adaptive mean pooling, then passes through a 1×1 convolution layer, and finally the feature map is sampled back from the 1×1 size to the original size;

[0036] The output X′ of the 5 branches i =R C×H×W , where i = (1, 2...5) performs Concat splicing at the channel level, and then uses Conv (1×1) to reduce the dimension to restore the original number of channels, and finally obtains the output feature map Y∈R C×H×W .

[0037] Preferably, the improved module CONV_NS for improving small target detection performance in S21 includes:

[0038] The input feature map of CONV_NS is X∈R C×H×W ;

[0039] The features are extracted by convolution with a kernel size of 3×3 and stride=1. The spatial size of the input feature map is not changed. The channel dimension is first reduced to 1 / 2 of the original, that is, C / 2, to obtain the feature Figure X ′=R C / 2×H×W ;

[0040] Features Figure X ′=R C / 2×H×W Perform a reshape operation. The feature map becomes X'=R C / 2×new_h×scale×new_w×scale , new_h and new_w are the height and width of the newly generated feature map, and scale is the scaling scale;

[0041] Transpose X' to X'' = R C / 2×scale×scale×new_h×new_w ;

[0042] After the transposition operation, the reshaping operation is performed to become Y∈R 2C×new_h×new_w , splicing is performed in the channel dimension, so that the channel dimension becomes twice as much as the original one, and the spatial size of the feature map is reduced to 1 / 2 of the original one.

[0043] Preferably, the specific steps of the resampling operation in S3 are:

[0044] Assign a learning weight to each category in the training set;

[0045] Calculate the number of each category in each picture in the training set, and then calculate the sampling weight of each picture based on the learning weight of each category;

[0046] According to the sampling weight of each picture, the picture index is generated by the random.choices() function, and the pictures are loaded according to each index each time the data is loaded.

[0047] Compared with the prior art, the present invention has at least the following advantages:

[0048] The present invention improves the improved MS-YOLOv5 as the basic framework (baseline). In order to improve the situation of missed detection of occluded targets in congested scenes and further improve the detection ability of small-scale targets, the attention mechanism in the field of computer vision is introduced, and a lightweight inverse bottleneck residual structure ATTShuffle_res2net_s structure based on channel reorganization that integrates the attention mechanism is proposed. The common attention mechanism CA is improved, and a new hybrid attention mechanism MSCA is proposed and integrated with the path aggregation network PANet of MS-YOLOv5. The problem of sample category imbalance in road scenes is solved by introducing data resampling operation to the KITTI data set. Finally, the experimental verification results show that MSAT-YOLOv5 further improves the detection ability of small-scale targets and improves the situation of missed detection in congested scenes. Finally, the accuracy (Map(@.5)) of MSAT-YOLOv5 on the test set is 88.66%, which is 1.21% higher than that of MS-YOLOv5 (baseline). Each category has been improved to a certain extent, including Car (0.65%), Cyclist (0.53%), and Pedestrian (2.35%). Compared with the original YOLOv5, the MAP (@.5) of MSAT-YOLOv5 has increased by 5.4%. While ensuring the overall accuracy improvement, the accuracy of small-scale and small-sample targets has also been improved, with Cyclist increased by 8.59% and Pedestrain increased by 7.63%. At the same time, compared with several other mainstream target detection methods and improved road scene target detection methods, it has better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is the MSAT-YOLOv5 network framework diagram.

[0050] Figure 2 This is the framework diagram of the MS-YOLOv5 method.

[0051] Figure 3 This is the structure diagram of ATTShuffle_res2net_s.

[0052] Figure 4 This is the CSPAT_Bottle2Neck module structure diagram.

[0053] Figure 5 This is the structure diagram of the MSCA attention mechanism

[0054] Figure 6 The figure shows the fusion strategy approach.

[0055] Figure 7This is the structure diagram of the improved multi-scale atrous spatial pyramid pooling (Imp_ASPP).

[0056] Figure 8 A diagram showing the computational process of depth-wise separable atrous convolution.

[0057] Fig. 9 To improve the structure diagram of the small target detection module (CONV_NS).

[0058] Fig.10 This is the MSAT-YOLOv5 detection effect diagram. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] See also Figure 1 and Figure 2 , the target detection method based on the hybrid attention mechanism includes the following steps:

[0061] S1: Obtain an existing data set and preprocess the data set. All preprocessed images constitute a training set.

[0062] S2: Construct a multi-scale road scene target detection model MSAT-YOLOv5, which is an improvement on the small target detection model MS-YOLOv5.

[0063] S21: The MS-YOLOv5 replaces the residual structure in CSP_BottleNeck in the YOLOv5 backbone feature extraction network with the lightweight inverse bottleneck residual module shuffle_res2net_s based on channel reorganization, which is defined as the MSL_backbone model.

[0064] The improved multi-scale atrous spatial pyramid pooling module Imp_ASPP is replaced with the spatial pyramid pooling module SPP in YOLOv5.

[0065] The improved module CONV_NS, which is used to improve the performance of small target detection, is used to replace the downsampling module in the YOLOv5 network.

[0066] S22: Take MS-YOLOv5 as the basic model, and replace the shuffle_res2net_s structure in each CSP_Bottle2Neck in the MS-YOLOv5 backbone network with the ATTShuffle_res2net_s structure, which is defined as CSPAT_Bottle2Neck;

[0067] The hybrid attention mechanism MSCA is integrated with the path aggregation network PANet in MS-YOLOv5.

[0068] S3: Introduce data resampling operation on the input data set and perform training. After optimal convergence, save the model parameters to obtain the optimal small target detection model.

[0069] S4: Call the optimal small target detection model, input the test image for detection, and the output is the recognition result.

[0070] Specifically, the preprocessing in S1 refers to: removing the Misc and DontCare categories in the existing data, merging the three categories of Van, Truck and Tram with a small number of samples into the large category of Car, merging the Person_sitting category into the Pedestrain category, and performing a standardization operation on all images in the data set, and finally obtaining images of the three large categories of Car, Cyclist, and Pedestrain. The present invention uses the currently more widely used traffic road scene data set KITTI, which includes a total of 9 categories: Car, Van, Truck, Tram, cyclist, Pedestrian, Person_sitting, Misc and DontCare.

[0071] See also Figure 3 Specifically, ATTShuffle_res2net_s in S22 includes:

[0072] The input feature map of ATTShuffle_res2net_s is X∈R C×H×W , after extracting the hierarchical feature information, the output feature map y is finally obtained i , where i∈(1,2,3,4).

[0073] The output feature map is concatenated on the channel, and the feature information is fused and the channel dimension is reduced through a convolution operation with a convolution kernel size of 1×1. Then, the ATTention(.) attention mechanism is used, and finally the feature map enhanced by attention is added to the original input feature map at the element level. [ATTention(.) uses CA(.)]

[0074] The expression of ATTShuffle_res2net_s is as follows:

[0075]

[0076]

[0077] Each CSP_Bottle2Neck in the MS-YOLOv5 backbone network is modified by replacing the residual structure in the CSP_BottleNeck in the YOLOv5 backbone feature extraction network with ATTShuffle_res2net_s, that is, the attention mechanism is only added to the CSP_Bottle2Neck with jump connections. The improved CSPAT_Bottle2Neck module structure is shown in the figure below: Figure 4 shown.

[0078] See also Figure 5 , the hybrid attention mechanism MSCA in S22 includes:

[0079] The input feature map is X∈R C×H×W ;

[0080] After multiple convolution kernels with a size of 3×3, the depth of the separable dilated convolution DWATCon3v ×3 (.) Multi-scale feature extraction is performed, and the sampling rates are 1, 2, 3, and 4 respectively. The feature map after multi-scale feature extraction is X i ∈R C×H×W , where i∈(1,2,3,4);

[0081] Then, each feature map is added and fused at the element level;

[0082] The fused feature map is enhanced by CA attention mechanism, which is represented by CA(.);

[0083] The weight coefficient g of the height and width will be obtained h ,g w Multiply and send to the Softmax layer for processing;

[0084] The output obtained after Softmax processing is multiplied by the output feature maps of the previous four depth-separable dilated convolutions to obtain four output feature maps;

[0085] Finally, the four output feature maps are added and fused at the element level, and then added to the initial input feature map to implement the attention weighting operation on the original feature map to prevent the gradient from dissipating due to the increase in network depth, and the output feature map Y∈R is obtained. C×H×W .

[0086] The expression of the hybrid attention mechanism MSCA is as follows:

[0087] X i =DWATConv 3x3 (X)(3)

[0088]

[0089]

[0090] refer to Figure 6 The fusion strategy diagram is shown in Figure 1. The fusion strategy structure diagram of MSCA and path aggregation network (PANet) is shown in Figure 1. Figure 6 As shown in the figure, it is named MSCA_PANet.

[0091] See also Figure 7 The improved multi-scale atrous spatial pyramid pooling module in S21 includes

[0092] The input feature map of Imp_ASPP is X∈R C×H×W ;

[0093] The input feature map is divided into 5 branches. The first branch uses a convolution kernel size of 1×1 for convolution operation;

[0094] For the 2nd to 4th branches, depth-wise separable dilated convolutions with different sampling rates are used to obtain image context information of different scales in parallel, and a sampling channel shuffle operation is performed after each depth-wise separable dilated convolution. For the input of the branch, the output of the previous branch is concatenated with the input of the current branch as the input of the current branch. At the same time, for the output of the branch, the output of the previous branch is concatenated with the output of the current branch as the output of the current branch;

[0095] The fifth branch first passes through an adaptive mean pooling, then passes through a 1×1 convolution layer, and finally the feature map is sampled back from the 1×1 size to the original size;

[0096] The output X′ of the 5 branches i =R C×H×W , where i = (1, 2...5) performs Concat splicing at the channel level, and then uses Conv (1×1) to reduce the dimension to restore the original number of channels, and finally obtains the output feature map Y∈R C×H×W .

[0097] The expression of Imp_ASPP is as follows:

[0098]

[0099]

[0100] All convolution operations in the formula include BN layers and activation functions, as shown in the reference Figure 7 As shown in the dotted rectangle. i represents the i-th branch of the input, Conv 1×1 (.) indicates a convolution with a kernel size of 1×1, which aims to halve the number of input channels entering the depthwise separable atrous convolution to prevent the model from being too wide. DWATConv(.) indicates a depthwise separable atrous convolution with a kernel size of 3, shuffle indicates a channel shuffle operation, concat(.) indicates concatenation at the channel level, pool(.) indicates the use of an adaptive mean pooling operation for the fifth branch, and up_sample(.) indicates an upsampling operation.

[0101] The number of parameters of the original ASPP module is 2644480, and the number of parameters of the improved multi-scale void space pyramid pooling Imp_ASPP module is 1576448, which is about 40% less. And under the verification of subsequent comparative experiments, under the premise of reducing the number of parameters, Imp_ASPP brings higher detection accuracy to the method than the original ASPP.

[0102] See also Figure 8 , the depthwise separable atrous convolution is composed of channel-by-channel atrous convolution (Depthwise Atrous Convolution, DAConv) and pointwise convolution (Pointwise Convolution, PWConv). The difference from the depthwise separable convolution is that the convolution in the first part is replaced by atrous convolution. The ordinary atrous convolution is decomposed into two steps, the number of parameters is reduced by 2 / 3, and the storage space is greatly saved.

[0103] See also Fig. 9 , the improved module CONV_NS in S21 for improving the small target detection performance includes:

[0104] The input feature map of CONV_NS is X∈R C×H×W ;

[0105] The features are extracted by convolution with a kernel size of 3×3 and stride=1. The spatial size of the input feature map is not changed. The channel dimension is first reduced to 1 / 2 of the original, that is, C / 2, to obtain the feature Figure X ′=R C / 2×H×W ;

[0106] Features Figure X ′=R C / 2×H×W Perform a reshape operation. The feature map becomes X'=R C / 2×new_h×scale×new_w×scale, new_h and new_w are the height and width of the newly generated feature map, and scale is the scaling scale. Here, scale = 2, which is equivalent to reducing the spatial size of the feature map to 1 / 2 of the original size;

[0107] Transpose X' to X'' = R C / 2×scale×scale×new_h×new_w ;

[0108] After the transposition operation, the reshaping operation is performed to become Y∈R 2C×new_h×new_w , splicing is performed in the channel dimension, so that the channel dimension becomes twice as much as the original one, and the spatial size of the feature map is reduced to 1 / 2 of the original one.

[0109] The expression of CONV_NS is as follows:

[0110] Y=reshape(transpose(reshape(Conv 3x3 (X)))) (8)

[0111] The CONV_NS module can implement a downsampling convolution operation similar to stride = 2 without losing information in the image. Subsequent experiments have shown that using the CONV_NS module to replace the downsampling convolution module in the YOLOv5 method framework can significantly improve the accuracy of small target detection based on road scenes.

[0112] Experimental verification:

[0113] Taking MS-YOLOv5 as the baseline model, the 2D object detection sub-dataset in the KITTI dataset is used for training and testing. The performance effects of various improved modules are analyzed and compared, and the detection results are displayed.

[0114] 1. Experimental Dataset

[0115] The KITTI dataset, a traffic road scene dataset that is currently widely used, is used to prove that the detection performance of MSAT-YOLOv5, which integrates various improved modules, is improved in traffic road scenes.

[0116] The scenes in the KITTI dataset are real traffic scenes (rural areas, highways, urban areas), which are collected by using vehicles equipped with multiple sensors. This dataset is commonly used for tasks such as 2D target detection, 3D target detection, and 3D tracking. This paper uses a sub-dataset of 2D target detection in KITTI. This sub-dataset includes 7481 training images with labeled information and 7518 test images without labeled information. The present invention uses 7481 images with labeled information for experiments. The number of samples of each category in KITTI after data preprocessing is shown in Table 1.

[0117] Table 2 Number of samples in each category in KITTI

[0118] category Car Pedestrain Cyclist Total quantity 32750 4709 1627 39086

[0119] 2. Evaluation Criteria

[0120] Since the experimental data set of the present invention is produced based on the format of the VOC data set, in order to evaluate the effectiveness of the designed target detection method, the evaluation index of the VOC data set is adopted.

[0121] IOU (Intersection over Union): refers to the ratio of the overlapping area between the detection box and the real box to the area of ​​the union.

[0122] TP (True Positive): When the actual value is a positive sample, the target detection model predicts that it is also a positive sample, which is recorded as TP.

[0123] FP (False Positive): The target detection model predicts a positive sample, but the actual true value is a negative sample, which is recorded as FP. Also known as false positive, the IOU between the predicted Bounding Box and the Ground Truth is less than the detection box threshold (positioning error) or the predicted type does not match the label type (classification error).

[0124] FN (False Negative): The target detection model predicts a negative sample, but the actual true value is a positive sample, which is recorded as FN. It is also called missed detection, which refers to the ground truth area that is not detected.

[0125] TN (True Negative): When the actual value is a negative sample, the target detection model predicts a negative sample. In target detection, TN is usually not paid attention to.

[0126] Precision is also called precision rate, which refers to the ratio of TP to the identified objects. It can be used to indicate the accuracy of the target detection model in predicting a certain category of targets.

[0127]

[0128] Recall, also known as recall rate, refers to the ratio of objects that are recognized as correct to objects that are actually correct.

[0129]

[0130] AP (Average Precision) refers to the area enclosed by the PR curve with recall as the horizontal axis and precision as the vertical axis. MAP (Mean AP) refers to the value obtained by averaging the AP of all categories after detecting the target using different thresholds. Most general target detections use MAP (@.5) and MAP (@.5:.95) as evaluation indicators. MAP (@.5) indicates the MAP size when the IOU threshold is 0.5, and MAP (@.5:.95) indicates the MAP size when the IOU threshold is in the discrete interval [0.5:0.05:0.95]. The present invention uses MAP (@.5) as an indicator to evaluate the performance of the target detection model.

[0131] 3. Experimental Setup

[0132] The experiment of the present invention uses the 2D target detection sub-dataset in the KITTI dataset, and divides the dataset into training set, validation set and test set according to the ratio of trainval:test (8:2) and train:val (9:1). The train dataset contains 5385 pictures, the val dataset contains 599 pictures, and the test dataset contains 1497 pictures. At the same time, since the pictures in the KITTI dataset are basically captured from the video frame by frame, most of the scenes in a video are similar. In order to prevent the occurrence of model overfitting, similar scenes should be avoided as much as possible when dividing the dataset, that is, they appear in both the training set and the test set. For all models, the experimental settings are: the image input size is 640×640, batch_size is set to 10, the training cycle (epoch) is 100, the loss function Loss is GIOU_Loss, the initial learning rate learning rate=0.001, the optimizer uses Adam, the momentum momentum=0.937, the weight decay term weight_decay=5e-4, the cosine annealing (cos) is used to adjust the learning rate in an orderly manner, and Mosica data enhancement is used for data set preprocessing. Mosica data enhancement is only turned on in the first 70% epochs, for example, 70 epochs will be turned on for 100 epochs. The last 30 epochs use ordinary data enhancement (random flipping, color gamut conversion, etc.). The reason is that the training images generated by Mosaic are far from the real distribution of natural images, and the images should be restored to the natural distribution state when the training is about to end.

[0133] 4. Experimental results analysis

[0134] According to the proposed structure diagram of ATTShuffle_res2net_s, the ATTention module is replaced with several mainstream attention mechanism methods (SE(.), ECA(.), CBAM(.), CA(.)) in the current computer vision field. And the replaced ATTShuffle_res2net_s structure is combined with the backbone network of MS-YOLOv5 according to the introduced fusion strategy. Experiments are carried out using the improved models, and finally the performance of each model is compared. The experimental results are shown in Table 2:

[0135] Table 2 Comparison of experimental results of various improved models

[0136]

[0137] As can be seen from the above table, several common attention mechanism methods in the current computer vision field are integrated into ATTShuffle_res2net_s, and compared with MS-YOLOv5 (baseline), the overall accuracy can be improved to a certain extent. At the same time, due to the addition of data resampling operation in the data set preprocessing link, the experimental results show that the detection effect of small-scale targets with small sample numbers can be improved. However, due to the fact that the under-sampling data sampling method is adopted for large-scale targets with large sample numbers, the accuracy of the Car category has decreased. However, after the ATTShuffle_res2net_s proposed in the present invention introduces the MS-YOLOv5 method, the accuracy of the Car category has been improved, and other categories have also been improved to a certain extent. Finally, ATTShuffle_res2net_s (CA) is selected as the final structure of the multi-scale information inverse bottleneck residual structure that integrates the attention mechanism. The accuracy on the test set (Map (@.5)) is 87.67%, which is 0.22% higher than the baseline. The accuracy of Car and Pedestrian is 93.00% and 82.14%, respectively, with an increase of 0.2% and 2.05%. Finally, val_loss is reduced to about 0.0421.

[0138] According to the reference Figure 6 The fusion strategy is shown in the figure, and MSCA is integrated into it. In order to compare the experiments, the CA attention mechanism method is also integrated into PANet. Using the improved model, experiments are carried out separately, and finally the performance of each model is compared.

[0139] The experimental results are shown in Table 3. It can be seen from the experimental results that the proposed MSCA attention mechanism is integrated into the path aggregation network to a certain extent. Under the premise of improving the overall detection accuracy, the accuracy of the Car category caused by data resampling has been restored to a level similar to that of the YOLOv5 method. At the same time, the detection accuracy of the other two categories (Cyclist and Pedestrian) has been further improved. Finally, the accuracy of the MSAT-YOLOv5 method on the test set (Map(@.5)) is 88.66%, which is 1.21% higher than the baseline. Each category has a certain improvement, which is 0.65%, 0.53%, and 2.35% respectively. The final val_loss is reduced to about 0.0417.

[0140] Table 3 Comparison of experimental results of various improved models

[0141]

[0142] The proposed MSAT-YOLOv5 is compared with the improved road scene target detection method. Compared with the original YOLOv5, the MAP(@.5) of MSAT-YOLOv5 is improved by 5.4%. While ensuring the overall accuracy improvement, the accuracy of small-scale and small-sample targets is also improved, with Cyclist improved by 8.59% and Pedestrain improved by 7.63%. At the same time, compared with several other mainstream target detection methods and improved road scene target detection methods, it has better results.

[0143] Table 3 Comparison experimental results of different methods and models

[0144]

[0145] The MS-YOLOv5 model and the MSAT-YOLOv5 model are used to detect the images in the test set, and the detection effect diagram is visualized. The detection effect diagram is shown in the reference Fig.10 As shown. The left column is the detection effect diagram of MSAT-YOLOv5, and the right column is the detection effect diagram of MS-YOLOv5. There are still some improvements to the several MS-YOLOv5 methods proposed earlier in this chapter: 1) Since the MS-YOLOv5 method uses data resampling operations, the accuracy of the Car category has decreased. 2) There is still room for improvement in the detection accuracy of small-scale targets with small sample sizes (Cyclist, Pedestrian). 3) When occlusion occurs, there will be missed detections. By comparing the detection effect diagrams, it can be found that the improved MSAT-YOLOv5 method has a better detection effect on road scene targets in the presence of occlusion, and also further improves the detection effect on long-distance small-scale targets.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. The target detection method based on the hybrid attention mechanism is characterized by: The steps include: S1: Obtain an existing data set and preprocess the data set. All preprocessed images constitute a training set. S2: Construct a multi-scale road scene target detection model MSAT-YOLOv5, which is an improvement on the small target detection model MS-YOLOv5; S21: The MS-YOLOv5 replaces the channel-reorganized lightweight inverse bottleneck residual module shuffle_res2net_s with the residual structure in CSP_BottleNeck in the YOLOv5 backbone feature extraction network, and is defined as the MSL_backbone model; Replace the improved multi-scale atrous spatial pyramid pooling module Imp_ASPP with the spatial pyramid pooling module SPP in YOLOv5; The improved multi-scale atrous spatial pyramid pooling module includes: The input feature map of Imp_ASPP is X∈R C×H×W ; The input feature map is divided into 5 branches. The first branch uses a convolution kernel size of 1×1 for convolution operation; For the 2nd to 4th branches, depth-separable dilated convolutions with different sampling rates are used to obtain image context information of different scales in parallel, and a sampling channel shuffle operation is performed after each depth-separable dilated convolution. For the input of the branch, the output of the previous branch is concatenated with the input of the current branch as the input of the current branch; at the same time, for the output of the branch, the output of the previous branch is concatenated with the output of the current branch as the output of the current branch; The fifth branch first passes through an adaptive mean pooling, then passes through a 1×1 convolution layer, and finally the feature map is sampled back from the 1×1 size to the original size; The output of the 5 branches is X' i =R C×H×W , where i = (1, 2...5) performs Concat splicing at the channel level, and then uses Conv (1×1) to reduce the dimension to restore the original number of channels, and finally obtains the output feature map Y∈R C×H×W ; The improved module CONV_NS for improving small target detection performance is used to replace the downsampling module in the YOLOv5 network; The improved module CONV_NS for improving the small target detection performance includes: The input feature map of CONV_NS is X∈R C×H×W ; The features are extracted by convolution with a kernel size of 3×3 and stride=1. The spatial size of the input feature map is not changed. The channel dimension is first reduced to 1 / 2 of the original, to C / 2, and the feature map X'=R is obtained. C / 2×H×W ; For feature map X' = R C / 2×H×W Perform reshaping operation; the feature map becomes X" = R C / 2×new_h×scale×new_w×scale , new_h and new_w are the height and width of the newly generated feature map, and scale is the scaling scale; Transpose X' to X'' = R C / 2×scale×scale×new_h×new_w ; After the transposition operation, the reshaping operation is performed to become Y∈R 2C×new_h×new_w , splicing is performed in the channel dimension, so that the channel dimension becomes twice as much as the original one, and the spatial size of the feature map is reduced to 1 / 2 of the original one; S22: Take MS-YOLOv5 as the basic model, and replace the shuffle_res2net_s structure in each CSP_Bottle2Neck in the MS-YOLOv5 backbone network with the ATTShuffle_res2net_s structure, which is defined as CSPAT_Bottle2Neck; The ATTShuffle_res2net_s includes: The ATTShuffle_res2net_s input feature map is X∈R C ×H×W , after extracting the hierarchical feature information, the output feature map y is finally obtained i , where i∈(1,2,3,4), X represents the input feature map, C is the input channel, H is the feature map height, and W is the feature map width; the output feature map is spliced ​​on the channel, and the feature information is fused and the channel dimension is reduced by a convolution operation with a convolution kernel size of 1×1, and then the CA(.) attention mechanism is used. Finally, the feature map enhanced by attention is added to the initial input feature map at the element level; The hybrid attention mechanism MSCA is integrated with the path aggregation network PANet in MS-YOLOv5. The hybrid attention mechanism MSCA includes: The input feature map is X∈R C×H×W ; After multiple convolution kernels with a size of 3×3, the depth-separable dilated convolution DWATConv 3×3 (.) Multi-scale feature extraction is performed, and the sampling rates are 1, 2, 3, and 4 respectively. The feature map after multi-scale feature extraction is X i ∈R C×H×W , where i∈(1,2,3,4); Then, each feature map is added and fused at the element level; The fused feature map is enhanced by CA attention mechanism, which is represented by CA(.); The weight coefficient g of the height and width h ,g w Multiply and send to the Softmax layer for processing; The output obtained after Softmax processing is multiplied by the output feature maps of the previous four depth-separable dilated convolutions to obtain four output feature maps; Finally, the four output feature maps are added and fused at the element level, and then added to the initial input feature map to implement the attention weighting operation on the original feature map, and the output feature map Y∈R is obtained. C×H×W ; S3: Introduce data resampling operation on the input data set and perform training. After optimal convergence, save the model parameters to obtain the optimal small target detection model; S4: Call the optimal small target detection model, input the test image for detection, and the output is the recognition result.

2. The object detection method based on the hybrid attention mechanism according to claim 1, characterized in that: The preprocessing in S1 refers to: removing the Misc and DontCare categories in the existing data KITTI, merging the three categories with very small sample numbers of Van, Truck and Tram into the large category of Car, merging the Person_sitting category into the Pedestrain category, and standardizing all images in the dataset to finally obtain images of the three large categories of Car, Cyclist, and Pedestrain.

3. The object detection method based on the hybrid attention mechanism according to claim 2, characterized in that: The specific steps of the resampling operation in S3 are: Assign a learning weight to each category in the training set; Calculate the number of each category in each picture in the training set, and then calculate the sampling weight of each picture based on the learning weight of each category; According to the sampling weight of each picture, the picture index is generated by the random.choices() function, and the pictures are loaded according to each index each time the data is loaded.

Images