Cargo detection method for unmanned engineering vehicle in complex working environment
By introducing channel and spatial attention mechanisms into vehicle visual perception, combined with improved loss function, the robustness and real-time problems of existing object detection algorithms in complex environments are solved, and the accuracy and speed of cargo detection are improved.
Patent Information
- Application Number
- CN202411234190.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-03
AI Technical Summary
The existing object detection algorithms in vehicle visual perception lack robustness in complex environments, are difficult to deal with light and weather changes, and have low detection accuracy for occluded or overlapping targets, which cannot meet the real-time processing needs.
Using channel attention mechanism and spatial attention mechanism, multi-scale multi-channel image features are generated through the combination of backbone network and head network, and improved cross-entropy loss and composite cross-crossing loss are used to train, improving the robustness of the detection model. Sex and real-time.
It improves the robustness of the object detection model and the detection accuracy of occluded or overlapping objects, and enhances the real-time detection capability in complex environments.
Smart Images

Figure CN119068177A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection and vehicle visual perception, and specifically relates to a cargo detection method for an unmanned engineering vehicle in a complex operating environment. Background Art
[0002] Object detection in the field of vehicle visual perception is an important research direction, and has very broad application prospects in the fields of automatic operation of logistics vehicles and unmanned driving. In recent years, the development of deep learning, especially convolutional neural networks, has greatly promoted the progress of object detection technology. Deep learning models can automatically learn relevant features and perform complex pattern recognition through back propagation of parameters, which is more accurate and efficient than traditional object detection methods based on manual features (such as Haar features and HOG). Despite significant technological advances, object detection in vehicle visual perception still faces many challenges. For example, changes in environmental conditions (such as lighting and weather changes), different target types and postures, occlusion and overlap of targets, etc., all put forward higher requirements for detection accuracy.
[0003] Existing deep learning-based object detection research is usually divided into two-stage object detection (such as the R-CNN series) and single-stage object detection (such as the YOLO series). However, in the field of vehicle visual perception, many conventional object detection algorithms have some shortcomings: first, they lack robustness to changes in lighting and weather. Under different driving conditions, changes in the environment may affect image quality and thus detection accuracy; second, the detection accuracy of occluded or overlapping objects is low; finally, the vehicle vision system needs to process a large amount of image data in real time and respond quickly. Many conventional object detection algorithms may not be able to meet the real-time requirements in terms of processing speed. Summary of the invention
[0004] The present invention aims to solve the deficiencies of the above-mentioned prior art and proposes a cargo detection method for unmanned engineering vehicles in complex operating environments, in order to improve the robustness, accuracy and speed of target detection, thereby realizing cargo detection for unmanned engineering vehicles in complex operating environments.
[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:
[0006] The cargo detection method of an unmanned engineering vehicle in a complex operating environment of the present invention is also characterized in that it includes the following steps:
[0007] Step 1: Collect and preprocess the images of logistics cargo in complex operating environments to obtain a logistics cargo image dataset in complex operating environments. , where S irepresents the i-th logistics cargo image, and the size of each logistics cargo image is H×W×C, H is the height of the logistics cargo image, W is the width of the logistics cargo image, C is the number of channels of the logistics cargo image, N S is the total number of logistics cargo images;
[0008] Let S i middle The real annotation information of the cargo detection anchor frame is recorded as , and includes: location tag collection and a collection of category labels ,in, is the location label of the anchor box for the ath cargo detection, and =( , , , ),( , ) represents the coordinates of the center pixel of the a-th cargo detection anchor box; and are the height and width of the a-th cargo detection anchor frame respectively; Represents the category label of the a-th cargo detection anchor box;
[0009] Step 2: Build a backbone network and i Extract semantic information and generate multi-scale and multi-channel image features ;
[0010] Step 2.1, the backbone network consists of c1 CBS3×3 modules, c2 multi-scale feature extraction modules and a pyramid pooling SPPF module;
[0011] Step 2.2, the CBS3×3 module is composed of a convolutional layer Conv2d with a kernel of 3×3, a batch normalization layer and a SiLu activation function;
[0012] S i After being processed by c1 CBS3×3 modules in sequence, the single-scale feature map F of the cargo image is output i ;
[0013] Step 2.3, the multi-scale feature extraction module is composed of a CBS3×3 module and a CA-C2F module in sequence;
[0014] F i After being processed by c2 multi-scale feature extraction modules in sequence, each multi-scale feature extraction module outputs its own multi-scale feature map ,in, Indicates F iAfter the feature map output by the jth feature extraction module, The size is ,and ;in, for Height, for The width of for The number of channels, F i The feature map output by the j-1th feature extraction module Height, for The width of for The number of channels;
[0015] Step 2.4, the pyramid pooling SPPF module is composed of a CBS3×3 module, three maximum pooling layers, a splicing layer and a CBS1×1 module for controlling the number of output channels. After processing, the initial multi-scale fusion features are obtained as follows: , The size is , and with The sizes are consistent, among which, for Height, for The width of for The number of channels;
[0016] Step 3: Build the head network and use it for and Process and obtain S i The test results of each cargo in the
[0017] Step 3.1, the head network is composed of feature fusion module and It is composed of detection heads;
[0018] Step 3.2: When k=1, Input into the kth feature fusion module for processing, and the size is The k-th multi-scale fusion feature ;in, express Height, express Height, express The number of channels;
[0019] Step 3.3: c2 detection heads respectively detect the output of SPPF module and Multi-scale fusion features Processing is performed and the corresponding output is the predicted detection box classification result Detection box position results ,in, represents the classification result of the detection box output by the kth detection head, Represents the detection frame position result output by the kth detection head;
[0020] based on and , and use the improved cross entropy loss VFL Loss to construct the classification loss;
[0021] based on and , and use the composite intersection-over-union loss CIOU Loss and distribution focal length loss DFL Loss to construct the regression loss;
[0022] After weighted summation of classification loss and regression loss, we get the total loss function Loss;
[0023] Step 4: Use the gradient descent method to train the network composed of the backbone network and the head network, and calculate the total loss function Loss to update the network parameters until the total loss function Loss converges, thereby obtaining the trained cargo detection model, which is used to perform cargo detection on logistics images collected in complex operating environments.
[0024] The cargo detection method for an unmanned engineering vehicle in a complex operating environment described in the present invention is also characterized in that step 2.3 includes the following steps:
[0025] Step 2.3.1, when j=1, F i Input the CBS3×3 module in the j-th multi-scale feature extraction module for processing and output the preliminary feature map , The size is ,in, for Height, for The width of for The number of channels;
[0026] Step 2.3.2, the CA-C2F module After processing, a multi-scale feature map is obtained .
[0027] Furthermore, the CA-C2F module in step 2.3.2 is composed of a channel attention module, a CBS3×3 module with c3 residual connections, and a CBS1×1 module responsible for controlling the output dimension;
[0028] Step 2.3.2.1, when j=1, Input into the CA-C2F module in the j-th multi-scale feature extraction module, and Divided into Sub-features F ˜ i j [ 1 ] , F ˜ i j [ 2 ] , …, F ˜ i j [ e ] , …, F ˜ i j [ C i j , 1 ] ,in, F ˜ i j [ e ] for The e-th sub-feature of ; and F ˜ i j [ e ] The dimension is ,in, for F ˜ i j [ e ] Height, for F ˜ i j [ e ] Width;
[0029] Step 2.3.2.2: Use formula (1) to get the maximum attention vector The jth element of V i j , m a x [ e ] :
[0030] V i j , m a x [ e ] = M a x ( F ˜ i j [ e ] ) , e = 1 , 2 , ⋯ C ˜ i , F j (1)
[0031] Step 2.3.2.3, use formula (2) to get the average attention vector The jth element of V i j , a v e [ e ] :
[0032] V i j , a v e [ e ] = A v e ( F ˜ i j [ e ] ) , e = 1 , 2 , ⋯ C ˜ i , F j (2)
[0033] Step 2.3.2.4, the channel attention module will and Splice into a long After the attention vector is obtained, it is input into the multi-layer perceptron module MLP for processing, and the number of neurons is The output layer outputs the j-th channel attention weight ;
[0034] Step 2.3.2.5: and After corresponding multiplication on each channel, the j-th attention-weighted channel feature map is obtained ;
[0035] Step 2.3.2.6: The elements in are sorted in descending order according to their channel attention weights, and the first After the elements corresponding to the channels form a channel attention feature map, they are sequentially input into c3 residual connected CBS3×3 modules for processing to obtain the residual feature map ,in, Indicates The residual feature map output by the CBS3×3 module;
[0036] Step 2.3.2.7. and After concatenation in the channel dimension, input a channel number The feature map output by the j-th multi-scale feature extraction module is obtained by processing it in the CBS1×1 module ;
[0037] Step 2.3.2.8, when j=2,3,…, When the feature map output by the j-1th multi-scale feature extraction module is Input into the jth multi-scale feature extraction module for processing and output feature map , thus by The feature map output by the multi-scale feature extraction module .
[0038] Further, the step 3.2 includes the following steps:
[0039] Step 3.2.1, the kth feature fusion module is composed of the kth spatial attention-based upsampling module SA-UpSamp, the kth Concat module and the kth CA-C2F module in sequence;
[0040] Step 3.2.2: The kth spatial attention-based upsampling module SA-UpSamp is Processing is performed to obtain the kth upsampled feature map ;
[0041] Step 3.2.3, the kth Concat module will With Multi-scale feature maps After concatenation in the channel dimension, we get Multi-scale stitching features , The size is ,in, ;in, express The number of channels, express The number of channels;
[0042] Step 3.2.4 After being processed by the kth CA-C2F module, the kth multi-scale fusion feature is obtained. ;
[0043] Step 3.2.5: When the k-1th multi-scale fusion feature map With Multi-scale feature maps Input into the kth feature fusion module for processing to obtain the kth multi-scale fusion feature , The size is , and there is , , thus by The feature fusion module outputs Multi-scale fusion features ,in, express Height, express Width.
[0044] Furthermore, the kth spatial attention-based upsampling module SA-UpSamp in step 3.2.2 is composed of the kth CBS3×3 module and the kth spatial attention splicing module in sequence;
[0045] Step 3.2.2.1, the CBS3×3 module pair in the upsampling module SA-UpSamp of the kth feature fusion module After processing, the size is Preliminary multi-scale fusion features ;in, express Height, express The width of express The number of channels;
[0046] Step 3.2.2.2, the kth spatial attention splicing module is The length and width dimensions of Divided into Sub-features , , …, , …, ,in, express The sub-features of the elements of each channel on the hth row and wth column in , and each sub-feature is a length of A vector of
[0047] Step 3.2.2.3: Use formula (3) to get the maximum attention matrix The element in row h and column w A i j , m a x [ h , w ] :
[0048] A i j , m a x [ h , w ] = M a x ( Q ˜ i , h , w k − 1 ) , h = 1 , 2 , ⋯ H i , Q k − 1 ; w = 1 , 2 , ⋯ W i , Q k − 1 (3)
[0049] Step 3.2.2.4: Use formula (4) to get the average attention matrix The element in row h and column w A i j , a v e [ h , w ] :
[0050] A i j , a v e [ h , w ] = A v e ( Q ˜ i , h , w k − 1 ) h = 1 , 2 , ⋯ H i , Q k − 1 ; w = 1 , 2 , ⋯ W i , Q k − 1 (4)
[0051] Step 3.2.2.5: and Stitched into a size of After the attention matrix is obtained, it is input into a convolutional layer and a Sigmoid activation function for processing. The spatial attention weight , The size is ;in, express Height, express Width;
[0052] Step 3.2.2.6, according to The 4 channels are sequentially Divided into 4 sub-weight matrices , , , , and the four sub-weight matrices are respectively Multiply the corresponding elements of to get the corresponding 4 splicing sub-features , , , ; Then, using formula (5), we can get Upsampled feature map , The size is ;in, express Height, express Height, express The number of channels;
[0053] (5)
[0054] In formula (5), Represents the upsampled feature map The element at the hth row and wth column of .
[0055] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the cargo detection method, and the processor is configured to execute the program stored in the memory.
[0056] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the cargo detection method when the computer program is executed by a processor.
[0057] Compared with the prior art, the present invention has the following beneficial effects:
[0058] 1. The present invention adopts a channel attention mechanism to automatically assign different importance weights to each channel in the feature map, thereby helping the detection method to focus on more valuable feature channels and suppress irrelevant or noisy feature channels, thereby improving the robustness of the target detection model.
[0059] 2. The present invention proposes an upsampling module based on the spatial attention mechanism, which helps the detection method focus on important areas in the spatial dimension of the feature map, improves the detection capability of key areas, and improves the detection accuracy of occluded or overlapping targets.
[0060] 3. The present invention is based on a single-stage target detection method, which enhances the real-time performance of the detection method compared to a two-stage method. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] Figure 1 is an overall flow chart of the method of the present invention;
[0062] Figure 2 The network overall composition diagram of the method of the present invention;
[0063] Figure 3 CA-C2F module diagram of the method of the present invention;
[0064] Figure 4 SA-UpSamp module diagram of the method of the present invention. DETAILED DESCRIPTION
[0065] In this embodiment, a method for detecting cargo in an unmanned engineering vehicle in a complex operating environment is provided. Figure 1 As shown, the following steps are included:
[0066] Step 1: Collect and preprocess the images of logistics cargo in complex operating environments to obtain a logistics cargo image dataset in complex operating environments. , where S i represents the i-th logistics cargo image, and the size of each logistics cargo image is H×W×C, H is the height of the logistics cargo image, W is the width of the logistics cargo image, C is the number of channels of the logistics cargo image, N S is the total number of logistics cargo images; in this embodiment, the scenes of cargo images include cloudy days, sunny days, rainy days and occlusions, the total number of cargo images is 13536, and the scale of cargo images is unified as 640×640×3.
[0067] Let S i The real labeling information of the goods is recorded as ,and Include cargo detection anchor frames, where the coordinates of the center pixel of any a-th cargo detection anchor frame are ( , ), the height and width of the a-th cargo detection anchor frame are recorded as and , the position label of the a-th cargo detection anchor box ( , , , ) is recorded as , the category label of the a-th cargo detection anchor box is recorded as , hence, S i The location tag set is , S i The category label set is .
[0068] Step 2: Build a backbone network and i Extract semantic information and generate multi-scale and multi-channel image features ;
[0069] Step 2.1, the backbone network is composed of c1 CBS3×3 modules, c2 multi-scale feature extraction modules and a pyramid pooling SPPF module; in this embodiment, c1 is 2, c2 is 4, and the backbone network structure is as follows Figure 2 shown.
[0070] Step 2.2, the CBS3×3 module is composed of a convolutional layer Conv2d with a kernel of 3×3, a batch normalization layer and a SiLu activation function;
[0071] S i After being processed by c1 CBS3×3 modules in sequence, the single-scale feature map F of the cargo image is output i ;
[0072] Step 2.3, the multi-scale feature extraction module is composed of a CBS3×3 module and a CA-C2F module in sequence;
[0073] F i After being processed by c2 multi-scale feature extraction modules in sequence, each multi-scale feature extraction module outputs its own multi-scale feature map ,in, Indicates F i After the feature map output by the jth feature extraction module, The size is ,and ;in, for Height, for The width of for The number of channels, F i The feature map output by the j-1th feature extraction module Height, for The width of for The number of channels.
[0074] Step 2.3.1, when j=1, F i Input the CBS3×3 module in the j-th multi-scale feature extraction module for processing and output the preliminary feature map , The size is ,in, for Height, for The width of for The number of channels;
[0075] Step 2.3.2, the CA-C2F module is composed of a channel attention module, c3 residual connected CBS3×3 modules, and a CBS1×1 module responsible for controlling the output dimension; in this embodiment, c3 is 5, and the structure of the CA-C2F module is as follows: Figure 3shown.
[0076] Step 2.3.2.1, when j=1, Input into the CA-C2F module in the j-th multi-scale feature extraction module, and Divided into Sub-features F ˜ i j [ 1 ] , F ˜ i j [ 2 ] , …, F ˜ i j [ e ] , …, F ˜ i j [ C i j , 1 ] ,in, F ˜ i j [ e ] for The e-th sub-feature of ; and F ˜ i j [ e ] The dimension is ,in, for F ˜ i j [ e ] Height, for F ˜ i j [ e ] Width;
[0077] Step 2.3.2.2: Use formula (1) to get the maximum attention vector The jth element of V i j , m a x [ e ] :
[0078] V i j , m a x [ e ] = M a x ( F ˜ i j [ e ] ) , e = 1 , 2 , ⋯ C ˜ i , F j (1)
[0079] Step 2.3.2.3, use formula (2) to get the average attention vector The jth element of V i j , a v e [ e ] :
[0080] V i j , a v e [ e ] = A v e ( F ˜ i j [ e ] ) , e = 1 , 2 , ⋯ C ˜ i , F j (2)
[0081] Step 2.3.2.4, the channel attention module will and Splice into a long After the attention vector is obtained, it is input into the multi-layer perceptron module MLP for processing, and the number of neurons is The output layer outputs the j-th channel attention weight .
[0082] Step 2.3.2.5: and After corresponding multiplication on each channel, the j-th attention-weighted channel feature map is obtained ;
[0083] Step 2.3.2.6: The elements in are sorted in descending order according to their channel attention weights, and the first After the elements corresponding to the channels form a channel attention feature map, they are sequentially input into c3 residual connected CBS3×3 modules for processing to obtain the residual feature map ,in, Indicates The output of a CBS3×3 module.
[0084] Step 2.3.2.7. and After concatenation in the channel dimension, input a channel number The feature map output by the j-th multi-scale feature extraction module is obtained by processing it in the CBS1×1 module ;
[0085] Step 2.3.2.8, when j=2,3,…, When the feature map output by the j-1th multi-scale feature extraction module is Input into the jth multi-scale feature extraction module for processing and output feature map , thus by The multi-scale feature extraction module outputs a feature map ;
[0086] Advantages of the CA-C2F module: Before selecting channels, the attention of each channel feature map is calculated first, and then the feature maps of half of the channels with large attention are selected as the input of the next layer of the C2F module. The advantage of this is that it allows the network to pay more attention to some features with high importance, and better play the role of optimizing features in the subsequent residual connection module, thereby improving the robustness of the model.
[0087] Step 2.4, the pyramid pooling SPPF module is composed of a CBS3×3 module, three maximum pooling layers, a splicing layer and a CBS1×1 module for controlling the number of output channels. After processing, the initial multi-scale fusion features are obtained as follows: , The size is , and with The sizes are consistent, among which, for Height, for The width of for The number of channels.
[0088] Step 3: Build the head network and use it for and Process and obtain S i The test results of each cargo in the
[0089] Step 3.1, the head network is composed of feature fusion module and In this embodiment, the head network structure is as follows: Figure 2 shown.
[0090] Step 3.2: When k=1, Input into the kth feature fusion module for processing, and the size is The k-th multi-scale fusion feature ;in, express Height, express Height, express The number of channels.
[0091] Step 3.2.1, the kth feature fusion module is composed of the kth spatial attention-based upsampling module SA-UpSamp, the kth Concat module and the kth CA-C2F module in sequence;
[0092] Step 3.2.2, the kth spatial attention-based upsampling module SA-UpSamp is composed of the kth CBS3×3 module and the kth spatial attention splicing module in sequence; the structure of the SA-UpSamp module is as follows Figure 4 shown.
[0093] Step 3.2.2.1: The CBS3×3 module pair in the upsampling module SA-UpSamp of the kth feature fusion module After processing, the size is Preliminary multi-scale fusion features ;in, express Height, express The width of express The number of channels.
[0094] Step 3.2.2.2, the kth spatial attention concatenation module is The length and width dimensions of Divided into Sub-features , , …, , …, ,in, express The elements of each channel in the hth row and wth column are composed of sub-features, and each sub-feature is a length of A vector of
[0095] Step 3.2.2.3: Use formula (3) to get the maximum attention matrix The element in row h and column w A i j , m a x [ h , w ] :
[0096] A i j , m a x [ h , w ] = M a x ( Q ˜ i , h , w k − 1 ) , h = 1 , 2 , ⋯ H i , Q k − 1 ; w = 1 , 2 , ⋯ W i , Q k − 1 (3)
[0097] Step 3.2.2.4: Use formula (4) to get the average attention matrix The element in row h and column w A i j , a v e [ h , w ] :
[0098] A i j , a v e [ h , w ] = A v e ( Q ˜ i , h , w k − 1 ) h = 1 , 2 , ⋯ H i , Q k − 1 ; w = 1 , 2 , ⋯ W i , Q k − 1 (4)
[0099] Step 3.2.2.5: and Stitched into a size of After the attention matrix is obtained, it is input into a convolutional layer and a Sigmoid activation function for processing. The spatial attention weight , The size is ;in, express Height, express Width.
[0100] Step 3.2.2.6, according to The 4 channels are sequentially Divided into 4 sub-weight matrices , , , , and the four sub-weight matrices are respectively Multiply the corresponding elements of to get the corresponding 4 splicing sub-features , , , ; Then, using formula (5), we can get Upsampled feature map , The size is ;in, express Height, express Height, express The number of channels;
[0101] (5)
[0102] In formula (5), Represents the upsampled feature map The element at the hth row and wth column of .
[0103] Advantages of the SA- UpSamp module: The key to connecting the backbone network with the head network is the upsampling module. However, conventional interpolation-based upsampling modules may introduce pseudo pixels or unreal details in the output, which may have a negative impact on subsequent target detection tasks. Based on this, the present invention proposes a spatial attention-based upsampling module SA-UpSamp (a Spatial Attention-based Up-Sampling module). The SA- UpSamp module introduces a spatial attention mechanism. For the input feature map of the upsampling module, the channel attention of each position in the feature map is calculated. Then, the channel attention is used as the weight of each position of the feature map. During the upsampling process, each expanded element is calculated by the weighted sum of the normalized channel attention of the adjacent original elements and their corresponding elements. In this way, the obtained upsampled feature map not only enlarges the feature map according to the output size, but also contains the spatial attention features of the original feature map after expansion, so that the position features with high importance get more attention in the subsequent network layers, thereby improving the detection accuracy of the model for occluded goods.
[0104] Step 3.2.3, the kth Concat module will With Multi-scale feature maps After concatenation in the channel dimension, we get Multi-scale stitching features , The size is ,in, ;in, express The number of channels, express The number of channels;
[0105] Step 3.2.4 After being processed by the kth CA-C2F module, the kth multi-scale fusion feature is obtained. ;
[0106] Step 3.2.5: When the k-1th multi-scale fusion feature map With Multi-scale feature maps Input into the kth feature fusion module for processing to obtain the kth multi-scale fusion feature , The size is , and there is , , thus by The feature fusion module outputs Multi-scale fusion features ,in, express Height, express Width.
[0107] Step 3.3: c2 detection heads respectively detect the output of SPPF module and Multi-scale fusion features Processing is performed and the corresponding output is the predicted detection box classification result Detection box position results ,in, represents the classification result of the detection box output by the kth detection head, Represents the detection frame position result output by the kth detection head;
[0108] based on and , and use the deformable bounding box loss VFL Loss to construct the classification loss;
[0109] based on and , and use the composite intersection-over-union loss CIOU Loss and distribution focal length loss DFL Loss to construct the regression loss;
[0110] After weighted summation of classification loss and regression loss, the total loss function Loss is obtained.
[0111] Step 4: Use the gradient descent method to train the network composed of the backbone network and the head network, and calculate the total loss function Loss to update the network parameters until the total loss function Loss converges, thereby obtaining the trained cargo detection model, which is used to perform cargo detection on logistics images collected in complex operating environments.
[0112] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.
[0113] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.
[0114] In order to verify the effectiveness of the method of the present invention, in this embodiment, the network model designed above is experimented on the logistics cargo image dataset under the constructed complex operating environment, and compared with the current mainstream target detection method, as shown in Table 1.
[0115] Table 1 Performance comparison with mainstream methods in cargo detection accuracy
[0116]
[0117] In "Methods" in Table 1, "Faster R-CNN" and "YOLOv5" are the current mainstream anchor-based target detection benchmark methods, "FCOS" and "ObjectBox" are the current mainstream anchor-free target detection benchmark methods, and "Year" indicates the year in which the corresponding method in the table was proposed. "AP50" indicates the average accuracy when the IoU threshold is 0.5, that is, when the IoU between the predicted detection box and the true detection box is greater than or equal to 0.5, the detection result is considered correct. "mAP" is the mean average precision, which is a common performance evaluation index of the target detection model. Specifically, in the range of IoU thresholds from 0.5 to 0.95, the step size is 0.05 to calculate the average precision AP respectively, and then the mean of these values is taken as the evaluation result. In this way, the comprehensive performance of the target detection model under different IoU thresholds can be measured, so as to more comprehensively evaluate the detection accuracy. From the comparison results in the table, it can be seen that the method of the present invention has reached the highest level in both "AP50" and "mAP" evaluation indicators, which verifies the effectiveness of the method of the present invention in cargo detection under complex environments.
Claims
1. A cargo detection method for unmanned engineering vehicles in a complex operating environment, characterized in that: The steps include: Step 1: Collect and preprocess the images of logistics cargo in complex operating environments to obtain a logistics cargo image dataset in complex operating environments. , where S i represents the i-th logistics cargo image, and the size of each logistics cargo image is H×W×C, H is the height of the logistics cargo image, W is the width of the logistics cargo image, C is the number of channels of the logistics cargo image, N S is the total number of logistics cargo images; Let S i middle The real annotation information of the cargo detection anchor frame is recorded as , and includes: location tag collection and a collection of category labels ,in, is the location label of the anchor box for the ath cargo detection, and =( , , , ),( , ) represents the coordinates of the center pixel of the a-th cargo detection anchor box; and are the height and width of the a-th cargo detection anchor frame respectively; Represents the category label of the a-th cargo detection anchor box; Step 2: Build a backbone network and i Extract semantic information and generate multi-scale and multi-channel image features ; Step 2.1, the backbone network consists of c1 CBS3×3 modules, c2 multi-scale feature extraction modules and a pyramid pooling SPPF module; Step 2.2, the CBS3×3 module is composed of a convolutional layer Conv2d with a kernel of 3×3, a batch normalization layer and a SiLu activation function; S i After being processed by c1 CBS3×3 modules in sequence, the single-scale feature map F of the cargo image is output i ; Step 2.3, the multi-scale feature extraction module is composed of a CBS3×3 module and a CA-C2F module in sequence; F i After being processed by c2 multi-scale feature extraction modules in sequence, each multi-scale feature extraction module outputs its own multi-scale feature map ,in, Indicates F i After the feature map output by the jth feature extraction module, The size is ,and ;in, for Height, for The width of for The number of channels, F i The feature map output by the j-1th feature extraction module Height, for The width of for The number of channels; Step 2.4, the pyramid pooling SPPF module is composed of a CBS3×3 module, three maximum pooling layers, a splicing layer and a CBS1×1 module for controlling the number of output channels. After processing, the initial multi-scale fusion features are obtained as follows: , The size is , and with The sizes are consistent, among which, for Height, for The width of for The number of channels; Step 3: Build the head network and use it for and Process and obtain S i The test results of each cargo in the Step 3.1, the head network is composed of feature fusion module and It is composed of detection heads; Step 3.2: When k=1, Input into the kth feature fusion module for processing, and the size is The k-th multi-scale fusion feature ;in, express Height, express Height, express The number of channels; Step 3.3: c2 detection heads respectively detect the output of SPPF module and Multi-scale fusion features Processing is performed and the corresponding output is the predicted detection box classification result And the detection box position result ,in, represents the classification result of the detection box output by the kth detection head, Represents the detection frame position result output by the kth detection head; based on and , and use the improved cross entropy loss VFL Loss to construct the classification loss; based on and , and use the composite intersection-over-union loss CIOU Loss and distribution focal length loss DFL Loss to construct the regression loss; After weighted summation of classification loss and regression loss, we get the total loss function Loss; Step 4: Use the gradient descent method to train the network composed of the backbone network and the head network, and calculate the total loss function Loss to update the network parameters until the total loss function Loss converges, thereby obtaining the trained cargo detection model, which is used to perform cargo detection on logistics images collected in complex operating environments.
2. The cargo detection method for an unmanned engineering vehicle in a complex operating environment according to claim 1 is characterized in that: The step 2.3 comprises the following steps: Step 2.3.1, when j=1, F i Input the CBS3×3 module in the j-th multi-scale feature extraction module for processing and output the preliminary feature map , The size is ,in, for Height, for The width of for The number of channels; Step 2.3.2, the CA-C2F module After processing, a multi-scale feature map is obtained .
3. The cargo detection method for an unmanned engineering vehicle in a complex operating environment according to claim 2 is characterized in that: The CA-C2F module in step 2.3.2 is composed of a channel attention module, a CBS3×3 module with c3 residual connections, and a CBS1×1 module responsible for controlling the output dimension; Step 2.3.2.1, when j=1, Input into the CA-C2F module in the j-th multi-scale feature extraction module, and Divided into Sub-features , , …, , …, ,in, for The e-th sub-feature of ; and The dimension is ,in, for Height, for Width; Step 2.3.2.2: Use formula (1) to get the maximum attention vector The jth element of : (1) Step 2.3.2.3, use formula (2) to get the average attention vector The jth element of : (2) Step 2.3.2.4, the channel attention module will and Splice into a long After the attention vector is obtained, it is input into the multi-layer perceptron module MLP for processing, and the number of neurons is The output layer outputs the j-th channel attention weight ; Step 2.3.2.
5. and After corresponding multiplication on each channel, the j-th attention-weighted channel feature map is obtained ; Step 2.3.2.6: The elements in are sorted in descending order according to their channel attention weights, and the first After the elements corresponding to the channels form a channel attention feature map, they are sequentially input into c3 residual connected CBS3×3 modules for processing to obtain the residual feature map ,in, Indicates The residual feature map output by the CBS3×3 module; Step 2.3.2.
7. and After concatenation in the channel dimension, input a channel number The feature map output by the j-th multi-scale feature extraction module is obtained by processing it in the CBS1×1 module ; Step 2.3.2.8, when j=2,3,…, When the feature map output by the j-1th multi-scale feature extraction module is Input into the jth multi-scale feature extraction module for processing and output feature map , thus by The feature map output by the multi-scale feature extraction module .
4. The cargo detection method for an unmanned engineering vehicle in a complex operating environment according to claim 3 is characterized in that: The step 3.2 comprises the following steps: Step 3.2.1, the kth feature fusion module is composed of the kth spatial attention-based upsampling module SA-UpSamp, the kth Concat module and the kth CA-C2F module in sequence; Step 3.2.2: The kth spatial attention-based upsampling module SA-UpSamp is Processing is performed to obtain the kth upsampled feature map ; Step 3.2.3, the kth Concat module will With Multi-scale feature maps After concatenation in the channel dimension, we get Multi-scale stitching features , The size is ,in, ;in, express The number of channels, express The number of channels; Step 3.2.4 After being processed by the kth CA-C2F module, the kth multi-scale fusion feature is obtained. ; Step 3.2.5: When the k-1th multi-scale fusion feature map With Multi-scale feature maps Input into the kth feature fusion module for processing to obtain the kth multi-scale fusion feature , The size is , and there is , , thus by The feature fusion module outputs Multi-scale fusion features ,in, express Height, express Width.
5. The cargo detection method for unmanned engineering vehicles in a complex operating environment according to claim 4 is characterized in that: The kth spatial attention-based upsampling module SA-UpSamp in step 3.2.2 is composed of the kth CBS3×3 module and the kth spatial attention splicing module in sequence; Step 3.2.2.1, the CBS3×3 module pair in the upsampling module SA-UpSamp of the kth feature fusion module After processing, the size is Preliminary multi-scale fusion features ;in, express Height, express The width of express The number of channels; Step 3.2.2.2, the kth spatial attention splicing module is The length and width dimensions of Divided into Sub-features , , …, , …, ,in, express The sub-features of the elements of each channel on the hth row and wth column in , and each sub-feature is a length of A vector of Step 3.2.2.3: Use formula (3) to get the maximum attention matrix The element in row h and column w : (3) Step 3.2.2.4: Use formula (4) to get the average attention matrix The element in row h and column w : (4) Step 3.2.2.5: and Stitched into a size of After the attention matrix is obtained, it is input into a convolutional layer and a Sigmoid activation function for processing. The spatial attention weight , The size is ;in, express Height, express Width; Step 3.2.2.6, according to The 4 channels are sequentially Divided into 4 sub-weight matrices , , , , the four sub-weight matrices are respectively Multiply the corresponding elements of to get the corresponding 4 splicing sub-features , , , ; Then, using formula (5), we can get Upsampled feature map , The size is ;in, express Height, express Height, express The number of channels; (5) In formula (5), Represents the upsampled feature map The element at the hth row and wth column of .
6. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the cargo detection method according to any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cargo detection method according to any one of claims 1 to 5 are executed.
Citation Information
Cited By
Lightweight detection method and device for road small target recognition scene, and medium
CN122618352A
