A car boom detection method based on an improved YOLOv11 model
By improving the YOLOv11 model and depth map generation technology, the problems of low efficiency and low accuracy of traditional boom detection methods have been solved, realizing automated and intelligent monitoring of booms, adapting to complex construction environments, and reducing equipment costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA RAILWAY DESIGN GRP CO LTD
- Filing Date
- 2025-12-24
- Publication Date
- 2026-05-29
Smart Images
Figure CN122116249A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and specifically to a method for detecting automobile crane booms based on an improved YOLOv11 model. Background Technology
[0002] With the continuous development of urban infrastructure construction and modern engineering projects, the accurate monitoring of crane booms is particularly important in railway engineering projects involving railway lines, tracks and bridges. Accurate monitoring of the booms is not only related to operational safety, but also directly affects construction efficiency and the service life of the equipment.
[0003] Traditional methods for inspecting truck crane booms rely heavily on manual inspections, which suffer from low efficiency, low accuracy, heavy workload, and difficulty in real-time monitoring. The use of fixed sensors is often constrained by complex construction environments and cannot effectively adapt to different working scenarios. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a method for detecting automotive crane booms based on an improved YOLOv11 model. This method, by introducing depth map generation technology, an improved YOLOv11 model, and a loss function, effectively improves the detection accuracy of automotive crane booms, adapts to complex construction environments, and achieves automated and intelligent crane boom monitoring.
[0005] Therefore, the present invention adopts the following technical solution:
[0006] A method for detecting automotive crane booms based on an improved YOLOv11 model, comprising:
[0007] S1, collect panoramic images of the truck crane boom at different working heights, angles and positions under different weather and lighting conditions;
[0008] S2, manually mark and select the car crane boom in the panoramic image, and process the marked panoramic image using data augmentation technology to obtain a preprocessed panoramic image;
[0009] S3, input the preprocessed panoramic image into the DepthAnythingv2 pretrained model to obtain the depth map corresponding to the preprocessed panoramic image;
[0010] S4. Combine the preprocessed panoramic image and its corresponding depth map into a sample. The label of each sample is the position and size of the truck crane boom in the preprocessed panoramic image. Divide the dataset composed of multiple samples into training set and test set according to the proportion.
[0011] S5, constructing an improved YOLOv11 model, including a multi-source feature fusion module, an improved backbone network, an improved neck network, a detection head, and an optimal detection result output module connected in sequence, wherein:
[0012] The multi-source feature fusion module takes samples as input and outputs a fused feature map to the improved backbone network.
[0013] The improved backbone network includes a first convolutional layer, first to fourth feature extraction modules with the same structure, an SPPF module, and a deep space global attention mechanism module connected in sequence. Each feature extraction module includes a convolutional layer and a convolutional module structure module.
[0014] The improved Neck network includes a fifth, sixth, seventh, and eighth feature extraction module connected in sequence. The fifth and sixth feature extraction modules have the same structure, including an upsampling layer, a weighted fusion module, and a convolutional module structure module connected in sequence. The seventh and eighth feature extraction modules have the same structure, including a convolutional layer, a weighted fusion module, and a convolutional module structure module connected in sequence.
[0015] The input to the weighted fusion module in the fifth feature extraction module also includes the feature vector output by the third feature extraction module;
[0016] The input to the weighted fusion module in the sixth feature extraction module also includes the feature vector output by the second feature extraction module;
[0017] The input to the weighted fusion module in the seventh feature extraction module also includes the feature vector output by the fifth feature extraction module;
[0018] The input to the weighted fusion module in the eighth feature extraction module also includes the feature vector output by the improved backbone network;
[0019] S6, the improved YOLOv11 model is trained using the training set, and the trained model is tested using the test set;
[0020] S7: Collect panoramic images of the truck crane boom at different working heights, angles, and positions under different weather and lighting conditions, and input them into the DepthAnythingv2 pre-trained model to obtain the depth map corresponding to the panoramic image; input the panoramic image and its corresponding depth map into the model obtained in S6 to obtain the detection results of the truck crane boom.
[0021] In S2, the label for each annotated panoramic image includes the location and dimensions of the car crane boom.
[0022] In S5, the multi-source feature fusion module includes a channel-level weighted fusion module and a reshape module. The channel-level weighted fusion module uses a channel-level weighted fusion method to perform weighted feature fusion of the depth map and panoramic image in the sample to obtain the fused features.
[0023] The deep space global attention mechanism module described in S5 includes:
[0024] The first processing branch is used to process the feature vectors. Feature extraction is performed to capture the correlation between channels and reduce redundancy, resulting in feature vectors. ;
[0025] The second processing branch is used to process the feature vectors. Extract global information to obtain feature vectors ;
[0026] The third processing branch is used to process the feature vectors. Spatial attention modeling is performed to obtain feature vectors. ;
[0027] The first multiplication module is used to multiply the feature vectors. and eigenvectors Element-wise multiplication yields a vector. ;
[0028] The scaling module is used to adjust the vector Convert to a spatial attention map;
[0029] The second multiplication module is used to combine the spatial attention map and the feature vector. Perform element-wise multiplication;
[0030] The final addition module is used to combine the output of the second multiplication module with the feature vector. The features are added together to obtain the final output feature map, which serves as the output of the improved backbone network.
[0031] In the above technical solution:
[0032] The first processing branch includes two bottleneck operations performed sequentially. The input to the first bottleneck operation is a feature vector. The output of the second bottleneck operation is a feature vector. ;
[0033] The second processing branch includes a first convolution operation, a global pooling operation, a reshape operation, and a softmax operation performed sequentially;
[0034] The third processing branch includes a first CBS unit and a reshape operation connected in sequence. The first CBS unit includes a second convolution operation, a second batch normalization, and a Silu activation function operation performed in sequence.
[0035] In S5, each feature extraction module's convolutional module structure includes a first CBG unit, a splitting module, an asymmetric bottleneck structure group, a feature fusion module, and a second CBG unit connected in sequence, wherein:
[0036] The first CBG unit receives input from the convolutional module structure and outputs a feature vector. ;
[0037] The splitting module processes the feature vector Perform channel segmentation to divide the feature vector Uniformly split into feature vectors along the channel dimension. and eigenvectors eigenvectors The feature vector serves as the input to the asymmetric bottleneck structure group; Together with the output of the asymmetric bottleneck structure group, it serves as the input to the feature fusion module.
[0038] In the above technical solution:
[0039] The asymmetric bottleneck structure group includes first to third asymmetric bottleneck structure modules that are connected in sequence and have the same structure. Each asymmetric bottleneck structure module includes a first one-dimensional convolution, a first asymmetric convolution, a second asymmetric convolution, and a first summing module, an SE attention module, a second one-dimensional convolution, and a second summing module that are connected in sequence; wherein:
[0040] The input of the first one-dimensional convolution is the input of the asymmetric bottleneck structure module, and the output of the first one-dimensional convolution is used as the input of the first asymmetric convolution and the second asymmetric convolution, respectively. The outputs of the first asymmetric convolution and the second asymmetric convolution are used together as the input of the first summing module. The input of the second summing module also includes the output of the first asymmetric convolution, and the output of the second summing module is used as the output of the asymmetric bottleneck structure module.
[0041] In S5, the weighted fusion modules in the fifth to eighth feature extraction modules have the same structure. Any weighted fusion module performs a weighted sum of the two input feature vectors according to the weights corresponding to the two input feature vectors, and uses the weighted result as the output of the weighted fusion module.
[0042] In S6, the loss function of the improved YOLOv11 model for:
[0043] ;
[0044] In the formula, To focus the weights, they are dynamically determined based on the degree of shape matching between the predicted bounding box and the ground truth bounding box; Loss at the center point The weighting coefficients, For shape loss Weighting coefficients; IoU loss is used to measure the overlap between the predicted bounding box and the ground truth bounding box; It is the center point loss, which is the distance between the center points of the predicted box and the ground truth box; This is a shape loss function used to optimize the width and height differences between the predicted bounding box and the ground truth bounding box;
[0045] The focus weight The calculation formula is:
[0046] ;
[0047] In the formula, This is the adjustment coefficient for the focus factor. The width of the prediction box. The height of the predicted bounding box, The width of the actual bounding box. This is the height of the actual bounding box.
[0048] In the above technical solution:
[0049] The formula for calculating the IoU loss is:
[0050] ;
[0051] The center point loss The calculation formula is:
[0052] ;
[0053] In the formula, The distance between the center points The diagonal length of the minimum bounding box. The coefficient for exponential decay, This represents the coordinates of the center point of the true bounding box. Indicates the coordinates of the center point of the prediction box;
[0054] The shape loss The calculation formula is:
[0055] ;
[0056] In the formula, The standardization factor for the width. The coefficient is highly standardized. The exponential decay coefficient of the width is . It is a highly exponentially decaying coefficient.
[0057] Compared with the prior art, the present invention has the following beneficial effects:
[0058] 1. The improved YOLOv11 model, combined with depth maps, accurately identifies the position and status of the boom. The improved YOLOv11 model and loss function result in higher accuracy and recall of detection results, effectively reducing false positives and making it suitable for complex construction environments.
[0059] 2. Compared to traditional fixed sensors or manual inspection methods, using high-definition cameras for real-time monitoring eliminates the need for complex sensor equipment and extensive manual intervention, significantly reducing equipment installation and maintenance costs. Attached Figure Description
[0060] Figure 1 This is a flowchart of the automobile crane boom testing method of the present invention;
[0061] Figure 2 This is a schematic diagram of the structure of the improved YOLOv11 model of the present invention;
[0062] Figure 3 This is a schematic diagram of the structure of the convolution module in the improved YOLOv11 model of the present invention;
[0063] Figure 4 This is a schematic diagram of the asymmetric bottleneck structure group in the improved YOLOv11 model of the present invention.
[0064] Figure 5 This is a schematic diagram of the structure of the deep spatial global attention module in the improved YOLOv11 model of the present invention;
[0065] Figure 6 The test results are output by the improved YOLOv11 model of this invention. Detailed Implementation
[0066] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0067] Example
[0068] like Figure 1 As shown, a method for detecting automotive crane booms based on an improved YOLOv11 model includes the following steps:
[0069] S1, Data Acquisition:
[0070] High-precision cameras are used to capture panoramic images of the crane boom at different working heights, angles, and positions under different weather and lighting conditions; in locations where it is difficult to install high-precision cameras, other image acquisition devices (such as mobile phones) are used to capture panoramic images of the crane boom.
[0071] The high-precision cameras were installed at appropriate locations on the construction site; different weather conditions were used, including sunny, cloudy, and rainy days; and all panoramic images were RGB images. The panoramic images acquired under various actual working scenarios of the truck crane boom enable the improved YOLOv11 model to better adapt to complex construction environments, enhancing its stability and accuracy in practical applications.
[0072] S2, Data Preprocessing:
[0073] The crane booms in all panoramic images obtained by manually labeling are selected. The label of each labeled panoramic image includes the location and size of the crane boom. Data augmentation technology is used to process the labeled panoramic images to obtain pre-processed panoramic images.
[0074] The data augmentation techniques employed include: sequential affine transformation, Gaussian transformation, adaptive noise addition, mixture domain augmentation, and target region occlusion simulation. These data augmentation techniques effectively increase the diversity of the dataset, thereby improving the robustness and adaptability of the improved YOLOv11 model to different scenarios.
[0075] S3, Obtain the depth map:
[0076] The preprocessed panoramic image is input into the DepthAnythingv2 (DAv2) pre-trained model, and the output of the DAv2 pre-trained model is the depth map corresponding to the preprocessed panoramic image.
[0077] In the depth map output by the DAv2 pre-trained model, the coordinates are... depth value Represented as:
[0078] ;
[0079] In the formula, The coordinates in the preprocessed panoramic image are pixels, for The coordinates in the depth map output by the pre-trained model are The depth value;
[0080] The above operations introduce spatial depth information into the improved YOLOv11 model, enhancing its ability to perceive the crane boom at different locations and improving detection accuracy in complex environments. The aerial operation scenario of a crane boom is complex; detection using only RGB images is easily affected by background interference, making it difficult to distinguish targets. Introducing depth information clarifies the spatial relationship between the boom and the background. Considering the high cost of depth sensors, step S3 generates a depth map as additional feature input to the improved YOLOv11 model, eliminating the need for depth cameras, LiDAR, and other equipment. This allows the improved YOLOv11 model to achieve improved detection accuracy with lower computational power.
[0081] S4, Obtain the dataset and split it:
[0082] The preprocessed panoramic image and its corresponding depth map are combined into a sample, and the label of each sample is the position and size of the truck crane boom in the preprocessed panoramic image; multiple samples are combined into a dataset, and the samples in the dataset are divided into training set and test set in a ratio of 8.5:1.5.
[0083] In this embodiment, the dataset includes 6,000 samples.
[0084] S5, construct an improved YOLOv11 model, such as Figure 2 As shown, it includes: a multi-source feature fusion module, an improved backbone network, an improved neck network, a detection head, and an optimal detection result output module, all connected in sequence. Wherein:
[0085] The improved backbone network consists of: a first convolutional layer, first to fourth feature extraction modules with identical structures, an SPPF module, and a depth-space global attention mechanism module connected sequentially. The first feature extraction module is composed of a second convolutional layer and a first convolutional module structure; the second feature extraction module is composed of a third convolutional layer and a second convolutional module structure; the third feature extraction module is composed of a fourth convolutional layer and a third convolutional module structure; and the fourth feature extraction module is composed of a fifth convolutional layer and a fourth convolutional module structure.
[0086] The improved Neck network comprises a fifth, sixth, seventh, and eighth feature extraction module connected in sequence. The fifth feature extraction module comprises a first upsampling layer, a first weighted fusion module, and a fifth convolutional module structure connected in sequence; the sixth feature extraction module comprises a second upsampling layer, a second weighted fusion module, and a sixth convolutional module structure connected in sequence; the seventh feature extraction module structure comprises a sixth convolutional layer, a third weighted fusion module, and a seventh convolutional module structure connected in sequence; and the eighth feature extraction module structure comprises a seventh convolutional layer, a fourth weighted fusion module, and an eighth convolutional module structure connected in sequence.
[0087] The detection head includes: a first head module, a second head module, and a third head module.
[0088] The multi-source feature fusion module takes samples as input and outputs a fused feature map; the first convolutional layer takes the fused feature map as input and outputs a feature vector. The input to the second convolutional layer is The output is a feature vector. The input to the first convolutional module is the feature vector. The output is a feature vector. The input to the third convolutional layer is the feature vector. The output is a feature vector. The input to the second convolutional module is the feature vector. The output is a feature vector. The input to the fourth convolutional layer is the feature vector. The output is a feature vector. The input to the third convolutional module is the feature vector. The output is a feature vector. The input to the fifth convolutional layer is the feature vector. The output is a feature vector. The input to the fourth convolutional module is the feature vector. The output is a feature vector. The input to the SPPF module is a feature vector. The output is a feature vector. The input to the deep space global attention mechanism module is a feature vector. The output is a feature vector. ;
[0089] The input to the first upsampling layer is the feature vector. The output is a feature vector. The input to the first weighted fusion module is the feature vector. and eigenvectors The output is a feature vector. The input to the fifth convolutional module is the feature vector. The output is a feature vector. The input to the second upsampling layer is the feature vector. The output is a feature vector. The input to the second weighted fusion module is the feature vector. and eigenvectors The output is a feature vector. The input to the sixth convolutional module is the feature vector. and eigenvectors The output is a feature vector. The input to the sixth convolutional module is the feature vector. The output is a feature vector. The input to the first head module is the feature vector. The output is the first detection result.
[0090] The input to the sixth convolutional layer is the feature vector. The output is a feature vector. The input to the third weighted fusion module is the feature vector. and eigenvectors The output is a feature vector. The input to the seventh convolutional module is the feature vector. The output is a feature vector. The input to the second head module is the feature vector. The output is the second detection result.
[0091] The input to the seventh convolutional layer is the feature vector. The output is a feature vector. The input to the fourth weighted fusion module is the feature vector. and eigenvectors The output is a feature vector. The input to the eighth convolutional module is the feature vector. The output is a feature vector. The input to the third head module is the feature vector. The output is the third detection result.
[0092] The optimal detection result output module takes the first detection result, the second detection result, and the third detection result as inputs and outputs the optimal detection result.
[0093] The multi-source feature fusion module includes a channel-level weighted fusion module and a reshape module. The channel-level weighted fusion module uses a channel-level weighted fusion method to perform weighted feature fusion between the depth map in the sample and its corresponding preprocessed panoramic image to obtain the fused features. The coordinates in the fused features output by the channel-level weighted fusion module are... pixels The calculation formula is:
[0094]
[0095] In the formula, For pixels The weighting coefficients, Depth value Weighting coefficients, weighting coefficients and The improved YOLOv11 model is automatically optimized through backpropagation, and the weight coefficients are dynamically adjusted to ensure that the improved YOLOv11 model can adaptively select the best feature fusion method.
[0096] The reshape module flattens the spatial and channel dimensions of the fused features output by the channel-level weighted fusion module into a one-dimensional vector, thus obtaining the fused feature map.
[0097] The kernel size of the first to seventh convolutional layers is 3×3, and the stride is 2.
[0098] The structures of the first to eighth convolutional module structures are the same, such as... Figure 3 As shown, each includes: a first CBG unit, a splitting module, an asymmetric bottleneck structure group, a feature fusion module, and a second CBG unit connected in sequence;
[0099] In any convolutional module structure module:
[0100] The input to the first CBG unit is the input to the convolutional module structure, and the output feature vector of the first CBG unit is... The input to the splitting module and the output of the splitting module are feature vectors. and eigenvectors The splitting module processes the feature vectors Perform channel segmentation; divide the feature vector As input to the asymmetric bottleneck structure group, the output feature vector of the asymmetric bottleneck structure group is... The input to the feature fusion module is the feature vector. and eigenvectors The output of the feature fusion module is a feature vector. The input to the second CBG unit is the feature vector. The output of the second CBG unit serves as the output of the convolutional module structure module.
[0101] The first and second CBG units both include: sequential standard convolution operations, batch normalization (BatchNorm), and the Gelu activation function; the splitting module processes the feature vectors... Perform channel segmentation to divide the feature vector It is uniformly split into feature vectors along the channel dimension. and eigenvectors The asymmetric bottleneck structure group includes: a first asymmetric bottleneck structure module, a second asymmetric bottleneck structure module, and a third asymmetric bottleneck structure module connected in sequence. The structures of the first to third asymmetric bottleneck structure modules are identical. Figure 4As shown, each module includes: a first one-dimensional convolution, a first asymmetric convolution, a second asymmetric convolution, and sequentially connected first summing modules, SE (Squeeze-and-Excitation) attention modules, second one-dimensional convolutions, and second summing modules. Wherein:
[0102] The input of the first one-dimensional convolution is the input of the asymmetric bottleneck structure module, and the output of the first one-dimensional convolution is used as the input of the first asymmetric convolution and the second asymmetric convolution, respectively. The outputs of the first asymmetric convolution and the second asymmetric convolution are used together as the input of the first summing module. The input of the second summing module also includes the output of the first asymmetric convolution, and the output of the second summing module is used as the output of the asymmetric bottleneck structure module.
[0103] In this system, both the first and second one-dimensional convolutions have a kernel size of 1×1 and a stride of 1. The first one-dimensional convolution is used for channel dimensionality reduction (reducing the output to 1 / 4 of the input channels) to reduce the complexity of subsequent calculations. The second one-dimensional convolution is used for channel adjustment (reducing the output to 1 / 2 of the input channels). The first asymmetric convolution has a kernel size of 3×1 and a stride of 1, and focuses on extracting the contextual information of features in the vertical direction. The second asymmetric convolution has a kernel size of 1×3 and a stride of 1, and focuses on extracting the contextual information of features in the horizontal direction.
[0104] The first and second upsampling layers are implemented through deconvolution operations.
[0105] The first to fourth weighted fusion modules have the same structure. Each weighted fusion module performs a weighted sum of the two input feature vectors based on the weights corresponding to the two input feature vectors, and uses the weighted result as the output of the weighted fusion module.
[0106] The weights corresponding to the feature vectors represent the contribution of each input feature vector to the weighted result. These weights are adaptively adjusted or selected through a pre-defined strategy. The weighted fusion module avoids information redundancy and enhances the YOLOv11 model's focus on important features. The advantage of the weighted fusion module lies in dynamically adjusting the weights corresponding to the feature vectors, reducing the influence of unimportant features while strengthening key features, thus improving the expressive power and performance of the YOLOv11 model. This not only improves the efficiency of information utilization in the YOLOv11 model but also reduces the computational overhead caused by the high-dimensional feature space to some extent.
[0107] like Figure 5 As shown, the global attention mechanism module in deep space includes: a first processing branch, a second processing branch, a third processing branch, a first multiplication module, a scale adjustment module, a second multiplication module, and a final addition module.
[0108] Among them, the input feature vector of the deep space global attention mechanism module and its output feature vector The dimensions are the same, both being [C(512),H(25),W(20)].
[0109] The first processing branch is used to process the feature vectors. Feature extraction is performed to capture the correlation between channels and reduce redundancy, thereby enhancing semantic information. The first processing branch includes two bottleneck operations performed sequentially. The input of the first bottleneck operation is the feature vector. The output of the first bottleneck operation is used as the input to the second bottleneck operation, and the output of the second bottleneck operation is the feature vector. Each bottleneck operation is used to compress and extract feature information, and the feature vector... Encodes feature vectors Advanced information.
[0110] The second processing branch is used to process the feature vectors. Extracting global information and feature vectors Represented as C×H×W, where C is the number of channels, and H and W are the spatial dimensions of the feature map, the second processing branch includes: a first convolution operation, a global pooling operation, a reshape operation, and a softmax operation performed sequentially. Specifically, the kernel size of the first convolution operation is 3×3; the global pooling operation compresses the spatial dimension of the output of the first convolution operation to C×1×1, and is used to extract global information from each channel in the output of the first convolution operation; the reshape operation converts the output of the global pooling operation into a 1×C vector, which represents the global response of each channel; the softmax operation calculates the attention weights of each channel in the output of the reshape operation and uses this as the output feature vector of the second processing branch. The softmax operation assigns a different attention value to each channel, thereby capturing global channel dependencies.
[0111] The third processing branch is used to process the feature vectors. Spatial attention modeling is performed, and the third processing branch includes: a first CBS unit and a reshape operation connected sequentially; the first CBS unit includes: a second convolution operation, a second batch normalization, and a Silu activation function operation performed sequentially; the input of the first CBS unit is a feature vector. The output is a feature map containing spatial information, with dimensions C×H×W; the kernel size of the second convolution operation is 3×3; the reshape operation converts this feature map into a C×HW two-dimensional matrix, which serves as the output feature vector of the third processing branch. eigenvectors The features of each spatial location are unfolded.
[0112] The first multiplication module performs multiplication on the feature vector. and eigenvectors Perform element-wise multiplication to obtain a 1×HW vector. ,vector This represents the weight of each spatial location; the scaling module uses a reshape operation to reshape the vector. The input is converted into a 1×H×W spatial attention map; the second multiplication module combines the spatial attention map with the feature vector output from the first processing branch. Perform element-wise multiplication; the final addition module combines the output of the second multiplication module with the feature vector output of the first processing branch. The features are added together to obtain the final output feature map, which serves as the output of the deep spatial global attention mechanism module (the output of the improved backbone network). The final output feature map integrates channel information, spatial information, and their global dependencies.
[0113] The final addition module enables the improved YOLOv11 model to combine channel-level information with spatial-level attention information, thereby better capturing and weighting important regions in the feature map. The deep spatial global attention mechanism module makes the improved YOLOv11 model pay more attention to important spatial regions and reduce interference from background and unimportant regions.
[0114] The first, second, and third detection results all include: bounding box coordinates, category, and confidence level;
[0115] The optimal detection result output module uses a non-maximum suppression (NMS) strategy to select the optimal detection result from the first, second, and third detection results as the output of the improved YOLOv11 model.
[0116] In this embodiment, the dimensions of the preprocessed panoramic image in the sample are 1920×1080×3, the dimensions of the depth map in the sample are 1920×1080×1, the dimensions of the fused feature map are 800×640×4, and the feature vectors are... The dimensions are 400×320×32, and the feature vector is... The dimensions are 200×160×64, and the feature vector is... The dimensions are 100×80×128, and the feature vector is... The dimensions are 50×40×256, and the feature vectors are... The dimensions are 25×20×512; the feature vector The dimensions are 100×80×128, and the feature vector is... The dimensions are 50×40×256, and the feature vectors are... The dimensions are 25×20×512.
[0117] In the asymmetric bottleneck structure module: Asymmetric convolution is introduced, decomposing the standard N×N convolution into a concatenation of 1×N and N×1 convolutions. This enhances spatial awareness through orthogonal feature extraction while maintaining parameter efficiency. The first summing module element-wise adds the outputs of the first and second asymmetric convolutions. This first summing module is equivalent to a convolution with a larger receptive field but fewer parameters, capturing richer and more directional spatial features. The SE attention module strengthens inter-channel interaction. It "compresses" the global spatial information of the original features through global average pooling, then learns the importance weights (activations) of each channel through two fully connected layers, multiplying the importance weights by the original features to achieve adaptive strong inter-channel fusion. The second summing module effectively alleviates the gradient vanishing problem in deep networks, ensuring stable network training.
[0118] The original YOLOv11 model uses the concat method to fuse features from different layers. However, when dealing with multi-level and multi-scale features, the concat method may lead to information redundancy or ignore the relationships between features. The first to fourth weighted fusion modules (Fusion) are used to replace the concat method in the original YOLOv11 model to optimize the feature fusion effect of skip connections.
[0119] S6, Training the improved YOLOv11 model:
[0120] The improved YOLOv11 model is trained using the training set until the maximum number of iterations is reached or the loss function is minimized, thus obtaining the trained improved YOLOv11 model.
[0121] Among them, the loss function of the improved YOLOv11 model for:
[0122]
[0123] In the formula, To focus the weights, they are dynamically determined based on the shape matching degree between the predicted bounding box and the ground truth bounding box. The loss response for each sample is dynamically adjusted to ensure that the improved YOLOv11 model pays more attention to samples with greater differences. Loss at the center point The weighting coefficients, For shape loss Weighting coefficients, weighting coefficients and Used to control the impact of each loss item.
[0124] Focus on weight The calculation formula is:
[0125]
[0126] In the formula, This is the adjustment coefficient for the focus factor. The width of the prediction box. The height of the predicted bounding box, The width of the actual bounding box. This represents the height of the ground truth bounding box. When the predicted bounding box is close to the shape of the ground truth bounding box... Approaching 1; when the shape difference between the predicted bounding box and the ground truth bounding box is large. Greater than 1. The width and height of the predicted bounding box are obtained from the best detection results output by the improved YOLOv11 model, while the width and height of the ground truth bounding box are obtained from the labels of the samples.
[0127] IoU loss Used to measure the overlap between the predicted bounding box and the ground truth bounding box. The calculation formula is:
[0128]
[0129] Centroid Loss The distance between the center point of the predicted bounding box and the center point of the ground truth bounding box:
[0130]
[0131] In the formula, The distance between the center points The diagonal length of the minimum bounding box. The coefficient for exponential decay, This represents the coordinates of the center point of the true bounding box. This represents the coordinates of the center point of the predicted bounding box. Center point loss. It emphasizes the difference in the center position between the predicted box and the true box, and reduces the influence of samples far from the true center through exponential decay.
[0132] Shape loss Optimize for the width and height differences between the predicted and ground truth bounding boxes, and reduce shape loss. :
[0133]
[0134] In the formula, The standardization factor for the width. The coefficient is highly standardized. The exponential decay coefficient of the width is . The coefficient of exponential decay is the height. Shape loss. The difference between the width and height of the predicted bounding box and the true bounding box was measured, and the optimization was strengthened for samples with large shape errors through an exponential decay mechanism.
[0135] The loss function of the improved YOLOv11 model The YOLOv11 model can be optimized and improved more accurately by comprehensively considering the overlap, center position difference and shape difference in object detection.
[0136] The improved YOLOv11 model was tested using a test set, and the results were as follows: Figure 6 The test results shown are the coordinate vectors of the detected crane boom image. The format of the crane boom image coordinate vector is [horizontal coordinate, vertical coordinate, width, length, probability]. The crane boom image coordinate vector is visualized in the preprocessed panoramic image to obtain a panoramic image containing the detection box.
[0137] S7: Collect panoramic images of the truck crane boom at different working heights, angles, and positions under different weather and lighting conditions, and input them into the DepthAnythingv2 pre-trained model to obtain the depth map corresponding to the panoramic image; input the panoramic image and its corresponding depth map into the model obtained in S6 to obtain the detection results of the truck crane boom.
[0138] Comparative Example 1
[0139] A method for detecting automobile crane booms is basically the same as the embodiment, except that: step S3 in the embodiment is deleted so that the samples in the dataset only include the preprocessed panoramic images; the improved YOLOv11 model in step S5 in the embodiment is replaced with the original YOLOv11 model; and the loss function in step S6 in the embodiment is replaced with the loss function of the original YOLOv11 model.
[0140] Comparative Example 2
[0141] A method for detecting automobile crane booms is basically the same as the embodiment, except that step S3 in the embodiment is deleted, so that the samples in the dataset only include the preprocessed panoramic images.
[0142] Comparative Example 3
[0143] A method for detecting automobile crane booms is basically the same as the embodiment, except that the improved YOLOv11 model in step S5 of the embodiment is replaced with the original YOLOv11 model.
[0144] Comparative Example 4
[0145] A method for detecting automobile crane booms is basically the same as the embodiment, except that the loss function in step S6 of the embodiment is replaced with the loss function of the original YOLOv11 model.
[0146] The test results of Comparative Examples 1, 2, 3, and 4 and the embodiments of the present invention are shown in Table 1.
[0147] Table 1
[0148]
[0149] As shown in Table 1, the test results of the embodiments demonstrate significant improvements across multiple key metrics. Precision increased from 95.132% (Comparative Example 1) to 99.773%, indicating a substantial improvement in target detection accuracy and a significantly reduced false positive rate. Recall increased from 95.235% (Comparative Example 1) to 98.162%, with the 3% increase in recall indicating a significant reduction in the false negative rate of the improved YOLOv11 model in the embodiments of this invention, making it highly valuable for construction site applications and ensuring no false negatives in identifying most targets. The average accuracy (mAP50) improved from 98.172% (Comparative Example 1) to 99.44%, indicating that the improved YOLOv11 model has a significant performance improvement at an IoU threshold of 50%. In the calculation of mAP50-95, it also improved from 73.22% to 78.105%, further demonstrating that the improved YOLOv11 model is more robust at different IoU thresholds, especially with better detection accuracy at higher thresholds.
[0150] Overall, the improved YOLOv11 model in the embodiments of the present invention shows significant improvements in accuracy and a variety of average accuracy metrics, verifying the effectiveness of the improved YOLOv11 model in improving detection accuracy and reducing false alarms.
Claims
1. A method for detecting automotive crane booms based on an improved YOLOv11 model, characterized in that, include: S1, collect panoramic images of the truck crane boom at different working heights, angles and positions under different weather and lighting conditions; S2, manually mark and select the car crane boom in the panoramic image, and process the marked panoramic image using data augmentation technology to obtain a preprocessed panoramic image; S3, input the preprocessed panoramic image into the DepthAnythingv2 pretrained model to obtain the depth map corresponding to the preprocessed panoramic image; S4. Combine the preprocessed panoramic image and its corresponding depth map into a sample. The label of each sample is the position and size of the truck crane boom in the preprocessed panoramic image. Divide the dataset composed of multiple samples into training set and test set according to the proportion. S5, Construct an improved YOLOv11 model, which includes a multi-source feature fusion module, an improved backbone network, an improved neck network, a detection head, and an optimal detection result output module connected in sequence, wherein: The multi-source feature fusion module takes samples as input and outputs a fused feature map to the improved backbone network. The improved backbone network includes a first convolutional layer, first to fourth feature extraction modules with the same structure, an SPPF module, and a deep space global attention mechanism module connected in sequence. Each feature extraction module includes a convolutional layer and a convolutional module structure module. The improved Neck network includes a fifth, sixth, seventh, and eighth feature extraction module connected in sequence. The fifth and sixth feature extraction modules have the same structure, including an upsampling layer, a weighted fusion module, and a convolutional module structure module connected in sequence. The seventh and eighth feature extraction modules have the same structure, including a convolutional layer, a weighted fusion module, and a convolutional module structure module connected in sequence. The input to the weighted fusion module in the fifth feature extraction module also includes the feature vector output by the third feature extraction module; The input to the weighted fusion module in the sixth feature extraction module also includes the feature vector output by the second feature extraction module; The input to the weighted fusion module in the seventh feature extraction module also includes the feature vector output by the fifth feature extraction module; The input to the weighted fusion module in the eighth feature extraction module also includes the feature vector output by the improved backbone network; S6, the improved YOLOv11 model is trained using the training set, and the trained model is tested using the test set; S7: Collect panoramic images of the truck crane boom at different working heights, angles, and positions under different weather and lighting conditions, and input them into the DepthAnythingv2 pre-trained model to obtain the depth map corresponding to the panoramic image; input the panoramic image and its corresponding depth map into the model obtained in S6 to obtain the detection results of the truck crane boom.
2. The method for detecting an automobile crane boom according to claim 1, characterized in that: In S2, the label for each annotated panoramic image includes the location and dimensions of the car crane boom.
3. The method for detecting an automobile crane boom according to claim 1, characterized in that: In S5, the multi-source feature fusion module includes a channel-level weighted fusion module and a reshape module. The channel-level weighted fusion module uses a channel-level weighted fusion method to perform weighted feature fusion of the depth map and panoramic image in the sample to obtain the fused features.
4. The method for detecting an automobile crane boom according to claim 1, characterized in that, The deep space global attention mechanism module described in S5 includes: The first processing branch is used to process the feature vectors. Feature extraction is performed to capture the correlation between channels and reduce redundancy, resulting in feature vectors. ; The second processing branch is used to process the feature vectors. Extract global information to obtain feature vectors ; The third processing branch is used to process the feature vectors. Spatial attention modeling is performed to obtain feature vectors. ; The first multiplication module is used to multiply the feature vectors. and eigenvectors Element-wise multiplication yields a vector. ; The scaling module is used to adjust the vector Convert to a spatial attention map; The second multiplication module is used to combine the spatial attention map and the feature vector. Perform element-wise multiplication; The final addition module is used to combine the output of the second multiplication module with the feature vector. The features are added together to obtain the final output feature map, which serves as the output of the improved backbone network.
5. The method for detecting an automobile crane boom according to claim 4, characterized in that: The first processing branch includes two bottleneck operations performed sequentially. The input to the first bottleneck operation is a feature vector. The output of the second bottleneck operation is a feature vector. ; The second processing branch includes a first convolution operation, a global pooling operation, a reshape operation, and a softmax operation performed sequentially; The third processing branch includes a first CBS unit and a reshape operation connected in sequence. The first CBS unit includes a second convolution operation, a second batch normalization, and a Silu activation function operation performed in sequence.
6. The method for detecting an automobile crane boom according to claim 1, characterized in that: In S5, each feature extraction module's convolutional module structure includes a first CBG unit, a splitting module, an asymmetric bottleneck structure group, a feature fusion module, and a second CBG unit connected in sequence, wherein: The first CBG unit receives input from the convolutional module structure and outputs a feature vector. ; The splitting module processes the feature vector Perform channel segmentation to divide the feature vector Uniformly split into feature vectors along the channel dimension. and eigenvectors eigenvectors The feature vector serves as the input to the asymmetric bottleneck structure group; Together with the output of the asymmetric bottleneck structure group, it serves as the input to the feature fusion module.
7. The method for detecting an automobile crane boom according to claim 6, characterized in that: The asymmetric bottleneck structure group includes first to third asymmetric bottleneck structure modules that are connected in sequence and have the same structure. Each asymmetric bottleneck structure module includes a first one-dimensional convolution, a first asymmetric convolution, a second asymmetric convolution, and a first summing module, an SE attention module, a second one-dimensional convolution, and a second summing module that are connected in sequence; wherein: The input of the first one-dimensional convolution is the input of the asymmetric bottleneck structure module, and the output of the first one-dimensional convolution is used as the input of the first asymmetric convolution and the second asymmetric convolution, respectively. The outputs of the first asymmetric convolution and the second asymmetric convolution are used together as the input of the first summing module. The input of the second summing module also includes the output of the first asymmetric convolution, and the output of the second summing module is used as the output of the asymmetric bottleneck structure module.
8. The method for detecting an automobile crane boom according to claim 1, characterized in that: In S5, the weighted fusion modules in the fifth to eighth feature extraction modules have the same structure. Any weighted fusion module performs a weighted sum of the two input feature vectors according to the weights corresponding to the two input feature vectors, and uses the weighted result as the output of the weighted fusion module.
9. The method for detecting an automobile crane boom according to claim 1, characterized in that: In S6, the loss function of the improved YOLOv11 model for: ； In the formula, To focus the weights, they are dynamically determined based on the degree of shape matching between the predicted bounding box and the ground truth bounding box; Loss at the center point The weighting coefficients, For shape loss Weighting coefficients; IoU loss is used to measure the overlap between the predicted bounding box and the ground truth bounding box; It is the center point loss, which is the distance between the center points of the predicted box and the ground truth box; This is a shape loss function used to optimize the width and height differences between the predicted bounding box and the ground truth bounding box; The focus weight The calculation formula is: ； In the formula, This is the adjustment coefficient for the focus factor. The width of the prediction box. The height of the predicted bounding box, The width of the actual bounding box. This is the height of the actual bounding box.
10. The method for detecting an automobile crane boom according to claim 9, characterized in that: The formula for calculating the IoU loss is: ； The center point loss The calculation formula is: ； In the formula, The distance between the center points The diagonal length of the minimum bounding box. The coefficient for exponential decay, This represents the coordinates of the center point of the true bounding box. Indicates the coordinates of the center point of the prediction box; The shape loss The calculation formula is: ； In the formula, The standardization factor for the width. As a highly standardized coefficient, The exponential decay coefficient of the width, It is a highly exponentially decaying coefficient.