Traffic sign detection device in complex environment based on improved YOLOv8
By improving the YOLOv8 model and combining a classification and denoising module, deformable convolution, and multi-scale aggregation attention mechanism, the accuracy and real-time performance of traffic sign detection in complex environments were addressed, and the model's detection performance under adverse weather and lighting conditions was improved.
Patent Information
- Application Number
- CN202511515350.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-22
- Publication Date
- 2026-01-13
AI Technical Summary
In complex environments, traditional and existing deep learning methods struggle to effectively identify traffic signs, especially under adverse weather conditions such as rain, snow, and fog, as well as under varying lighting conditions. Noise, occlusion, deformation, and scale changes lead to a decline in detection performance.
An improved YOLOv8 model is adopted, which combines a classification and denoising module (CDM), an efficient deformable convolutional module (E-DCN), and a multi-scale convergent attention mechanism (MAA), and uses an improved loss function WIoU v3 to enhance the detection capabilities of noise, deformation, and multi-scale traffic signs.
It improves the accuracy and real-time performance of traffic sign detection in complex environments, reduces noise interference, and enhances the detection capabilities for deformation and small targets, making it suitable for real-time application scenarios.
Smart Images
Figure CN121330645A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of automatic detection and recognition of traffic signs, and particularly relates to a multi-scale traffic sign detection device in complex environment based on YOLOv8. BACKGROUND
[0002] Traffic sign detection and recognition technology has attracted widespread attention in the field of computer vision, and this technology plays a crucial role in advanced driver assistance systems. Considering that traffic signs carry a large amount of information that is crucial for driving safety, this information may be overlooked due to driver fatigue or distraction, especially in adverse weather conditions. Therefore, in the task of traffic sign detection, accurately identifying these traffic signs is of great significance to ensure traffic safety and safe driving of vehicles.
[0003] Based on traditional traffic sign detection algorithms rely on hand-crafted feature extraction and simple classifiers, but these methods are usually limited by environmental changes (such as light, occlusion, sign wear) and computing power. With the development of deep learning technology, models based on convolutional neural networks have become mainstream due to their superior performance in feature extraction and pattern recognition. CNN has been widely used in computer vision, natural language processing and other fields in recent years due to its powerful feature extraction capability.
[0004] With the development of deep learning, traffic sign detection and recognition algorithms based on convolutional neural networks have been widely studied, and these algorithms can automatically locate and recognize traffic signs, greatly improving the recognition speed. However, traffic sign detection and recognition still faces the following challenges:
[0005] (1) In rainy, snowy, foggy and other complex weather conditions, the photos captured by the camera contain a large amount of noise.
[0006] (2) Under different lighting conditions, the color and saturation of traffic signs will change.
[0007] (3) A part of the traffic sign may be blocked by barriers, trees, or snow.
[0008] (4) In different shooting angles, the shape of the traffic sign may be distorted.
[0009] (5) Some traffic signs are too small to be recognized.
[0010] In recent years, many scholars have proposed theoretical frameworks and methods to enhance the detection and recognition performance of traffic signs in complex environments. One method, based on an improved YOLOv4, was proposed for detecting and recognizing multi-scale traffic signs in complex environments. This method designs an improved feature pyramid structure, employs an adaptive feature fusion module and a multi-scale feature transfer mechanism to enhance the representational power of the feature pyramid; however, it performs poorly when faced with lens distortion and traffic sign deformation. Another method for traffic sign recognition under complex lighting conditions was proposed, incorporating a novel adaptive image enhancement algorithm to improve image quality under complex illumination conditions; however, this method suffers from poor real-time performance and cannot meet the real-time traffic sign detection requirements in complex environments.
[0011] Traditional traffic sign detection methods primarily rely on color and shape. However, in complex environments, factors such as light intensity, sign occlusion, and lens distortion significantly limit the effectiveness of these methods, resulting in less than ideal detection performance. With technological advancements, deep learning-based traffic sign detection methods have demonstrated significantly better performance than traditional methods in handling complex environments. Currently, traffic sign detection performance in standard environments has reached a good level; however, in complex environments, such as rain, fog, snow, and changes in lighting, the visibility of traffic signs is significantly reduced, thus impacting the model's detection performance. Therefore, traffic sign detection in complex environments remains a significant challenge. Summary of the Invention
[0012] To address the issue of poor traffic sign detection capabilities in complex environments, this invention proposes a traffic sign detection device for complex environments based on an improved YOLOv8. Firstly, a Classify Denoizer Module (CDM) is proposed to classify and denoise noise in the image, reducing its impact on traffic sign detection. Simultaneously, an EfficientDeformable Convolution module (E-DCN) is proposed to handle the problem of insufficient receptive field at detection points when traffic signs are too small, thereby improving the model's detection capability for deformed traffic signs. Furthermore, a Multi-scale Aggregate Attention module (MAA) is proposed, which helps reduce the impact of global noise on detection and improves the detection capability for multi-scale traffic signs. Finally, to further improve the model's ability to distinguish between traffic signs and complex backgrounds, the WIoU loss is replaced with the original loss. This improvement to YOLOv8 effectively enhances the accuracy of traffic sign detection and recognition in complex environments. The specific technical solution is as follows:
[0013] A traffic sign detection device for complex environments based on an improved YOLOv8, comprising:
[0014] The Classification Denoising Module (CDM) is used to classify the input image according to noise type and selectively denoise it, outputting a denoised image.
[0015] The backbone network module is used to extract multi-scale features from the denoised image. The backbone network module contains, in sequence:
[0016] First convolutional block CBS1, second convolutional block CBS2, first C2f module, third convolutional block CBS3, first high-efficiency deformable convolutional module E-DCN1, fourth convolutional block CBS4, second high-efficiency deformable convolutional module E-DCN2, fifth convolutional block CBS5, third high-efficiency deformable convolutional module E-DCN3, spatial pyramid pooling module SPPF.
[0017] The neck network module is used to fuse the multi-scale features output by the backbone network module. The neck network module contains, in sequence:
[0018] First upsampling layer, first stitching layer, second C2f module, second upsampling layer, second stitching layer, third C2f module, sixth convolutional block CBS6, third stitching layer, fourth C2f module, seventh convolutional block CBS7, fourth stitching layer, fifth C2f module;
[0019] The head network module is used for target detection based on the fused multi-scale features. The head network module includes:
[0020] The first multi-scale aggregation attention module MAA1 receives the output of the third C2f module, the second multi-scale aggregation attention module MAA2 receives the output of the fourth C2f module, and the third multi-scale aggregation attention module MAA3 receives the output of the fifth C2f module. Three parallel detection heads are connected to MAA1, MAA2, and MAA3 respectively to output the final detection box and category.
[0021] The present invention has the following beneficial effects:
[0022] 1. This invention proposes a plug-and-play module—Classification and Denoising Module (CDM)—to reduce the impact of noise on target classification. CDM mainly consists of a Challenge Classifier and a Denoising Block. The Challenge Classifier extracts and classifies features from the original image based on the FastestDet network model, categorizing the image according to noise type. The Denoising Block uses four different denoising methods to reduce image noise caused by rain, snow, fog, and lens blur. If the detected challenge type is "no challenge," meaning there is no complex noise in the image, the Challenge Classifier will classify the image as a normal image and skip the Denoising Block, directly transmitting the image to the improved YOLOv8. Traditional detection methods do not consider the impact of weather and other factors in complex environments on traffic sign detection, or only consider the impact of one type of complex environment, resulting in unsatisfactory detection results. CDM considers the impact of multiple complex environments on traffic signs and performs denoising through the Denoising Block; therefore, CDM greatly reduces the impact of noise generated by complex environments on traffic sign detection.
[0023] 2. This invention proposes an efficient deformable convolutional module (E-DCN), which improves upon DCNv3 by using 3×3 convolutions, layer normalization, GELU, and linear layers. It also removes the additional LN-GELU layer and uses the original separable convolutional structure. The resulting E-DCN reduces computational cost and improves computational efficiency. Furthermore, the addition of E-DCN further enhances the model's ability to detect deformed traffic signs.
[0024] 3. This invention proposes a multi-scale convergent attention mechanism (MAA). Compared to existing coordinated attention mechanisms, MAA uses... A shared component for convolution, and added a Convolution and Parallel convolutional branches are used to improve processing efficiency. Furthermore, EMA's efficient attention mechanism enhances the model's focus on key features in the image, such as the symbols and colors of traffic signs. Especially in the presence of complex or occluded backgrounds, this focused attention effectively reduces the impact of background noise and interference, thereby improving detection accuracy. Simultaneously, EMA's design optimizes the computational flow, allowing for additional performance improvements without significantly increasing computational burden, making the overall model more suitable for real-time processing applications.
[0025] 4. This invention improves the loss function of the benchmark model, replacing the original CIOU Loss of YOLOv8 with WIoU v3 Loss. WIoU v3 combines some advantages of EIoU and SIoU, conforming to the design philosophy of a superior loss function. Furthermore, WIoU v3 employs a dynamic non-monotonic mechanism to evaluate anchor box quality, making the model pay more attention to anchor boxes of average quality, thus improving the model's ability to locate objects. In traffic sign detection tasks in complex environments, where traffic signs are small and image quality is poor, WIoU v3 can dynamically optimize the loss weights for small targets, improving the model's detection performance. Attached Figure Description
[0026] Figure 1 Improved model structure diagram;
[0027] Figure 2 CDM structure diagram;
[0028] Figure 3 MAA structure diagram. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above objectives, this invention adopts the following technical solution.
[0030] This invention proposes a traffic sign detection device for complex environments based on an improved YOLOv8 model. The device uses YOLOv8 as the baseline model and incorporates the proposed classification and denoising module CDM, the efficient deformable convolutional module E-DCN, the multi-scale convergent attention mechanism MAA, and an improved loss function. The improved model structure is as follows: Figure 1As shown, the improved YOLOv8 network model is mainly divided into three parts: Backbone, Neck, and Head. In the Backbone, the image first enters the Classification and Denoising Module (CDM) to perform classification and denoising processing. The processed image then enters a Convolutional Block (CBS) consisting of multiple convolutional layers, an Efficient Deformable Convolutional Network (E-DCN), and a C2f module for further processing, gradually scaling down the original 640×640×3 image to multiple different feature maps. This includes multiple CBS modules at different levels: CBS1 (320×320×64×w), CBS2 (160×160×128×w), CBS4 (80×80×256×w), CBS6 (40×40×512×w), and CBS8 (20×20×512×w×r), where w represents the width and r represents the scaling ratio. In addition, the C2f module is used for feature map concatenation and further convolutional processing, while the Efficient Deformable Convolutional Module (E-DCN) further improves feature representation capabilities. The backbone consists of the following modules connected in sequence: Layer 0 Classification and Denoising Module (CDM), Layer 1 Convolutional Block (CBS), Layer 2 Convolutional Block (CBS), Layer 3 C2f module, Layer 4 Convolutional Block (CBS), Layer 5 Efficient Deformable Convolutional Module (E-DCN), Layer 6 Convolutional Block (CBS), Layer 7 Efficient Deformable Convolutional Module (E-DCN), Layer 8 Convolutional Block (CBS), and Layer 9 Efficient Deformable Convolutional Module (E-DCN). The features processed by each module in the backbone are then fed into Layer 10 Spatial Pyramid Pooling Module (SPPF). By performing pooling operations of different sizes on the feature maps and integrating the results, a fixed-size output is obtained.
[0031] The features undergo multiple upsampling, concatenation, C2f, and CBS processing at the neck layer. The neck layer sequentially includes: layer 11 (upsample), layer 12 (concatenation), layer 13 (C2f module), layer 14 (upsample), layer 15 (concatenation), layer 16 (C2f module), layer 17 (CBS convolutional block), layer 18 (concatenation), layer 19 (C2f module), layer 20 (CBS convolutional block), layer 21 (concatenation), and layer 22 (C2f module). The features output from the Spatial Pyramid Pooling (SPPF) module are upsampled and then combined with the features from layer 7 (efficient pooling). The features output by the deformable convolutional module E-DCN are concatenated, then passed through the C2f module, and then upsampled. They are then concatenated with the features output by the 5th layer efficient deformable convolutional module E-DCN, then passed through the 16th layer C2f module, then through the 17th layer convolutional block CBS, and then concatenated with the features output by the 13th layer C2f module in the 18th layer feature concatenation concat. After passing through the 20th layer convolutional block CBS and the 21st layer feature concatenation concat, they are fused with the features output by the 22nd layer C2f module and the 10th layer spatial pyramid pooling module SPPF.
[0032] Finally, a multi-scale convergent attention mechanism (MAA) is added to the head to enhance the network's focus on features and improve detection capabilities. The final features are input into the detection module (Detect), which outputs the target's bounding box and category. The head includes MAA layers 23, 24, and 25. The 23rd layer MAA receives the output from the 16th layer C2f module, processes it, and outputs it to the detection module Detect. The 24th layer MAA receives the output from the 19th layer C2f module, processes it, and outputs it to another detection module Detect. The 25th layer MAA receives the output from the 22nd layer C2f module, processes it, and outputs it to yet another detection module Detect.
[0033] Figures (d)-(g) show the detailed structures of the Spatial Pyramid Pooling (SPPF), Detect, Convolutional Block (CBS), and C2f modules, respectively. The SPPF module performs convolution operations on the input features, executes consecutive max pooling (MaxPool), concatenates the features, and finally performs another convolution operation to output the features. The CBS module consists of three steps: two-dimensional convolution (Conv2d), two-dimensional batch normalization (BatchNorm2d), and execution of the SiLU activation function. In the C2f module, the input feature map is first convolved, then split. The split feature map is processed through multiple bottleneck layers, which are typically used to reduce the number of feature channels, making computation more efficient. The processed feature maps are then concatenated to form a new feature map. The Detect module is the final part of the network structure; it is responsible for object detection and outputting prediction results based on the feature maps obtained through convolution and feature fusion. The core task of this module is to perform target localization and classification by calculating bounding box regression loss (Bbox_Loss) and classification loss (Cls_Loss) after using convolutional blocks (CBS) and two-dimensional convolution (Conv2d), thereby achieving detection.
[0034] Select and collect datasets while performing preliminary processing on the datasets to obtain large-scale images containing traffic signs in various complex environments;
[0035] The Classification Denoising Module (CDM) preprocesses the image before feeding it into the improved detection model YOLOv8. YOLOv8 incorporates the CDM, Efficient Deformable Convolutional Network (E-DCN), Multi-Scale Converging Attention (MAA), and WIOU loss. The CDM includes a challenge classifier and multiple denoisers, such as... Figure 3 As shown;
[0036] Efficient Deformable Convolution (E-DCN) is added to the backbone of YOLOv8. This module is an adjustment and optimization of DCNv3, which has lower computational cost and better performance in detecting deformable targets.
[0037] The Multiscale Converging Attention (MAA) mechanism is an improvement on the Coordinating Attention (CA) mechanism, and MAA is added to the YOLOv8 header.
[0038] An improved loss function is used to optimize the training model. The WIOU loss is used to replace the original loss. The improved model can improve the localization ability of small targets.
[0039] A pre-trained challenge classifier is used and further optimized before training the challenge classifier and detection model.
[0040] Collecting the dataset involves the following steps:
[0041] Step 1.1: Select and collect the dataset. Select 10,000 images from Tsinghua-Tencent 100K (TT-100K) containing a large number of complex environments with real-world scenes as the basic dataset.
[0042] Step 1.2: In order to expand the dataset and increase the number of complex environment images in the dataset, 3000 additional images of complex environments were taken as expansion, and these images were labeled.
[0043] Step 1.3: Data augmentation. To further expand the scale of the dataset, this invention uses a random erasure algorithm and methods such as random flipping, random rotation, and random color transformation to expand the original 13,000 images to 39,000 images.
[0044] The CDM (Classification and Noise Reduction) module structure is as follows: Figure 2 As shown, it includes:
[0045] The challenge classifier uses the FastestDet model as a baseline framework and is adjusted and trained through methods such as transfer learning and fine-tuning. It can classify input images into 5 categories: no challenge, rainy day, snowy day, foggy day, and lens blur.
[0046] The noise reduction unit contains four noise reduction blocks: rain removal, snow removal, fog removal, and deblurring. These four modules correspond to the rain removal algorithm, snow removal algorithm, fog removal algorithm, and deblurring algorithm, respectively.
[0047] The merged output images, after being input into the classification and denoising module, are divided into 5 categories by the challenge classifier and denoised separately. The denoised images are then merged and the merged image is fed into YOLOv8.
[0048] The processing flow of Efficient Deformable Convolution (E-DCN) is as follows:
[0049] Standard convolution samples a set of pixels from the input feature map and then uses the convolution operation to compute the results. For each location in the input feature map... Standard convolution output feature map Each position on It can be represented as:
[0050] (1)
[0051] Where R represents the local region covered by the convolution kernel on the input, and w represents the weights of the convolution kernel. Indicates the relative offset position.
[0052] Introducing Deformable Convolutional Network (DCN), for each sampling location... The network will learn an offset. The calculation process of deformable convolution DCN is shown in the following formula:
[0053] (2)
[0054] The improved deformable convolutional network (DCN) introduces a multi-group mechanism, dividing the spatial aggregation process into G groups, each using a separate sampling offset. and modulation scale The deformable convolution DCNv3 obtained after expansion is shown in equation (3):
[0055] (3)
[0056] in, This represents the total number of aggregation groups. For the g-th group, This represents the position-independent projection weights of the group. In the g-th group, the first... The offset of each sampling point is normalized along dimension k by the softmax function. This represents the input feature map after slicing. Is with the first Grid sampling locations in the group The corresponding offset.
[0057] This invention removes the softmax layer and converts the modulation scalar into dynamic weights. Secondly, the original DCNv3 module uses... The complex subnetwork consisting of convolutions, layer normalization (LN), GELU, and linear layers is used to compute offsets and dynamic weights. This obviously increases computational overhead. Therefore, the extra LN-GELU layers are removed, and the original separable convolutional structure is used instead. Removing the LN-GELU layers and adopting the original separable convolution significantly reduces the model's computational complexity and number of parameters. Separable convolution greatly reduces parameters and computation by decomposing the traditional convolution into depthwise convolution and pointwise convolution.
[0058] The processing flow of the multi-scale convergent attention mechanism (MAA) is as follows: Figure 3 As shown, it includes the following:
[0059] Adding parallel branches, based on the construction concept of the CA module, MAA selects branches from the CA module. A shared component for convolution, and added a Convolution Conv and Convolutional branches are parallelized to improve processing efficiency.
[0060] Average pooling is performed. In MAA, for any given input feature map X, MAA divides it into multiple sub-features, i.e. In fact, MAA has three branches, including two... Branch and one Branches. Two of them. The branches are averaged along the X and Y directions, respectively, using X Avg Pool and Y Avg Pool.
[0061] Multi-scale feature fusion connects two average pooled features and then inputs them together. Convolution Conv. Then, The output of the convolution is decomposed into two vectors. Both vectors are fitted with a two-dimensional binomial distribution over the linear convolution using a non-linear sigmoid function. Furthermore, in In the branch, only a single one is stacked. Convolution is used to capture multi-scale feature representations.
[0062] To improve the richness of feature aggregation, in subsequent operations, the first step is to... Branches and Each branch performs a two-dimensional global average pooling (Avg Pool), which can be defined as follows:
[0063] (4)
[0064] Where H and W represent the height and width of the input feature map, respectively, and x c (i,j) represents the feature value of the input feature map at spatial position (i,j) in the c-th channel.
[0065] To improve computational efficiency, a natural nonlinear function, Softmax, is used after global average pooling to fit a linear transformation. Then... Branch output results and The branches are multiplied to obtain the first spatial attention map. Branch output results and The branches are multiplied to obtain a second spatial attention map. Finally, the two attention maps are added together and the sigmoid function is applied. In MAA, re-weighting weights different features or attention distributions, giving higher weights to more important features or regions, thereby improving the model's attention focus. Grouping reduces the number of features the network needs to process and decreases computational resource consumption by grouping similar features together. Furthermore, the grouping operation helps the model better handle regions with similar features or local consistency, allowing the network to learn richer representations within similar feature groups.
[0066] This invention uses WIOUv3 Loss instead of the original CIOU Loss in YOLOv8. WIOU v3 employs a reasonable gradient gain allocation strategy to dynamically optimize the weights of high-quality and low-quality anchor boxes in the loss, allowing the model to focus on average-quality samples and improving the overall performance of the model. The WIOU v3 formula is shown in equations (5-7). In equation (6)... and These are hyperparameters that can be adjusted to suit different models.
[0067] (5)
[0068] (6)
[0069] (7)
[0070] Where r represents the non-monotonic focusing coefficient, and β represents the caking factor, used to measure the quality of the anchor frame. This represents the IoU loss value for the current batch. express The moving average.
[0071] Pre-trained challenge classifiers include the following steps:
[0072] Step 6.1: Pre-train the challenge classifier. Introduce transfer learning and pre-train it on the ImageNet dataset to obtain model parameters that can recognize low-level features of images. The pre-training is divided into feature extraction stage and classification stage.
[0073] Step 6.2: Feature extraction stage: FastestDet uses an extremely lightweight CNN backbone network, employs depthwise separable convolution technology to reduce computation and parameter count, and uses a single-scale detector head for prediction to further simplify computation;
[0074] Step 6.3: In the feature classification stage, the Anchor-Free algorithm is adopted. The detection head directly predicts the center point, width and height of the target box and the class probability, eliminating the anchor box generation and matching steps, simplifying the post-processing operation. The dynamic positive and negative sample allocation strategy is adopted to finally output 5 labels (rain, snow, fog, blur and no noise).
[0075] Step 6.4: Challenge the classifier. Use Focal Loss as the classification loss and GIoU Loss as the regression loss. Use the Adam optimizer to optimize the network during training. Set the initial learning rate to 0.001. If the validation score does not improve within 3 epochs, reduce the learning rate by 0.5 times and train for 50 epochs.
[0076] Step 6.5: During training, the improved YOLOv8 model uses Adam as the optimizer to optimize the improved model. The initial learning rate is set to 0.001, the batch size is set to 64, and each training run is performed for 200 rounds.
Claims
1. A traffic sign detection device for complex environments based on an improved YOLOv8, characterized in that, include: The Classification Denoising Module (CDM) is used to classify the input image according to noise type and selectively denoise it, outputting a denoised image. The backbone network module is used to extract multi-scale features from the denoised image. The backbone network module contains, in sequence: First convolutional block CBS1, second convolutional block CBS2, first C2f module, third convolutional block CBS3, first high-efficiency deformable convolutional module E-DCN1, fourth convolutional block CBS4, second high-efficiency deformable convolutional module E-DCN2, fifth convolutional block CBS5, third high-efficiency deformable convolutional module E-DCN3, spatial pyramid pooling module SPPF. The neck network module is used to fuse the multi-scale features output by the backbone network module. The neck network module contains, in sequence: First upsampling layer, first stitching layer, second C2f module, second upsampling layer, second stitching layer, third C2f module, sixth convolutional block CBS6, third stitching layer, fourth C2f module, seventh convolutional block CBS7, fourth stitching layer, fifth C2f module; The head network module is used for target detection based on the fused multi-scale features. The head network module includes: The first multi-scale aggregation attention module MAA1 receives the output of the third C2f module, the second multi-scale aggregation attention module MAA2 receives the output of the fourth C2f module, and the third multi-scale aggregation attention module MAA3 receives the output of the fifth C2f module. Three parallel detection heads are connected to MAA1, MAA2, and MAA3 respectively to output the final detection box and category.
2. The apparatus according to claim 1, characterized in that, The classification noise reduction module (CDM) includes: The challenge classifier categorizes images into five types: no challenge, rain, snow, fog, and lens blur. A noise reduction block is coupled to the output of the challenge classifier. This block contains parallel rain removal, snow removal, fog removal, and blur removal blocks. The four noise reduction blocks are selected and activated based on the classification results. The activated path and the no-challenge bypass path are merged and connected to the input of the backbone network module.
3. The apparatus according to claim 1, characterized in that, The high-efficiency deformable convolution modules E-DCN1, E-DCN2, and E-DCN3 have the same structure and each includes the following steps in sequence: Step 3.1: Standard convolution samples a set of pixels from the input feature map and then uses a convolution operation to calculate the sampling results. For each position on the input feature map... Standard convolution output feature map Each position on Represented as: （1） Where R represents the local region covered by the convolution kernel on the input, and w represents the weights of the convolution kernel. Indicates the relative offset position; Step 3.2: Introduce Deformable Convolutional Network (DCN). In the Deformable Convolutional Network (DCN), for each sampling location... The network will learn an offset. The calculation process of deformable convolution DCN is shown in the following formula: （2） Step 3.3: Improved Deformable Convolution DCN. Deformable Convolution DCN introduces a multi-group mechanism, dividing the spatial aggregation process into G groups, each group using a separate sampling offset. and modulation scale The deformable convolution DCNv3 obtained after expansion is shown in Equation (3): （3） in, This represents the total number of aggregate groups; for the g-th group, Indicates the position-independent projective weights of the group; In the g-th group, the first... The offset of each sampling point is normalized along dimension k by the softmax function; This represents the input feature map after slicing; Is with the first Grid sampling locations in the group The corresponding offset.
4. The apparatus according to claim 1, characterized in that, The multi-scale convergent attention modules MAA1, MAA2, and MAA3 have the same structure and are all configured in parallel: The first 1×1 convolutional branch is used for feature encoding after average pooling along the X direction; The second 1×1 convolutional branch is used for feature encoding after average pooling along the Y direction; 3×3 convolutional branches are used to capture multi-scale context; And a feature fusion unit, which generates a spatial attention map by performing global average pooling, softmax, multiplication and sigmoid operations on the three-branch outputs, and then weights it back to the input features.
5. The apparatus according to claim 1, characterized in that, The Convolutional Block (CBS) consists of three steps: two-dimensional convolution (Conv2d), two-dimensional batch normalization (BatchNorm2d), and execution of the SiLU activation function.
6. The apparatus according to claim 1, characterized in that, In the neck network module: The input of the first upsampling layer is connected to the output of SPPF, and the input of the first splicing layer is connected to both the first upsampling layer and the output of E-DCN2, forming the first fusion path from top to bottom; The input of the second upsampling layer is connected to the output of the second C2f module, and the input of the second splicing layer is connected to both the second upsampling layer and the output of E-DCN1, forming a second fusion path from top to bottom. The sixth convolutional block, the third stitching layer, and the fourth C2f module are connected in series, and the third stitching layer additionally receives the output of the second C2f module, forming the first fusion path from bottom to top; The seventh convolutional block, the fourth stitching layer, and the fifth C2f module are connected in series, and the fourth stitching layer additionally receives the output of the fourth C2f module, forming a second fusion path from bottom to top.
7. The apparatus according to claim 1, characterized in that, The detection head performs target localization and classification by calculating bounding box regression loss Bbox_Loss and classification loss Cls_Loss after passing through convolutional block CBS and two-dimensional convolution Conv2d.
8. The apparatus according to claim 1, characterized in that, All convolutional blocks (CBS) in the backbone network module, neck network module, and head network module have the same structure, which includes, in sequence along the data flow direction: a two-dimensional convolutional layer Conv2d, a two-dimensional batch normalization layer BatchNorm2d, and a SiLU activation function layer.
9. The apparatus according to claim 1, characterized in that, Further includes: The pyramid pooling module SPPF performs convolution operations on the input features, executes continuous max pooling (MaxPool) and concatenates the features, and finally performs convolution operations again to output the features.
10. The apparatus according to claim 1, characterized in that, In the C2f module, the input feature map is first convolved, then split, and the split feature map is processed through multiple bottleneck layers.