Wheel part detection method based on multi-scale dynamic fusion

Abstract

The application provides a wheel part detection method based on multi-scale dynamic fusion, relates to the technical field of image recognition, and provides a wheel part detection process, which comprises an image preprocessing module, a backbone network module, a feature fusion module and a target detection module, and simultaneously provides MSA-T modules, DWFA modules and DAD-Head modules; the MSA-T module extracts local features of different receptive fields through multi-scale convolution kernels, a self-attention mechanism can capture long-distance dependence and global context information, the DWFA module adaptively adjusts a fusion ratio according to the contribution degree of different input features through a dynamic weight generation mechanism; the alignment and fusion of multi-scale features can capture the fine granularity of small targets while not losing high-level semantic information, and the DAD-Head module optimizes the feature requirements of classification tasks and regression tasks through a dynamic weight distribution mechanism.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, and specifically relates to a method for detecting wheel components based on multi-scale dynamic fusion. Background Technology

[0002] In vehicle detection, wheel component detection is a crucial step in ensuring vehicle qualification. The MSA-T module extracts local features from different receptive fields through multi-scale convolutional kernels, enabling the model to effectively capture size changes of wheel components at multiple scales. Simultaneously, combined with the Transformer's self-attention mechanism, MSA-T can capture long-distance dependencies and global contextual information between wheel components. This mechanism is particularly suitable for complex scenes where multiple components are closely arranged or partially occluded, effectively reducing false positives and false negatives and improving the robustness of object detection.

[0003] The DWFA module uses a dynamic weight generation mechanism to adaptively adjust the feature fusion ratio based on the contribution of different input features, achieving a balance between fine-grained information of small targets and high-level semantic information of large targets. Multi-scale feature alignment and fusion enable the DWFA module to perform well in detecting small parts while ensuring that the semantic understanding of large parts is not lost. Its dynamic characteristics allow the model to flexibly adjust the weights of features used in different scenarios, significantly enhancing the ability to detect wheel parts in complex backgrounds.

[0004] The DAD-Head module uses a dynamic weight allocation mechanism to optimize classification and regression tasks separately. This allows the classification branch to focus on improving class differentiation capabilities, while the regression branch more accurately fits the target bounding box. In wheel component detection scenarios, this design can more accurately identify the categories of different components and improve the accuracy of target localization. Especially in small target detection, the regression branch dynamically optimizes spatial detail features, greatly reducing the false negative rate. In complex backgrounds, the task-specific optimization of the classification branch can effectively reduce false positives.

[0005] The combination of the MSA-T module, DWFA module, and DAD-Head module not only improves the adaptability to targets of multiple sizes and categories, but also significantly enhances the detection accuracy and robustness of the model in complex scenes, thereby meeting the requirements of industrial detection for real-time performance, high accuracy, and high reliability. Summary of the Invention

[0006] This invention proposes a wheel component detection method based on multi-scale dynamic fusion. It includes three modules: MSA-T, DWFA, and DAD-Head. The MSA-T module extracts local features from different receptive fields using multi-scale convolutional kernels, and its self-attention mechanism captures long-distance dependencies and global contextual information. The DWFA module uses a dynamic weight generation mechanism to adaptively adjust the fusion ratio based on the contribution of different input features. The alignment and fusion of multi-scale features captures fine-grained details of small targets without losing high-level semantic information. The DAD-Head module uses a dynamic weight allocation mechanism specifically optimized for the feature requirements of classification and regression tasks.

[0007] This invention aims to improve upon image detection by providing a method for detecting wheel components based on multi-scale dynamic fusion, comprising the following steps:

[0008] S1. The wheel component image dataset is created by taking pictures of the wheel components using an industrial camera. The locations that need to be detected are labeled, and the locations that need to be detected are the spoke vents. Other objects are labeled as irrelevant targets. After the labeling is completed, the wheel component image dataset is obtained.

[0009] S2. Construct an image preprocessing module, including image scaling and normalization.

[0010] S3. Construct the backbone network modules, including the RepVGG module and the MSA-T module. The MSA-T module includes multi-scale convolutional feature extraction (MSC), Flatten operation, Transformer self-attention calculation, and multi-scale feature restoration.

[0011] S4. Construct a feature fusion module, including a DWFA module. The DWFA module includes feature alignment, dynamic weight generation, and dynamic weighted feature fusion to enhance the output features.

[0012] S5. Construct an object detection module, including a DAD-Head module and a post-processing module. The DAD-Head module includes dynamic feature task allocation, classification branch, and regression branch.

[0013] Preferably, in step S2, for the image preprocessing module, an image is input. The image is normalized and resized to obtain preprocessed input features. , It is a preprocessing function.

[0014] Preferably, in step S3, for the backbone network module, the input image features ,in, , These are the height and width of the feature, respectively. It refers to the number of channels, which is used to extract multi-level local features through RepVGG. ,in Including multi-scale features, image features The input is fed into the MSA-T module, which includes Multi-Scale Convolutional Feature Extraction (MSC), Flatten operation, Transformer self-attention calculation, multi-scale feature reconstruction, and image feature extraction. The input is fed into the MSC module, where three convolutional kernels of different scales are used to extract local features and generate three branch features. , , ,in, , , , representing the size of different convolution kernels, are used to merge the features extracted by different convolution kernels into multi-scale features by concatenating them channel by channel. ,in ;right use The convolution is adjusted to have d channels, and then flattened into a sequence. ,in ; will input Projection is a query Q, key K, value V. , , , in, , , It is a learnable projection matrix, used to calculate queries. s and keys dot product and through Normalization yields the attention weights. ,in It is an attention matrix, and the enhanced features are obtained by adding attention weights A and weighted values ​​V. Weight sum matrix Multiply, where The enhanced features are then converted from sequence form back into a two-dimensional feature map. ,in Residual connections are used to integrate input features and augmented features. ,in The output of the backbone network module contains multi-scale features and global self-attention enhancement information, which are then integrated with the input features and enhancement features using residual connections. ,in The output of the backbone network module contains multi-scale features and global self-attention enhancement information.

[0015] Preferably, in step S4, for the feature fusion module, the output features of the backbone network module are used as input features, and the MST-A module outputs multi-scale features. ,in It is a low-level, fine-grained feature. It is a mid-level semantic feature. It is a high-level semantic feature, , , The input is fed into the DWFA module, which adjusts features at different scales to the same resolution and number of channels. , , , ,in, It uses a downsampling operation with a convolution kernel of k=3. It uses bilinear interpolation for upsampling; the aligned features are concatenated and dynamic weights are generated. , Global pooling and fully connected layers are used to generate fusion weights. ,in, , representing the dynamic weights at each scale, satisfying The features are summed using dynamic weights. ,in, This is the fused output feature. The fused feature is adjusted for the number of channels through convolution and residual connections are added. The feature fusion module ultimately outputs the following features: .

[0016] Preferably, in step S5, for the target detection module, the feature fusion module outputs features. The features input into the DAD-Head module contain both spatial information and the contextual semantic information required for classification and regression. The DAD-Head module uses a dynamic weighting mechanism to... Assigned to two task branches: classification and regression, utilizing global pooling and Convolution generates feature weights for classification and regression tasks. , It is a channel-by-channel weight. It is the Sigmoid activation function, used to normalize to [0,1], based on the generated weights, Weighting is applied separately to the classification and regression branches. , , It is a channel-by-channel dot product. These are features used for classification. These are features used for regression; the classification branch is responsible for predicting the class probability of each pixel. ,in K is the number of target categories, and each pixel... The probability for the k-th class is ,in This is the Logit value for the k-th class. For each pixel, the index of the predicted class is... Confidence level is The regression branch predicts the bounding box parameters corresponding to each pixel. ,in Each pixel The output is ,in , Here, represents the coordinates of the bounding box center point, and w and h represent the width and height of the bounding box. This maps the bounding box parameters from the feature map to the original image. , , , ,in , The original image's height and width are used for post-processing, filtering out images with confidence scores below a threshold. The prediction box, , for For boxes with high confidence, they are sorted from highest to lowest confidence, and the intersection-union ratio (IUU) between candidate boxes is calculated. , For intersection and union ratio Only retain the boxes with higher confidence levels. , for The final output is Where N is the number of targets after post-processing, and each target object contains a category index. Confidence level Bounding box parameters .

[0017] Compared with the prior art, the present invention has the following technical effects:

[0018] The technical solution provided by this invention proposes an MSA-T module, a DWFA module, and a DAD-Head module. The MSA-T module extracts local features from different receptive fields through multi-scale convolutional kernels, and its self-attention mechanism can capture long-distance dependencies and global contextual information. The DWFA module adaptively adjusts the fusion ratio according to the contribution of different input features through a dynamic weight generation mechanism. The alignment and fusion of multi-scale features capture the fine-grained details of small targets without losing high-level semantic information. The DAD-Head module is specifically optimized for the feature requirements of classification and regression tasks through a dynamic weight allocation mechanism. Attached Figure Description

[0019] Figure 1 This is a flowchart of the wheel component testing process provided by the present invention.

[0020] Figure 2 This is the backbone network structure diagram provided by the present invention.

[0021] Figure 3 This is a feature fusion structure diagram provided by the present invention.

[0022] Figure 4 This is a diagram of the target detection structure provided by the present invention. Detailed Implementation

[0023] This invention proposes a wheel component detection method based on multi-scale dynamic fusion. It includes three modules: MSA-T, DWFA, and DAD-Head. The MSA-T module extracts local features from different receptive fields using multi-scale convolutional kernels, and its self-attention mechanism captures long-distance dependencies and global contextual information. The DWFA module uses a dynamic weight generation mechanism to adaptively adjust the fusion ratio based on the contribution of different input features. The alignment and fusion of multi-scale features captures fine-grained details of small targets without losing high-level semantic information. The DAD-Head module uses a dynamic weight allocation mechanism specifically optimized for the feature requirements of classification and regression tasks.

[0024] Please see Figure 1 As shown in the example, the wheel component detection method based on multi-scale dynamic fusion in this application is as follows:

[0025] S1. The wheel component image dataset was created by taking pictures of the wheel components using an industrial camera, resulting in 3,000 images. The air holes in the wheel components were located, and the air holes on the spokes were the locations to be detected. Other objects were labeled as irrelevant targets. After the labeling was completed, the wheel component image dataset was obtained, totaling 5,000 images. The dataset and validation set were divided in a 7:3 ratio.

[0026] S2. Construct an image preprocessing module, including image scaling and normalization.

[0027] S3. Construct the backbone network modules, including the RepVGG module and the MSA-T module. The MSA-T module includes multi-scale convolutional feature extraction (MSC), Flatten operation, Transformer self-attention calculation, and multi-scale feature restoration.

[0028] S4. Construct a feature fusion module, including a DWFA module. The DWFA module includes feature alignment, dynamic weight generation, and dynamic weighted feature fusion to enhance the output features.

[0029] S5. Construct the target detection module, including the DAD-Head module and the post-processing module. The DAD-Head module includes dynamic feature task allocation, classification branch, and regression branch.

[0030] The image is input into the image preprocessing module, where it is scaled and normalized. The image features are then input into the backbone network module, where the MSA-T module uses multi-scale convolutional kernels to extract local features from different receptive fields. The self-attention mechanism captures long-distance dependencies and global contextual information. The DWFA module's dynamic weight generation mechanism adaptively adjusts the fusion ratio based on the contribution of different input features. Next, multi-scale feature alignment and fusion are performed to capture fine-grained details of small targets without losing high-level semantic information. Finally, the DAD-Head module uses a dynamic weight allocation mechanism to optimize for the feature requirements of classification and regression tasks. The final output image is then generated.

[0031] Furthermore, in step S2, for the image preprocessing module, an image is input. The image is normalized and resized to obtain the preprocessed input features: This is a preprocessing function. H, W, and C represent the height, width, and number of channels of the input feature. The height and width are both 640, and the number of channels is 224.

[0032] Furthermore, in step S3, the structure of the backbone network module is as follows: Figure 2 As shown, input image features ,in, , These are the height and width of the feature, respectively. It refers to the number of channels, which is used to extract multi-level local features through RepVGG. ,in Including multi-scale features, image features The input is fed into the MSA-T module, which includes Multi-Scale Convolutional Feature Extraction (MSC), Flatten operation, Transformer self-attention calculation, multi-scale feature reconstruction, and image feature extraction. The input is fed into the MSC module, where three convolutional kernels of different scales are used to extract local features and generate three branch features. , , ,in, , , , representing the size of different convolution kernels, are used to merge the features extracted by different convolution kernels into multi-scale features by concatenating them channel by channel. ,in ;right use The convolution is adjusted to have d channels, and then flattened into a sequence. ,in ; will input Projection is a query Q, key K, value V. , , , in, , , It is a learnable projection matrix, used to calculate queries. s and keys dot product and through Normalization yields the attention weights. ,in It is an attention matrix, and the enhanced features are obtained by adding attention weights A and weighted values ​​V. Weight sum matrix Multiply, where The enhanced features are then converted from sequence form back into a two-dimensional feature map. ,in Residual connections are used to integrate input features and augmented features. ,in The output of the backbone network module contains multi-scale features and global self-attention enhancement information.

[0033] Furthermore, in step S4, the feature fusion module has the following structure: Figure 3 As shown, the output features of the backbone network module are used as input features, and the MST-A module outputs multi-scale features. ,in It is a low-level, fine-grained feature. It is a mid-level semantic feature. It is a high-level semantic feature, , , The input is fed into the DWFA module, which adjusts features at different scales to the same resolution and number of channels. , , , ,in, It uses a downsampling operation with a convolution kernel of k=3. It uses bilinear interpolation for upsampling; the aligned features are concatenated and dynamic weights are generated. , Global pooling and fully connected layers are used to generate fusion weights. ,in, , representing the dynamic weights at each scale, satisfying The features are summed using dynamic weights. ,in, This is the fused output feature. The fused feature is adjusted for the number of channels through convolution and residual connections are added. The feature fusion module ultimately outputs the following features: .

[0034] Furthermore, in step S5, the structure of the target detection module is as follows: Figure 4 As shown, the feature fusion module outputs features. The features input into the DAD-Head module contain both spatial information and the contextual semantic information required for classification and regression. The DAD-Head module uses a dynamic weighting mechanism to... Assigned to two task branches: classification and regression, utilizing global pooling and Convolution generates feature weights for classification and regression tasks. , It is a channel-by-channel weight. It is the Sigmoid activation function, used to normalize to [0,1], based on the generated weights, Weighting is applied separately to the classification and regression branches. , , It is a channel-by-channel dot product. These are features used for classification. These are features used for regression; the classification branch is responsible for predicting the class probability of each pixel. ,in K is the number of target categories, and each pixel... The probability for the k-th class is ,in This is the Logit value for the k-th class. For each pixel, the index of the predicted class is... Confidence level is The regression branch predicts the bounding box parameters corresponding to each pixel. ,in Each pixel The output is ,in , Here, represents the coordinates of the bounding box center point, and w and h represent the width and height of the bounding box. This maps the bounding box parameters from the feature map to the original image. , , , ,in , The original image's height and width are used for post-processing, filtering out images with confidence scores below a threshold. The prediction box, , for For boxes with high confidence, they are sorted from highest to lowest confidence, and the intersection-union ratio (IUU) between candidate boxes is calculated. , For intersection and union ratio Only retain the boxes with higher confidence levels. , for The final output is Where N is the number of targets after post-processing, and each target object contains a category index. Confidence level Bounding box parameters .

[0035] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements are all within the scope of protection of the present invention.

Claims

1. A wheel part detection method based on multi-scale dynamic fusion, characterized in that, The method comprises the following steps: S1, wheel part image dataset making, using an industrial camera to take pictures of the wheel parts, obtaining wheel part images, marking the positions that need to be detected, the positions that need to be detected being spoke air holes, marking other objects as irrelevant targets, after the marking is completed, obtaining a wheel part image dataset; S2, constructing an image preprocessing module, including picture scaling and normalization processing; S3, constructing a main network module, including a RepVGG module and an MSA-T module, the MSA-T module including a multi-scale convolution feature extraction MSC, a Flatten operation, a Transformer self-attention calculation, and a multi-scale feature restoration; S4, constructing a feature fusion module, including a DWFA module, the DWFA module including feature alignment, dynamic weight generation, dynamic weighted feature fusion, and enhanced output features; S5, constructing a target detection module, including a DAD-Head module and a post-processing module, the DAD-Head module including dynamic feature task allocation, a classification branch, and a regression branch; In the S2 step, for the image preprocessing module, an image I e R H×W×C is input, and the image is normalized and resized to obtain a preprocessed input feature x in = Preprocess(I), Preprocess being a preprocessing function. In the S3 step, for the backbone network module, the input image feature x in ∈I H×W×C , where H, W are the height and width of the feature respectively, and C is the number of channels, multi-level local features are extracted by RepVGG, x RepVGG = RepVGG(x in ), where x RepVGG contains multi-scale features, the image feature x RepVGG is input into the MSA-T module, and the MSA-T module includes a multi-scale convolution feature extraction MSC, a Flatten operation, a Transformer self-attention calculation, and a multi-scale feature restoration, the image feature x RepVGG is input into the MSC module, three different size convolution kernels are used to extract local features respectively, three branch features are generated, x1=Conv k＝1 (x RepVGG ), x2=Conv k＝3 (x RepVGG ), x3=Conv k＝5 (x RepVGG ), where k=1, k=3, and k=5 represent the sizes of different convolution kernels, the features extracted by different convolution kernels are fused into multi-scale features by channel-by-channel concatenation, x MSC = Concat(x1, x2, x3), where x MSC ∈R H×W×(3C) ; 1x1 convolution is used on x MSC to adjust the number of channels to d, and it is flattened into a sequence form, x flat = Flatten(Conv k＝1 (x MSC )), where x flat ∈R (H·W)×d ; the input x flat is projected into query Q, key K, and value V, Q=x flat W Q , K=x flat W k , V=x flat W V , where W Q , W K , and W V ∈R d×d are learnable projection matrices, the dot product of the query Q and the key K is calculated and normalized by Softmax to obtain the attention weight, where A∈R (H·W)×(H·W) is the attention matrix, the enhanced feature x att =AV is obtained by weighting the value V using the attention weight A, and x att ∈R (H·W)×d ; reshape the enhanced features from sequence form to two-dimensional feature map, x reshape = Reshape(x att ), where x reshape ∈R H×W×d ; integrate the input features and the enhanced features using a residual connection, x MST-A = x reshape + Conv k＝1 (x MSC ), where x MST-A ∈R H×W×d is the output of the backbone network module, containing multi-scale features and global self-attention enhanced information; In step S4, for the feature fusion module, the output features of the backbone network module are used as input features, and the MST-A module outputs multi-scale features, x MST-A = {x1, x2, x3}, where It is a low-level, fine-grained feature. It is a mid-level semantic feature. These are high-level semantic features. x1, x2, and x3 are input into the DWFA module, adjusting features of different scales to the same resolution and number of channels H×W×C. x′1 = Downsample(x1), x′2 = x2, x′3 = Upsample(x3), where Downsample(·) is a downsampling operation using a convolutional kernel k=3, and Upsample(·) is an upsampling operation using bilinear interpolation. The aligned features are then concatenated to generate dynamic weights α. i x cat =Concat(x′1, x′2, x′3), generates the fused weights α using global pooling and a fully connected layer. i =Softmax(FC(GlobalPool(x) cat ))), where α i ={α1, α2, α3}∈R 3 Represents the dynamic weights at each scale, satisfying Use dynamic weights to perform a weighted summation of features. Where, x DWFA ∈R H×W×C This is the fused output feature; the fused feature is then processed through convolution to adjust the number of channels, and residual connections are added, x out =x DWFA +Conv k＝1 (x cat The feature fusion module ultimately outputs feature x. out ∈R H×W×C ; In step S5, for the target detection module, the feature fusion module outputs the feature x out ∈R H×W×C is input into the DAD-Head module, which contains both spatial information and context semantic information required for classification and regression; the DAD-Head module assigns x out to two task branches of classification and regression through a dynamic weight mechanism, and generates feature weights w cls ,w reg for classification and regression tasks using global pooling and 1x1 convolution. k＝1 (GlobalPool(x out ))) is a sigmoid activation function for normalization to [0,1], and x cls , w reg ∈R C is weighted to the classification branch and the regression branch according to the generated weights, x out = x cls ·w out , x cls = x reg ·w out , · is a point-by-point multiplication, x reg ∈R cls H ×W×C is the feature for classification, and x reg ∈R H×W×C is the feature for regression; the classification branch is responsible for predicting the class probability of each pixel point, p class = Softmax(Conv k＝1 (x cls )), where p class ∈R H×W×K , K is the number of target classes, and the probability of the kth class for each pixel point (i, j) is where z k (i, j) is the Logit value of the kth class, and the index of the class prediction for each pixel point is The confidence is The regression branch predicts the target bounding box parameters corresponding to each pixel point, b box = Conv k＝1 (x reg ), where b box ∈R H×W×4 , and the output of each pixel point (i, j) is b box，i，j = {x center , y center , w, h}, where x center , y center ​where w, h are the width and height of the bounding box, the parameters of the bounding box on the feature map are mapped to the original picture, where W0, H0 are the height and width of the original image; in the post-processing stage, the prediction boxes with confidence lower than threshold T conf are filtered out, and the remaining boxes = {b box，i |s conf，i > T conf}, T conf is 0.5-0.6, for the boxes with high confidence, they are sorted according to the confidence from high to low, and the intersection over union IoU between the candidate boxes is calculated, for the boxes with intersection over union IoU(b box，i , b box，j ) > T IoU , only the box b box，i with higher confidence is kept, T IoU is 0.5-0.6, and the final output is Output = {(class i , s conf，i , b box，i )|i = 1, 2,..., N}, where N is the number of targets after post-processing, each target object contains class index class i , confidence s conf，i , and bounding box parameters b box，i .

Images