A fastener bolt detection method with feature extraction and feature interaction enhancement

Abstract

The present application relates to a kind of feature extraction and feature interaction enhanced fastener bolt detection method, comprising: a kind of feature extraction and feature interaction enhanced fastener bolt detection method, comprising: S1 gathers fastener bolt image;S2 labels and divides data set;S3 constructs detection model, contains backbone network and detection head: backbone network adopts structure reparameterization module, training state is multiple branch parallel, inference state fusion is single convolution, enhances feature extraction capability;Detection head is asymmetric decoupling structure, after convolution, the regression branch is divided into two ways, one transformation, one direct transmission, after splicing, execute channel mix to improve positioning accuracy, classification branch is connected in series by depth convolution and ordinary convolution, reduce the amount of calculation;S4 trains model and obtains weight;S5 tests and evaluates result.The present application is characterized in that: solve the problem that existing track fastener bolt detection model in feature extraction capability and positioning performance needs to be improved etc..

Description

Technical Field

[0001] This invention relates to the field of image processing and detection technology, specifically a method for detecting fastener bolts that combines feature extraction and feature interaction enhancement. Background Technology

[0002] Track fasteners, as crucial connecting components in railway track structures, play a vital role in fixing rails, maintaining track gauge, and transmitting train loads. Under long-term train operating loads and the influence of the natural environment, fastener bolts often loosen, seriously threatening the safety and stability of railway operations. Therefore, regularly tightening and maintaining track fastener bolts is one of the core tasks of the railway maintenance department. Traditional bolt tightening operations mainly rely on manual hand-held tools, which is not only labor-intensive and inefficient but also prone to safety hazards. In recent years, with the development of railway maintenance automation, fastener bolt tightening robots equipped with vision inspection systems have gradually become a research hotspot. However, the complex track environment, including variations in lighting, corrosion interference, and the complexity of the ballast background, poses a significant challenge to the vision system's ability to achieve high-precision bolt detection and positioning. Existing detection methods mostly focus on detecting the condition of the fasteners, emphasizing inspection. However, in actual track fastener bolt tightening operations, it is not only necessary to accurately detect the bolts but also to achieve high-precision positioning. Existing detection models still have room for improvement in feature extraction capabilities. While pursuing lightweight design and high inference speed, they sacrifice the accuracy of bounding box regression, and their localization performance needs further enhancement. Therefore, a detection method specifically designed for the detection and localization of track fastener bolts is needed to address these issues. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a fastener bolt detection method with enhanced feature extraction and feature interaction, so as to solve the problems of the existing track fastener bolt detection model having insufficient feature extraction capability and positioning performance.

[0004] To achieve the above objectives, a fastener bolt detection method based on feature extraction and enhanced feature interaction is designed, comprising the following steps: S1, collecting image feature data of rail fastener bolts; S2, labeling and dividing the dataset of image feature data of rail fastener bolts; S3, constructing a fastener bolt detection model, which includes input, backbone network, neck network, detection head, and output; the backbone network adopts a feature extraction module based on structural reparameterization technology, employing a multi-branch parallel structure in the training state of the backbone network, and equivalently fusing the multi-branch structure into a single convolutional structure in the inference state of the backbone network to improve the extraction capability of bolt edge and texture features; the detection head adopts an asymmetric decoupled detection head, designing the classification branch and regression branch of the detection head as an asymmetric structure; the regression branch consists of a convolutional layer, a channel shuffling module, and a convolutional layer connected in sequence. The system consists of two sub-branches. First, the preprocessed input features are convolved in the first convolutional layer and then equally divided into a first sub-tensor and a second sub-tensor along the channel dimension. The first sub-tensor undergoes a non-linear transformation and serves as the transformation branch, while the second sub-tensor is directly propagated as the direct propagation branch. The transformation and direct propagation branches are concatenated and then subjected to channel rearrangement to enhance the bounding box localization accuracy of the regression branch. Second, the classification branch is formed by concatenating deep convolutional layers and ordinary convolutional layers. The input features are first extracted spatially by the deep convolutional layer and then fused along the channel dimension by the ordinary convolutional layer, reducing the computational complexity and number of parameters in the fastener bolt detection model. Third, the fastener bolt detection model is trained using a dataset, and the weights of the trained model are obtained. Fourth, the trained fastener bolt detection model is tested using the dataset, and the test results are evaluated.

[0005] Preferably, the present invention further includes: in step S1, the data is enriched and the amount of data is expanded by using an offline data augmentation method, including different lighting, night, rainy and snowy operating scenarios. The specific method is as follows: S1.1: Obtain a dataset of fastener images in two states, tight and loose, of the track fastener system; S1.2: Enrich the dataset of operating scenarios by using an offline image data augmentation algorithm, including different lighting, night, rainy and snowy environments.

[0006] Preferably, the present invention further includes the following specific method for step S2: S2.1: Manually annotate the two types of detection targets in the track fastener bolt dataset using an annotation tool. The track fastener status includes: tight and loose; S2.2: Divide the annotated track fastener bolt dataset into a training set, a validation set, and a test set according to a certain ratio.

[0007] Preferably, the present invention further includes: the feature extraction module FRepBlock based on structural reparameterization technology in step S3 has the following structure: S3.1.1: The feature extraction module FRepBlock based on structural reparameterization technology is used to replace the cross-stage high-efficiency layer aggregation network CSP-ELAN architecture in the backbone network of the basic model, realizing topological decoupling between the model in the "training state" and the "inference state", so that the model can select a more suitable network structure at different stages; S3.1.2: In the model training stage, that is, the training state, the feature extraction module FRepBlock based on structural reparameterization technology adopts a multi-branch parallel structure, including a 3×3 convolution branch, a 1×1 convolution branch and an identity mapping branch. The input feature is represented as X, and the output F in the training state is... train It can be represented as: F train =σ(BN(W) 3×3 *X)+BN(W 1×1 *X)+BN id (X)); where σ represents the SiLU activation function, BN is the batch normalization operation, * represents the convolution operation, and W 3×3 W represents the weight parameters of the 3×3 convolution kernel, used to extract the local spatial features of the bolt. 1×1 The weights of the 1×1 convolution kernel are used for channel-dimensional feature transformation and information fusion. (BN) id This represents the batch normalization of the identity mapping branch, which directly passes the input features without performing convolution transformation; S3.1.3: During the model deployment and inference phase, leveraging the linear additivity of convolution operations, the module mathematically fuses the parallel multi-branch structure into a single 3×3 convolution. First, the 1×1 convolution kernel is expanded to a 3×3 scale through central symmetric zero padding. Then, the parameters of the batch normalization operation layer are absorbed into the weights W and biases B of the convolution kernel. The fused equivalent weights W fused and bias B fused The calculation is as follows: W fused =W 3×3 '+pad 3×3 (W) 1×1 ')+W id ';B fused =B 3×3 '+B 1×1 '+B id '; among which, W fused W represents the equivalent weights after fusion. 3×3 ' represents the weights of the original 3×3 convolutional branches, the result after equivalent transformation by a BN layer, W 1×1 ' represents the weights of a 1×1 convolution branch, the result after equivalent transformation by a BN layer, pad 3×3 (W) 1×1') indicates that a 1×1 convolution kernel is expanded to a 3×3 size through zero padding, W id ' represents the equivalent weight of the identity mapping branch, B fused B represents the equivalent bias term after fusion. 3×3 ' represents the equivalent bias of a 3×3 convolution branch, B 1×1 ' represents the equivalent bias of a 1×1 convolution branch, B id ' represents the equivalent bias of the identity mapping branch; S3.1.4: After structural reparameterization, the forward propagation only needs to be executed once, so that YOLO11s-RA can obtain the feature extraction capability of multi-branch networks while presenting a single-path topology during inference, and the output F of the inference stage. deploy Represented as: F deploy =σ(W fused *X+B fused ); where F deploy This represents the output feature map of the inference stage, where σ represents the nonlinear activation function, and W... fused The equivalent convolution weights after fusion, X is the input feature, and B is the equivalent convolution weight. fused This is the equivalent bias term after fusion.

[0008] Preferably, the present invention further includes: the constructed backbone network starts from the input image, first performs downsampling and channel expansion through two layers of convolution with a stride of 2, and then sequentially enters multiple sets of feature extraction modules FRepBlock based on structural reparameterization technology for feature extraction and channel deepening. Each convolution downsampling generates a feature map of the corresponding scale. The SPPF module is connected in the deep layer of the network to realize multi-scale receptive field fusion. Finally, the newly added C2PSA attention module enhances and filters high-level semantic features. Overall, the features are downsampled and refined step by step along the direction from shallow texture details to deep semantic information, and finally outputs three effective feature maps of different scales, P3, P4 and P5, which are fed into the neck network for subsequent fusion detection.

[0009] Preferably, the present invention further includes: the asymmetric decoupled detection head FADDet structure in step S3 is as follows: S3.2.1: The classification branch and regression branch in the detection head are designed asymmetrically. The regression branch adopts a feature enhancement path and channel reorganization mechanism to extract spatial detail features; the classification branch adopts depthwise separable convolution to compress redundant calculations to achieve lightweight mapping of semantic features; S3.2.2: The regression branch constructs a cascaded structure consisting of a 3×3 convolutional layer, a channel shuffling module, a 3×3 convolutional layer, and a 1×1 convolutional layer. After the input features are convolved in the first layer, they are divided into two sub-tensors along the channel dimension by a Chunk operation; one branch maintains the Identity direct transmission, aiming to preserve the original spatial details of the fastener edge; the other branch performs a nonlinear transformation through a bottleneck structure composed of 1×1 convolution, 3×3 depthwise separable convolution, and 1×1 convolution to extract more abstract bounding box position features; the two branches are connected to the Channel after Concat fusion. The Shuffle module rearranges the channel indices to force information exchange between the direct transmission branch and the transformation branch, which solves the feature isolation problem caused by grouping operations in deep networks, allowing the model to utilize both low-level spatial information and high-level semantic information. S3.2.3: The classification branch adopts 1×1 convolution, 3×3 depthwise separable convolution and 1×1 convolution structure: the first layer adopts 3×3 depthwise separable convolution to reduce the amount of computation and eliminate the channel shuffling and branching interaction logic in the regression branch.

[0010] Preferably, the present invention further includes the following specific method for step S5: S5.1: Inputting the image of the track fastener bolt to be located and detected into the trained track fastener bolt detection model; S5.2: After candidate box generation, classification, and position regression, the image outputs the final detection result, including the category, score, and selected target; S5.3: By collecting images of other fastener types, a generalized test set containing images of different fastener types is constructed; by performing depth image transformations such as strong exposure enhancement and low-light simulation on the original test set, a robust test set containing high-difficulty samples is constructed. After constructing the two test datasets, they are respectively processed... Perform generalization and robustness tests; S5.4: The following evaluation metrics are used to evaluate the results: Precision: The proportion of samples predicted as positive by the model that are actually positive, reflecting the model's precision ability. The formula is: P = (TP) / (TP + FP); where TP represents true positives, i.e., positive samples correctly predicted by the model; FP represents false positives, i.e., negative samples incorrectly predicted as positive by the model; Recall: The proportion of all true positive samples that are correctly detected by the model, reflecting the model's recall ability. The formula is: R = (TP) / (TP + FN); where FN represents false negatives, i.e., true positive samples that the model failed to detect.

[0011] Preferably, the present invention further includes: the evaluation index further includes: the mean average accuracy mAP50 is the average accuracy when the IoU threshold is 0.5, which is the core index for measuring the overall performance of the detection model; the strict mean average accuracy mAP50-95 is the average accuracy with a step size of 0.05 within the range of IoU threshold from 0.5 to 0.95. This index has higher requirements for positioning accuracy and is more in line with the actual needs of high-precision positioning of fastener bolts.

[0012] Preferably, the present invention further includes: the model training process uses the AdamW optimizer, with a total of 100 training rounds, a batch size of 32, an image size of 640×640, and online data augmentation including random flipping, color jittering, and random cropping.

[0013] Compared with the prior art, the advantages of this invention are: This invention enhances the detection scenarios of the model by performing offline data augmentation on the image data of track fastener bolts, and further expands the dataset, thereby improving the generalization and robustness of the model training. The model constructs an FRepBlock module based on structural reparameterization technology in the backbone network, replacing the original feature processing module, which significantly enhances the model's ability to extract bolt features. An asymmetric decoupled detection head FADDet is designed, which integrates the shuffle channel mixing mechanism into the regression branch, effectively promoting the information flow between feature channels, improving the accuracy of bounding box regression, and realizing high-precision detection and positioning of fastener bolts under complex working conditions. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method in an embodiment of the present invention; Figure 2 This is a network architecture diagram of the YOLO11s-RA model in an embodiment of the present invention; Figure 3 This is a structural diagram of the FRepBlock module in an embodiment of the present invention; Figure 4 This is a structural diagram of the FADDet detection head in an embodiment of the present invention; Figure 5 This is a comparison chart of test results for the improved model in the embodiments of the present invention; Figure 6 This is a distribution diagram of the positioning accuracy of the improved model test results in this embodiment of the invention; Figure 7 This is a comparison chart of the generalization test results of the improved model in the embodiments of the present invention; Figure 8 This is a distribution diagram of the localization accuracy of the generalization test results of the improved model in this embodiment of the invention; Figure 9This is a comparison chart of the robustness test results of the improved model in the embodiments of the present invention; Figure 10 This is a distribution diagram of the positioning accuracy of the robustness test results of the improved model in the embodiments of the present invention. Detailed Implementation

[0015] To make the purpose, principle and structure of the present invention clearer, the following description is provided in conjunction with the accompanying drawings and specific embodiments.

[0016] See Figures 1 to 10 This embodiment provides a fastener bolt detection method with enhanced feature extraction and feature interaction. This method is designed for the high-precision detection and positioning requirements in track fastener bolt tightening operations. It is based on the YOLO11s basic model and has been improved. Specifically, it includes dataset construction, model improvement design, model training, and performance testing and evaluation.

[0017] YOLO11 is a new generation computer vision model released by the Ultralytics team in 2024. While maintaining extremely high processing speed, it introduces new network modules such as C3k2 and C2PSA, which significantly improves feature extraction capabilities and detection accuracy.

[0018] First, the dataset was prepared. In this embodiment, the track fastener bolt image data was obtained manually, specifically by walking along the track test line with a handheld camera. The collected images included bolts in both tight and loose states, resulting in raw images from a single environment. To improve the generalization ability and robustness of the detection model, enabling it to adapt to different operating scenarios, this embodiment employed offline data augmentation methods to enrich the original data and expand its volume. The added operating scenarios included different lighting conditions, nighttime environments, rainy scenes, and snowy scenes. Through these offline augmentation processes, the original dataset was expanded into an image collection covering various complex operating conditions, effectively simulating various visual disturbances and lighting changes that might occur on the track. After data augmentation, the Labelimg annotation tool was used to manually annotate the two types of detection targets in the track fastener bolt dataset: bolts in a tight state and bolts in a loose state. Labelimg is an open-source graphical image annotation tool primarily used to manually draw bounding boxes for the target detection model and generate annotation files in VOC or YOLO format. During annotation, YOLO format annotation files are generated, containing the category label and bounding box coordinates for each target. After annotating all images, all data is divided into training, validation, and test sets according to a certain ratio. The training set is used for learning model parameters, the validation set is used for hyperparameter tuning and model selection during training, and the test set is used for independent evaluation of the final model performance. In this embodiment, the partitioning ratio adopts a conventional object detection dataset partitioning method to ensure consistent data distribution across sets.

[0019] Next, we will construct the model. This embodiment is based on the YOLO11s detection model and improved to construct a YOLO11s-RA model suitable for fastener bolt detection and positioning. The overall network architecture of this model includes an input layer, a backbone network, a neck network, a head detection layer, and an output layer. The input layer receives an image with a size of 640×640 pixels; the backbone network is responsible for extracting multi-scale features from the input image; the neck network fuses and enhances the features output by the backbone network; and the head detection layer completes target classification and bounding box regression based on the feature map.

[0020] The two core improvements in this embodiment are located in the backbone network and the detection head, respectively.

[0021] The first improvement involves building the FRepBlock module, based on structural reparameterization technology, into the backbone network, replacing the C3k2 module in the original YOLO11s. This significantly enhances the model's ability to extract bolt features without increasing inference latency. The C3k2 module is the core feature extraction block in YOLOv11, designed based on the CSP (Cross Stage Partial) architecture. It achieves an efficient balance between computational complexity and feature representation capability by improving parameter efficiency and optimizing gradient flow through channel segmentation and multi-branch fusion strategies. The FRepBlock module, also known as the feature extraction module based on structural reparameterization technology, is a basic network structure built on the reparameterization concept, specifically designed to optimize object detection models.

[0022] The second improvement involves the design of the asymmetric decoupled detection head FADDet. This head designs the classification and regression branches as asymmetric structures and incorporates a channel shuffling mechanism in the regression branch to promote information flow between feature channels and improve the accuracy of bounding box regression. FADDet is a novel object detection head that more efficiently addresses the inconsistency between the learning objectives of classification and localization tasks by assigning asymmetric branch structures of different depths and dimensions to the tasks and combining them with decoupled feature representations.

[0023] The core of the FRepBlock module lies in its structural reparameterization technique, which achieves topological decoupling between the "training state" and the "inference state." The training state and the inference state are two different operating modes of a deep learning model. The training state refers to the stage where the model iteratively updates its parameters using labeled data. The inference state refers to the stage where the model has been trained and is used to predict new samples.

[0024] During the model training phase, i.e., the training state, the FRepBlock module adopts a multi-branch parallel structure similar to ResNet, specifically including a 3×3 convolution branch, a 1×1 convolution branch, and an identity mapping branch. This multi-branch structure can effectively avoid the gradient vanishing problem caused by network deepening, enabling the model to learn richer and more robust bolt edge and texture features.

[0025] Let the input feature be X, then the output F of the training state is... train It can be expressed as the following formula.

[0026] F train =σ(BN(W) 3×3 *X)+BN(W 1×1 *X)+BN id (X)).

[0027] In the formula, σ represents the SiLU activation function, BN (batch normalization) is the batch normalization operation, * represents the convolution operation, and W 3×3 and W 1×1 The weights of the 3×3 and 1×1 convolution kernels are respectively, and the BN is... id This represents the batch normalization of the identity mapping branch.

[0028] During the model deployment and inference phase, the linear additivity of convolution operations is utilized to mathematically fuse the parallel multi-branch structure into a single 3×3 convolution. The specific fusion process is as follows: First, the 1×1 convolution kernel is expanded to a 3×3 scale through centrally symmetric zero-padding. Then, the BN layer parameters from each branch are absorbed into the weights W and biases B of the convolution kernel. These BN layer parameters include the scaling factor γ, bias β, mean μ, and variance σ. 2 .

[0029] The equivalent weight W after fusion fused and bias B fused The calculation is as follows.

[0030] W fused =W 3×3 '+pad 3×3 (W) 1×1 ')+W id '.

[0031] B fused =B 3×3 '+B 1×1 '+B id '.

[0032] Among them W 3×3 '、W 1×1 '、W id 'These are the weights of each branch after the BN equivalent transformation, B' and B' are respectively. 3×3 '、B 1×1 '、B id ' represents the corresponding equivalent bias.

[0033] After reparameterization of the structure, forward propagation only requires one standard convolution operation.

[0034] F deploy =σ(W fused *X+B fused ).

[0035] This allows YOLO11s-RA to achieve excellent feature extraction capabilities from multi-branch networks while exhibiting a minimalist single-path topology similar to VGG networks during inference, significantly reducing memory access costs and thus better meeting the requirements of edge computing platforms for computing power and real-time performance in rail automation equipment.

[0036] The specific workflow of the backbone network reconstructed based on the FRepBlock module is as follows: Starting with a 640×640 input image, downsampling and channel expansion are first achieved through two convolutional layers with a stride of 2. Then, multiple FRepBlock modules are sequentially used for feature extraction and channel deepening. Each convolutional downsampling generates a feature map at the corresponding scale. The SPPF module is then integrated into the deeper layers of the network to achieve multi-scale receptive field fusion. The SPPF (Spatial Pyramid Pooling Fast) module is a fast spatial pyramid pooling structure that extracts features at different scales by concatenating multiple max-pooling layers and concatenating them along the channel dimension, thereby efficiently fusing multi-scale receptive field information and enhancing the model's adaptability to changes in target scale. Finally, a newly added C2PSA attention module enhances and filters high-level semantic features. The entire backbone network progressively downsamples and refines features along the direction from shallow texture details to deep semantic information, ultimately outputting three effective feature maps at different scales (P3, P4, and P5) which are then fed into the neck network for subsequent fusion detection.

[0037] To improve the detection head, this embodiment designs an asymmetric decoupled detection head, FADDet. Considering the difference in feature distribution—"easy to classify, difficult to locate"—in the fastener bolt detection task, this detection head abandons the symmetrical design of classification and regression branches in traditional detection heads. Instead, it balances detection accuracy with computational overhead by configuring differentiated structural complexity for the two branches. The regression branch focuses on extracting spatial detail features, employing deeper feature enhancement paths and channel reorganization mechanisms; the classification branch focuses on lightweight mapping of semantic features, using depthwise separable convolutions to compress redundant computations. Specifically, the regression branch constructs a cascaded structure of 3×3Conv→Shuffle Module→3×3Conv→1×1Conv. 3×3Conv refers to a convolution operation with a kernel size of 3×3, and 1×1Conv refers to a convolution operation with a kernel size of 1×1, used to extract local spatial features. The Shuffle Module is a shuffling module that rearranges the channels of the feature map (channel shuffling), promoting information exchange between different channels, and is commonly used in lightweight networks.

[0038] This branch introduces a feature interaction mechanism based on channel segmentation: after the input features are convolved in the first layer, they are equally divided into two sub-tensors along the channel dimension through a Chunk operation. The Chunk operation is a block operation, which refers to the processing method of uniformly dividing the tensor into multiple sub-blocks along a specified dimension. One branch directly passes Identity, aiming to preserve the original spatial details of the fastener edge; the other branch performs a non-linear transformation through a bottleneck structure composed of 1×1 convolutions, 3×3 depthwise separable convolutions, and 1×1 convolutions to extract more abstract bounding box position features. After the two branches are concatenated, they are connected to the Channel Shuffle module, which rearranges the channel dimensions of the feature map, enabling information to flow between different channel groups, thereby enhancing the cross-channel feature interaction capability. This module forces information exchange between the direct transmission branch and the transformation branch by rearranging the channel indexes. This operation solves the feature isolation problem caused by grouping operations in deep networks, allowing the model to utilize both low-level spatial information and high-level semantic information, thereby improving the model's localization accuracy for small targets such as bolts in complex operation scenarios. For the classification branch, given that the fastener bolt detection task involves a small sample size and narrow categories, with only two detection states—tight and loose—the classification task is significantly less difficult than the localization task. To avoid redundant computation in the classification branch, a relatively simple 3×3DWConv→3×3Conv→1×1Conv structure is adopted: the first layer uses a 3×3 depthwise separable convolution, effectively reducing computational load with almost no loss of classification accuracy. DWConv is a depthwise convolution, which performs convolution operations independently on each channel of the input feature map without information fusion between channels, thus significantly reducing computational load and parameter count. The classification branch omits the channel shuffling and branching interaction logic found in the regression branch, tilting the overall parameter count of the detection head towards the regression task. This asymmetry ensures that, with limited computing resources, the model can prioritize critical localization capabilities, thereby effectively reducing the bolt false detection rate.

[0039] After model building is complete, the model training phase begins. This embodiment establishes a high-performance deep learning training environment. On the hardware side, an NVIDIA GeForce RTX 4090D graphics card is used as the acceleration device. This card is equipped with 24GB of video memory, coupled with an 18-core CPU and 60GB of RAM. Its powerful parallel computing capabilities significantly shorten training time and ensure smooth operation of data preprocessing and multi-threaded loading. On the software side, the training environment is built based on the CUDA 12.6 and PyTorch deep learning frameworks to fully utilize the GPU's computing performance. Key parameters used during training are as follows: total number of epochs set to 100, batch size set to 32, number of workers set to 12, input image size uniformly adjusted to 640×640 pixels, and AdamW optimized. Furthermore, online data augmentation, including random flipping, color dithering, and random cropping, is enabled during training to further improve the model's training effect. The YOLO11s-RA model was trained using training and validation sets. The network parameters were continuously optimized through backpropagation. After 100 iterations, the final bolt detection model weight file was obtained.

[0040] After model training, the model performance is comprehensively evaluated using the test set, as well as additionally constructed generalization and robustness test sets. The specific testing process is as follows: Images of the track fastener bolts to be detected are input into the trained YOLO11s-RA model. After candidate box generation, classification, and location regression, the final detection results are output, including the category, confidence score, and bounding box coordinates for each target. To evaluate the model's generalization ability, this embodiment constructs a generalization test set containing 200 images of different fastener types by collecting images of other fastener types. The fastener structures in these images differ from those in the training set, and the model has never seen these images during training. To evaluate the model's robustness, this embodiment constructs a robustness test set containing 200 high-difficulty samples by performing depth image transformations such as strong exposure enhancement and low-light simulation on the original test set. The images in both test sets were not used in the model training process. The model is then tested on these two test sets and compared with the benchmark model YOLO11s.

[0041] This embodiment uses commonly used evaluation metrics in the field of object detection to quantitatively evaluate the test results, including precision (P), recall (R), mean average precision (mAP50), and strict mean average precision (mAP50-95). Precision represents the proportion of samples that the model correctly predicts as positive, reflecting the model's "precision" capability. Its calculation formula is P = TP / (TP + FP), where TP represents true positives (the number of samples correctly predicted as positive) and FP represents false positives (the number of negative samples incorrectly predicted as positive). Recall represents the proportion of all true positive samples correctly detected by the model, reflecting the model's "recall" capability. Its calculation formula is R = TP / (TP + FN), where FN represents false negatives (the number of true positive samples that the model failed to detect). Mean average precision (mAP50) is the average precision when the intersection-union (IU) threshold is 0.5, and it is a core indicator for measuring the overall performance of the detection model. The strict average accuracy mean mAP50-95 is the average accuracy within the range of the cross-union ratio threshold from 0.5 to 0.95 (step size 0.05). This indicator has higher requirements for positioning accuracy and is more in line with the actual needs of high-precision positioning of fastener bolts.

[0042] This embodiment compares the YOLO11s-RA model with the baseline YOLO11s model on the test set, generalization test set, and robustness test set. The comparison results on the test set show that the proposed YOLO11s-RA model outperforms the baseline model in all four performance metrics: precision is improved by approximately 2 percentage points, recall by approximately 1.1 percentage points, mAP50 by approximately 0.6 percentage points, and mAP50-95 by approximately 1.9 percentage points. Visualized detection results show that in complex scenes containing targets with different lighting, angles, and scales, the improved model's bounding box for bolts more closely matches the true target boundary, and the overall classification confidence score is higher than the baseline model. The visualization of the localization accuracy distribution clearly reveals the difference in localization performance between the two models through the kernel density distribution of the intersection-union ratio (IU / U): the average IU / U of the baseline model is 0.766, while the average IU / U of the proposed model is improved to 0.774, with a higher distribution density in the high IU / U interval. On the generalization test set, the YOLO11s-RA model significantly outperformed the baseline model in terms of precision, recall, mAP50, and mAP50-95, indicating that the model's adaptability to different fastener types has been effectively improved. On the robustness test set, the model of this invention also outperformed the baseline model in all indicators, demonstrating enhanced anti-interference capabilities under conditions such as strong exposure and low illumination. These experimental results fully demonstrate the effectiveness and advancement of the fastener bolt detection method with enhanced feature extraction and feature interaction proposed in this invention, and can meet the practical needs for high-precision bolt detection and positioning in track fastener bolt tightening operations.

[0043] See Figure 2In the diagram, Conv represents a convolutional layer, used to downsample or transform the spatial dimension of the input feature map. C3k2 represents a basic feature extraction module in YOLOv11, designed based on the CSP (Cross Stage Partial) architecture, balancing computational efficiency and feature representation capability through channel segmentation and multi-branch convolutional fusion. FRepBlock is the proposed feature extraction module (FastenerRepVGG Block) based on structural reparameterization, used to replace the original C3k2 module. C2PSA is an attention module, which can be understood as C2fwith Position-Sensitive Attention, used to enhance and filter high-level semantic features, helping the model focus on the key areas of the bolt. SPPF is a fast spatial pyramid pooling module, which extracts receptive field features of different scales by concatenating multiple max pooling layers and splicing them to achieve multi-scale feature fusion, enhancing the model's adaptability to changes in bolt size. Backbone is the backbone network, responsible for extracting multi-scale features from the input image. The Neck network, located between the Backbone and Head, enhances multi-scale features through upsampling, concatenation, and the C3k2 / FRepBlock module, improving the detection capability for small targets (bolts). The Head is the detection head responsible for object classification and bounding box regression on the feature maps. FADDet is the FastenerAsymmetric Decoupled Detection Head. Concatenation combines multiple feature maps along the channel dimension for multi-scale feature fusion. Upsampling (typically bilinear interpolation or transposed convolution) enlarges small feature maps for fusion with shallower feature maps.

[0044] See Figure 3In the diagram, Mode-Train represents the training state, and Mode-Deploy represents the inference state. Input represents the input features, denoted as X. Output represents the output features, the result after processing by the module. 3×3Conv is a convolutional layer with a kernel size of 3×3, used to extract local spatial features. 1×1Conv is a convolutional layer with a kernel size of 1×1, used for channel transformation and information fusion. Identity is the identity mapping branch, which directly passes the input to the BN layer without performing convolutional transformation, helping to alleviate gradient vanishing. BatchNorm is a batch normalization layer, which standardizes the feature map (subtracting the mean and dividing the variance) before scaling and translation, accelerating training convergence. Split is a channel splitting operation (in the training state, multiple branches can be considered as splitting, processing separately, and then adding). Add is an element-wise addition operation, adding the outputs of the three branches (3×3, 1×1, and identity) to achieve multi-branch feature fusion. SiLU is the activation function (Sigmoid Linear Unit), which performs a non-linear transformation on the fusion result. RepVGGBlock is a basic module based on reparameterization. The entire structure shown in this embodiment is a variant of RepVGGBlock, named FRepBlock.

[0045] See Figure 4In the figure, Feature Maps (P3, P4, P5) are three-scale feature maps from the neck network, with resolution decreasing from high to low (P3 is the highest, P5 is the lowest), used to detect bolt targets at different scales. Input is the input feature map (from a layer of the neck network, such as P3, P4, or P5). REG Branch is the Regression Branch, responsible for predicting bounding box coordinates. CLS Branch is the Classification Branch, responsible for predicting the target category (tight or loose). 3×3Conv is a regular 3×3 convolution, used in both the regression and classification branches for feature extraction. 1×1Conv is a regular 1×1 convolution, used to adjust the number of channels or perform linear transformations of features. 3×3DWConv is a 3×3 depthwise separable convolution, separating spatial and channel convolutions, significantly reducing computation. Chunk divides the feature map into multiple sub-tensors along the channel dimension. In the regression branch shown in the diagram, after the initial 3×3 convolution, the feature map is divided into two groups (Identity branch and Transform branch) through a Chunk operation. The Identity branch is an identity mapping branch that directly passes the segmented sub-tensors without any convolutional transformation, preserving the original spatial details. The Concat operation concatenates the outputs of the Identity and Transform branches along the channel dimension and then feeds them into the Channel Shuffle module. The Channel Shuffle operation rearranges the channels of the concatenated feature map, forcing information exchange between different groups and resolving the feature isolation problem caused by grouped convolutions. This is located inside the Shuffle Module in the diagram. The Shuffle Module is the channel shuffling module, containing Chunk, two branches (Identity and non-linear transformation), Concat, and Channel Shuffle operations.

[0046] In YOLO, the score on each bounding box is a composite confidence score, usually obtained by multiplying the "probability that the box contains an object" by the "probability of correct classification." It represents the model's overall confidence level that an object of a specific class exists within that bounding box. A higher score (e.g., 0.9) indicates that the model is more confident that it has accurately bounded an object and classified it correctly; a lower score (e.g., 0.3) may correspond to false positives or poor localization.

[0047] See Figure 5The images show examples of detection results output by the YOLO11s baseline model and the improved YOLO11s-RA model on the test set. Each sub-image contains multiple bounding boxes, each labeled with its category: "Tightened" indicates a bolt in a tight state, and "Loosened" indicates a bolt in a loose state. The numbers following these boxes (e.g., 0.85, 0.92, etc.) represent the confidence scores predicted by the model. Comparing the confidence scores of tight and loose bolts in the same image dataset, the results from YOLO11s (top) and YOLO11s-RA (bottom) show that YOLO11s-RA, as an improved model, has a significantly higher confidence score for state recognition than the baseline YOLO11s, indicating that the improved model has a stronger ability to distinguish between the two types of bolts.

[0048] See Figure 6 The figure illustrates the distribution of the Intersection over Union (IoU) between the predicted bounding boxes and the ground truth bounding boxes on the test set. The horizontal axis represents the IoU value, ranging from 0.0 to 1.0, and the vertical axis represents the probability density. The blue curve represents the baseline model YOLO11s, and the yellow curve represents the improved model YOLO11s-RA. The figure shows the mean of 0.766 for the baseline model and 0.774 for the improved model, indicating that YOLO11s-RA has better bolt positioning accuracy than the baseline model YOLO11s, and the predicted bounding box fits the target's true boundary more closely. The Intersection over Union (IoU) is the ratio of the intersection area to the union area of ​​the predicted bounding box and the ground truth bounding box, used to measure the accuracy of localization in target detection.

[0049] See Figure 7 The image shows the detection results on a generalization test set, which includes fastener types different from those in the training set. The top and bottom of the image are both detection visualizations; each sub-image contains multiple bounding boxes, each labeled with the category "Tightened" and its corresponding confidence score. This demonstrates that the improved YOLO11s-RA model has stronger generalization ability for different fastener types, and the detection results are more reliable.

[0050] See Figure 9 This study compares the detection results of the baseline model YOLO11s and the improved model YOLO11s-RA on a robustness test set (including interference such as strong exposure and low light). The improved model YOLO11s-RA also shows higher confidence scores for all bolts marked "Tightened" than the low-scoring samples in the baseline model YOLO11s, indicating that the improved model is more stable and reliable in identifying fastening states under complex interference conditions, and has stronger robustness in localization and classification.

[0051] Example 1: This example is compared with the benchmark model YOLO11s. The comparison results include test set test results, generalization and robustness test results, as shown in Table 1. Figure 5 , Figure 6 As shown in Table 2, the generalization test results are as follows. Figure 7 , Figure 8 As shown in Table 3, the robustness test results are as follows. Figure 9 , Figure 10 As shown.

[0052] Table 1. Test results.

[0053]

[0054] Table 2. Generalization test results.

[0055]

[0056] Table 3. Robustness test results.

[0057]

[0058] The test results show that the proposed feature extraction and feature interaction enhancement method for fastener bolt detection demonstrates significant advantages in target detection performance. All four performance indicators show substantial improvements over the benchmark model: precision increased by 2 percentage points, recall by 1.1 percentage points, mAP50 by 0.6 percentage points, and mAP50-95 by 1.9 percentage points. Visualization results show that in complex scenes containing targets with varying lighting, angles, and scales, the improved model's bolt location boxes more closely match the true target boundaries, and the overall classification confidence score is higher than the benchmark model. The visualization of the localization accuracy distribution, through the kernel density distribution of IoU, clearly reveals the difference in localization performance between the two models. The average IoU of the benchmark model is 0.766, while the average IoU of the proposed model is increased to 0.774, with a higher distribution density in the high IoU range. This indicates that the proposed improvement strategy effectively enhances the model's localization accuracy, making it superior in small target detection tasks.

[0059] The generalization and robustness test results show that the model of this invention outperforms the benchmark model in all four main performance indicators, and also shows better detection performance and higher positioning accuracy in the visualization results. This indicates that the model has improved adaptability to different fastener types and improved anti-interference ability.

[0060] The above description is merely a specific embodiment of the invention, but the scope of protection of the invention is not limited thereto. Any equivalent substitutions or changes made by those skilled in the art within the technical scope disclosed in the invention, based on the technical solution and concept of the invention, should be covered within the scope of protection of the invention.

Claims

1. A method for detecting fastener bolts with feature extraction and feature interaction enhancement, characterized in that, Includes the following steps: S1 acquires image feature data of the track fastener bolts; S2 annotates and divides the dataset of image feature data for track fastener bolts; S3 constructs a fastener bolt detection model, which includes input, backbone network, neck network, detection head, and output; The backbone network employs a feature extraction module based on structural reparameterization technology. In the training state of the backbone network, a multi-branch parallel structure is used, and in the inference state of the backbone network, the multi-branch structure is equivalently fused into a single convolutional structure to improve the ability to extract bolt edge and texture features. The detection head adopts an asymmetric decoupled detection head, and the classification branch and regression branch of the detection head are designed as asymmetric structures; The regression branch consists of a convolutional layer, a channel shuffling module, and another convolutional layer connected in sequence. After the preprocessed input features are convolved by the first convolutional layer, they are divided into a first sub-tensor and a second sub-tensor along the channel dimension. The first sub-tensor undergoes a nonlinear transformation and becomes the transformation branch, while the second sub-tensor is directly passed and becomes the direct pass branch. The transformation branch and the direct pass branch are concatenated and then the channel rearrangement operation is performed to enhance the bounding box localization accuracy of the regression branch. The classification branch is formed by concatenating deep convolutional layers and ordinary convolutional layers. The input features are first extracted in the spatial dimension by deep convolutional layers, and then fused in the channel dimension by ordinary convolutional layers, which reduces the computational complexity and number of parameters of the fastener bolt detection model. S4 uses the dataset to train the fastener bolt detection model and obtains the weights of the trained fastener bolt detection model. S5 uses a dataset to test the trained fastener bolt detection model and evaluates the test results.

2. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 1, characterized in that, In step S1, offline data augmentation methods are used to enrich the operational scenarios and expand the data volume, including operational scenarios under different lighting conditions, at night, rainy days, and snowy days. The specific methods are as follows: S1.1: Obtain a dataset of fastener images in both tight and loose states of the track fastener system; S1.2: The dataset is enriched with different work scenarios using offline image data augmentation algorithms. The work scenarios include different lighting conditions, nighttime, rainy days, and snowy days.

3. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 1, characterized in that, The specific method for step S2 is as follows: S2.1: Use the annotation tool to manually annotate the two types of detection targets in the track fastener bolt dataset. The track fastener status includes: tight and loose. S2.2: Divide the labeled track fastener bolt dataset into training set, validation set, and test set according to a certain ratio.

4. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 1, characterized in that, The structure of the feature extraction module FRepBlock based on structural reparameterization technology in step S3 is as follows: S3.1.1: The feature extraction module FRepBlock based on structural reparameterization technology is adopted to replace the CSP-ELAN architecture of the cross-stage high-efficiency layer aggregation network in the backbone network of the basic model, realizing the topological decoupling of the model in the "training state" and "inference state", enabling the model to select a more appropriate network structure at different stages; S3.1.2: During the model training phase, i.e., the training state, the feature extraction module FRepBlock based on structural reparameterization technology adopts a multi-branch parallel structure, including a 3×3 convolution branch, a 1×1 convolution branch, and an identity mapping branch. The input feature is represented as X, and the output F in the training state is... train It can be represented as: F train =σ(BN(W 3×3 *X)+BN(W) 1×1 *X)+BN id (X); In the formula, σ represents the SiLU activation function, BN is the batch normalization operation, * represents the convolution operation, and W 3×3 W represents the weight parameters of the 3×3 convolution kernel, used to extract the local spatial features of the bolt. 1×1 The weights of the 1×1 convolution kernel are used for channel-dimensional feature transformation and information fusion. (BN) id This represents the batch normalization of the identity mapping branch, which directly passes the input features without performing a convolutional transformation; S3.1.3: During the model deployment and inference phase, leveraging the linear additivity of convolution operations, the module mathematically fuses parallel multi-branch structures into a single 3×3 convolution. First, the 1×1 convolution kernel is expanded to a 3×3 scale through centrally symmetric zero-padding. Then, the parameters of the batch normalization operation layer are incorporated into the weights W and biases B of the convolution kernel. The fused equivalent weights W... fused and bias B fused The calculation is as follows: W fused =W 3×3 '+pad 3×3 (W 1×1 ')+W id '; B fused =B 3×3 '+B 1×1 '+B id '; Among them, W fused W represents the equivalent weights after fusion. 3×3 ' represents the weights of the original 3×3 convolutional branches, the result after equivalent transformation by a BN layer, W 1×1 ' represents the weights of a 1×1 convolution branch, the result after equivalent transformation by a BN layer, pad 3×3 (W) 1×1 ') indicates that a 1×1 convolution kernel is expanded to a 3×3 size through zero padding, W id ' represents the equivalent weight of the identity mapping branch, B fused B represents the equivalent bias term after fusion. 3×3 ' represents the equivalent bias of a 3×3 convolution branch, B 1×1 ' represents the equivalent bias of a 1×1 convolution branch, B id ' represents the equivalent bias of the identity mapping branch; S3.1.4: After structural reparameterization, the forward propagation only needs to be executed once, allowing YOLO11s-RA to retain the feature extraction capabilities of multi-branch networks while exhibiting a single-path topology during inference. The output F during the inference phase... deploy Represented as: F deploy =σ(W fused *X+B fused ); Where F deploy This represents the output feature map of the inference stage, where σ represents the nonlinear activation function, and W... fused The equivalent convolution weights after fusion, X is the input feature, and B is the equivalent convolution weight. fused This is the equivalent bias term after fusion.

5. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 4, characterized in that, The constructed backbone network starts with the input image and first performs downsampling and channel expansion through two convolutional layers with a stride of 2. Then, it sequentially enters multiple feature extraction modules FRepBlock based on structural reparameterization technology for feature extraction and channel deepening. Each convolution downsampling generates a feature map of the corresponding scale. The SPPF module is connected in the deep layers of the network to realize multi-scale receptive field fusion. Finally, the newly added C2PSA attention module enhances and filters high-level semantic features. Overall, the network downsamples and refines features step by step from shallow texture details to deep semantic information, and finally outputs three effective feature maps of different scales, P3, P4 and P5, which are fed into the neck network for subsequent fusion detection.

6. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 4, characterized in that, The asymmetric decoupling detection head FADDet structure in step S3 is as follows: S3.2.1: The classification and regression branches in the detection head are designed asymmetrically. The regression branch uses feature enhancement paths and channel reorganization mechanisms to extract spatial detail features; the classification branch uses depthwise separable convolution to compress redundant calculations and achieve lightweight mapping of semantic features. S3.2.2: The regression branch constructs a cascaded structure consisting of a 3×3 convolutional layer, a channel shuffling module, another 3×3 convolutional layer, and a 1×1 convolutional layer. After the input features are convolved in the first layer, they are divided into two sub-tensors along the channel dimension by a Chunk operation. One branch directly passes the Identity feature, aiming to preserve the original details of the fastener edge space. The other branch performs a non-linear transformation through a bottleneck structure composed of 1×1 convolutions, 3×3 depthwise separable convolutions, and 1×1 convolutions to extract more abstract bounding box position features. After the two branches are merged by Concat, they are connected to the Channel Shuffle module. This module rearranges the channel indexes to force information exchange between the direct transmission branch and the transformation branch, which is used to solve the feature isolation problem caused by grouping operations in deep networks, so that the model can utilize both low-level spatial information and high-level semantic information at the same time. S3.2.3: The classification branch adopts 1×1 convolution, 3×3 depthwise separable convolution and 1×1 convolution structure: the first layer adopts 3×3 depthwise separable convolution to reduce the amount of computation and eliminate the channel shuffling and branching interaction logic in the regression branch.

7. A method for detecting fastener bolts by feature extraction and feature interaction enhancement as described in any one of claims 1 to 6, characterized in that, The specific method for step S5 is as follows: S5.1: Input the images of the track fastener bolts that need to be located and detected into the trained track fastener bolt detection model; S5.2: The image undergoes candidate box generation, classification, and location regression, outputting the final detection results, including the category and score, as well as the selected target. S5.3: By collecting images of other fastener types, a generalization test set containing images of different fastener types was constructed; by performing depth image transformations such as strong exposure enhancement and low light simulation on the original test set, a robustness test set containing high-difficulty samples was constructed. After constructing the two test datasets, generalization and robustness tests were performed respectively. S5.4: The following evaluation indicators shall be used to evaluate the results: Precision: The proportion of samples predicted as positive by the model that are actually positive; it reflects the model's precision. The formula for calculation is: P = (TP) / (TP + FP); Where TP represents true positives, which are positive examples correctly predicted by the model; FP represents false positives, which are negative examples incorrectly predicted as positive by the model. Recall: The proportion of all true positive samples correctly detected by the model, reflecting the model's recall capability. Its calculation formula is: R = (TP) / (TP + FN); Here, FN represents a false negative, which is a true positive that the model failed to detect.

8. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 7, characterized in that, The evaluation indicators also include: The mean accuracy mAP50 is the average accuracy when the IoU threshold is 0.5, and it is the core indicator for measuring the overall performance of the detection model. The strict average accuracy mAP50-95 is the average accuracy with a step size of 0.05 within the range of IoU threshold from 0.5 to 0.

95. This indicator has higher requirements for positioning accuracy and is more in line with the actual needs of high-precision positioning of fastener bolts.

9. The fastener bolt detection method with feature extraction and feature interaction enhancement as described in claim 6, characterized in that, The model training process uses the AdamW optimizer, with a total of 100 training epochs, a batch size of 32, an image size of 640×640, and online data augmentation including random flipping, color jittering, and random cropping.

Images