A railway fastener anomaly detection method based on variant YOLO

Abstract

The invention discloses a railway fastener anomaly detection method based on variant YOLO, comprising the following steps: S1: input image, extract features through backbone network, and generate at least three feature maps with different clarity; S2: fuse and optimize multiple feature maps through feature fusion network; S3: regress and classify the fused and optimized feature maps through original head detection layer. The accuracy of detecting railway fastener assembly anomalies under challenging conditions is improved, and it is applicable to objects with high feature similarity and suboptimal image quality.

Description

Technical Field

[0001] The present invention relates to the technical field of railway fastener detection, and in particular to a railway fastener anomaly detection method based on variant YOLO. Background Art

[0002] Railway fasteners are an important part of the railway transportation safety system. These fasteners frequently encounter high failure rates due to their high aspect ratio. However, there are about 6,700 fasteners per kilometer of railway transportation in my country. Manual inspection is too inefficient and easily affected by subjective judgment. Therefore, it is crucial to develop a railway line inspection system that utilizes advanced technologies such as deep learning.

[0003] In the past decade, researchers at home and abroad have proposed methods to detect track anomalies using digital technology and various sensors, such as vibration signal-based detection, ultrasonic detection, acoustic analysis detection, etc. However, these methods require expensive equipment and detailed track geometry parameters to help locate and determine the status of fasteners, so due to the high cost, this method is not popular.

[0004] With the development of machine learning, many scholars have worked hard to explore its application in the field of rail fastener detection. Some studies have extracted improved edge gradient features and macro texture features of fasteners, and used image fusion features and Bayesian compressed sensing models to achieve automatic recognition of rail fastener anomalies. Another method combines improved Canny edge detection and median filtering, and uses curve feature projection templates to match the edge contours of fasteners to achieve rapid detection of missing fasteners. In addition, there is a method based on extreme value theory (EVT) and Bayesian framework, which proves that the detection performance can be improved on various detectors. The accuracy of this method in railway track anomaly detection has been improved. In order to cope with the problem of insufficient abnormal fastener samples, virtual negative samples simulating broken fasteners are generated based on normal fastener images, and image features are extracted using the HOG feature calculation method. Finally, abnormal fastener detection is achieved through the trained SVM classifier. Although the above machine learning detection methods have improved the detection accuracy to a certain extent, due to complex image processing and feature extraction, there are still problems such as poor robustness, susceptibility to environmental influences, many constraints, large computational complexity, and low recognition speed. Summary of the invention

[0005] The present invention aims to solve the technical problems existing in the prior art, and in particular innovatively proposes a railway fastener anomaly detection method based on a variant YOLO, which improves the accuracy of detecting anomalies of railway fastener components under challenging conditions and is suitable for objects with high feature similarity and suboptimal image quality.

[0006] In order to achieve the above object, the present invention provides a railway fastener anomaly detection method based on variant YOLO, comprising the following steps:

[0007] S1: Input image, extract features through the backbone network, and generate at least three feature maps of different resolutions;

[0008] S2: Multiple feature maps are fused and optimized through a feature fusion network;

[0009] S3: The fused and optimized feature maps are subjected to regression and classification processes through the original head detection layer.

[0010] In the above scheme: step S1 also includes:

[0011] S1-1: The image data processed by the convolution layer, ELANBlock layer and MP-1 layer is processed by EFOV to form a feature map of the first resolution size, and then processed by the MP-1 layer again;

[0012] S1-2: The feature map processed by S1-1 is processed again by EFOV to form a feature map of the second resolution, and is processed again by MP-1 layer;

[0013] S1-3: The feature map processed by S1-2 is processed again by EFOV to form a feature map of the third clarity level.

[0014] In the above scheme: step S1-1 also includes:

[0015] S1-1-1: When EFOV is processing, the image data is first subjected to a 1x1 convolution operation, and S1-1-2, S1-1-3, S1-1-4, and S1-1-5 are executed simultaneously;

[0016] S1-1-2: Execute S1-1-6;

[0017] S1-1-3: Perform a 3x3 convolution operation on the image data and execute S1-1-6;

[0018] S1-1-4: Perform two 3x3 convolution operations on the image data and execute S1-1-6;

[0019] S1-1-5: Perform four 3x3 convolution operations on the image data and execute S1-1-7;

[0020] S1-1-6: The image data processed by S1-1-2, S1-1-3 and S1-1-4 are fused and connected through the residual branch. After fusion and connection, a 1x1 convolution operation is performed and S1-1-7 is executed;

[0021] S1-1-7: Connect the image data processed by S1-1-5 and S1-1-6, perform a 1x1 convolution operation, and modify the number of channels.

[0022] In the above scheme: step S2 also includes:

[0023] S2-1: Process the feature map with the highest clarity through SPPCSPC. After processing, execute S2-2 and S2-6 at the same time;

[0024] S2-2: The feature map with the highest clarity after SPPCSPC processing is upsampled through the upsample module;

[0025] S2-3: Connect the feature map after upsampling with the feature map of the second definition. After the connection is completed, process the feature map through the ELANBlock layer. After the ELANBlock layer is processed, execute S2-4 and S2-5 at the same time.

[0026] S2-4: Upsample the feature map again through the upsample module, and connect the upsampled feature map with the feature map with the lowest definition. After the connection is completed, the feature map is processed through the ELANBlock layer; after the ELANBlock layer is processed, S2-5 and S2-7 are executed at the same time;

[0027] S2-5: Process the feature map through the OD-MP module to improve the fineness of the feature map, and then process the feature map through the ELANBlock layer to execute S2-6;

[0028] S2-6: Process the feature map through the OD-MP module to improve the fineness of the feature map, and then process the feature map through the ELANBlock layer to execute S2-7;

[0029] S2-7: Processed by the EMA module to capture the local important information of the feature map and sent to the original head detection layer.

[0030] In the above scheme: Steps S2-5 and S2-6 also include:

[0031] When the OD-MP module is processing, the input feature map is sent to two processing branches at the same time. One of the processing branches first adjusts the channel through a 1×1 convolution operation, and then performs a 3×3 convolution operation with a stride of 2 through the ODConv module for downsampling;

[0032] The other processing branch first downsamples through the maximum pooling operation, and then the ODConv module performs a 1×1 convolution operation to modify the number of channels;

[0033] Finally, the data of the two processing branches are added to achieve super downsampling.

[0034] In the above scheme: step S2-7 also includes:

[0035] S2-7-1: Let the input feature map be X = [X0, X1, ..., XG-1], where Xi∈R C×H×W , the input feature map is divided into G sub-features based on the number of channels through the EMA module, and G<<C, C represents the number of channels of the feature map; S2-7-2, S2-7-3, and S2-7-9 are executed at the same time;

[0036] S2-7-2: Perform a horizontal global pooling operation on the input data, encode the channel in the X-axis direction, and execute S2-7-4;

[0037] S2-7-3: Perform vertical global pooling operation on the input data and encode channels in the Y-axis direction;

[0038] S2-7-4: The two sets of data are first connected and subjected to a 1×1 convolution operation, and then activated by the softmax activation function, and then the sigmoid function is used to represent the 2D binary distribution of the linear convolution;

[0039] S2-7-5: Process through GroupNorm and execute S2-7-6 and S2-7-7 at the same time;

[0040] S2-7-6: first activate through softmax, and send the data to the Matmul module, and execute S2-7-8;

[0041] S2-7-7: perform global equal pooling operation and send the data to the Matmul module to execute S2-7-12;

[0042] S2-7-8: Execute S2-7-13 by processing the two sets of data Matmul;

[0043] S2-7-9: Perform a 3×3 convolution operation on the input data, and execute S2-7-10 and S2-7-11 at the same time;

[0044] S2-7-10: Activate through softmax and send the data to the Matmul module to execute S2-7-12;

[0045] S2-7-11: perform global pooling operation and send the data to the Matmul module to execute S2-7-8;

[0046] S2-7-12: Execute S2-7-13 by processing the two sets of data Matmul;

[0047] S2-7-13: Fuse the two sets of data, and after fusion, perform nonlinear mapping on the input attention score through the Sigmoid function and output it.

[0048] In the above scheme: when performing the global average pooling operation, the global average pooling operation is performed by the following formula:

[0049]

[0050] Among them, Z c is the pooling result on channel c, H and W are the height and width of the input feature map respectively. xc(i,j) represents the value of channel c at position (i,j) in the input feature map.

[0051] In summary, the beneficial effects of the present invention are as follows: 1. A platform for collecting rail fastener anomaly datasets is established, which contains standardized railway fastener dataset categories. In addition, it also covers some fastener anomaly categories that are missing in the prior art, involving various railway fastener categories under different conditions.

[0052] 2. A multi-branch parallel enhanced field of view module, namely the EFOV module, is set up. This module runs in a parallel branch within the EFOV. By extracting features of different scales in the module and using dilated convolutions, the receptive field can be expanded without increasing the network depth and computational load.

[0053] 3. In actual scenarios, rail fastener anomaly detection may be affected by changes in lighting conditions, and the image may appear blurred due to the movement of the device. To solve this problem, this technical solution sets up OD_MP, which combines orthogonal dynamic convolution (ODConv) and builds a parallel four-dimensional attention mechanism to refine the local fine-grained features in the image and help extract salient features.

[0054] 4. An efficient multi-scale attention module based on cross-space learning, namely the EMA module, is introduced. It adopts a cross-space learning method and designs a multi-scale parallel sub-network to establish short-term and long-term dependencies to enhance multi-dimensional perception and multi-scale feature extraction capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a structural schematic diagram of the present invention.

[0056] Figure 2 Schematic diagram of the image acquisition system used in the present invention.

[0057] Figure 3This is the architecture diagram of the EFOV module.

[0058] Figure 4 This is the architecture diagram of OD_MP.

[0059] Figure 5 This is the architecture diagram of the EMA module.

[0060] Figure 6 It is an image acquired by the image acquisition system.

[0061] Figure 7 This is the enhanced comparison picture.

[0062] Figure 8 This is a data comparison chart of different models on the GKA dataset when the IoU threshold is 0.5.

[0063] Fig. 9 This is a comparison chart of the accuracy of different models on the GKA dataset.

[0064] Fig.10 This is a comparison chart of the positioning loss values ​​of different models on the GKA dataset.

[0065] Fig.11 This is a data comparison chart of different models on the GKA dataset when the IoU threshold is 0.5 to 0.9.

[0066] Fig.12 This is a comparison chart of the recall rates of different models.

[0067] Fig.13 This is a comparison chart of the classification loss values ​​of different models on the GKA dataset.

[0068] Fig.14 Detection diagram of the baseline and the proposed model on the GKA dataset. DETAILED DESCRIPTION

[0069] The present invention will be further described below by way of embodiments and in conjunction with the accompanying drawings:

[0070] like Figure 1 to Figure 14 As shown, a railway fastener anomaly detection method based on variant YOLO includes the following steps:

[0071] S1: Input image, extract features through the backbone network, and generate at least three feature maps of different resolutions;

[0072] S1-1: The image data processed by the convolution layer, ELANBlock layer and MP-1 layer is processed by EFOV to form a feature map of the first resolution size, and then processed by the MP-1 layer again;

[0073] S1-1-1: When EFOV is processing, the image data is first subjected to a 1x1 convolution operation, and S1-1-2, S1-1-3, S1-1-4, and S1-1-5 are executed simultaneously;

[0074] S1-1-2: Execute S1-1-6;

[0075] S1-1-3: Perform a 3x3 convolution operation on the image data and execute S1-1-6;

[0076] S1-1-4: Perform two 3x3 convolution operations on the image data and execute S1-1-6;

[0077] S1-1-5: Perform four 3x3 convolution operations on the image data and execute S1-1-7;

[0078] S1-1-6: The image data processed by S1-1-2, S1-1-3 and S1-1-4 are fused and connected through the residual branch. After fusion and connection, a 1x1 convolution operation is performed and S1-1-7 is executed;

[0079] S1-1-7: Connect the image data processed by S1-1-5 and S1-1-6, perform 1x1 convolution operation, and modify the number of channels;

[0080] First, the number of channels is reduced by 1x1 convolution. The three middle branches pass through three convolution layers, using convolution kernel sizes of 1, 3, and 5 respectively. The 5x5 convolution kernel in the branch is changed to a rate of 5 to stack two 3x3 convolutions to simulate the perception field at different scales and reduce the number of parameters. The tail uses dilated convolution to replicate the correlation between the receptive field size and eccentricity observed in the human visual system. Different dilation factors are used to simulate this relationship. By simulating the sensitivity of the human eye to the central area, this embodiment adjusts the effective receptive field size of the convolution kernel to improve the insensitivity to small spatial changes.

[0081] exist Figure 3 In Figure 2, the upper part of the EFOV structure shows the expansion of the feature map receptive field without losing image information, where zero values ​​are inserted in the middle of the convolution kernel. Figure 3 (a) is a common 3×3 convolution. Figure 3 As shown in (b) and (c), the 5×5 zero convolution is essentially a 3×3 convolution with 2 zero values ​​added in the middle. Similarly, the 7×7 zero convolution is essentially a 3×3 convolution with 4 zero values ​​added in the middle. This expands the perception range, but in the actual training process, the parameters are the same as the 3×3 convolution.

[0082] A residual branch is added on the far left to connect the input features directly with the receptive fields obtained at three eccentricities to ensure full fusion and prevent feature loss or distortion, thereby enhancing network performance and convergence speed; the rightmost branch is an efficient aggregation network structure that fuses features from different levels and effectively utilizes semantic information at different levels to improve the feature representation ability of the model. Finally, the branches are connected and the number of channels is modified through 1x1 convolution.

[0083] In the railway fastener anomaly category, areas with lower severity occupy fewer pixels, resulting in fewer features that can be extracted and lower detection accuracy. However, the information carried by feature points close to the center of the target is crucial to the detection results. For the feature map, each neuron captures a different receptive field area, and there is overlap between the receptive field areas. In addition, elements close to the center are scanned more frequently, resulting in more overlap in the receptive fields. As the depth of the convolutional neural network increases, the receptive field on the neuron becomes larger, which means that the neuron carries more information.

[0084] The EFOV module of this technical solution can extract features of different scales and use dilated convolution, which can expand the receptive field without increasing the network depth and computational load. Use convolution kernels of different sizes to capture features from feature maps, further improving the network's ability to fuse features. It can expand the receptive field and obtain information about fastener anomalies more centrally, so that the features of fastener anomalies can be better refined by the network. This helps to better classify and distinguish defective railway fasteners with similar features.

[0085] S1-2: The feature map processed by S1-1 is processed again by EFOV to form a feature map of the second resolution, and is processed again by MP-1 layer;

[0086] S1-3: The feature map processed by S1-2 is processed again by EFOV to form a feature map of the third definition size;

[0087] S2: Multiple feature maps are fused and optimized through a feature fusion network;

[0088] S2-1: Process the feature map with the highest clarity through SPPCSPC. After processing, execute S2-2 and S2-6 at the same time;

[0089] S2-2: The feature map with the highest clarity after SPPCSPC processing is upsampled through the upsample module;

[0090] S2-3: Connect the feature map after upsampling with the feature map of the second definition. After the connection is completed, process the feature map through the ELANBlock layer. After the ELANBlock layer is processed, execute S2-4 and S2-5 at the same time.

[0091] S2-4: Upsample the feature map again through the upsample module, and connect the upsampled feature map with the feature map with the lowest definition. After the connection is completed, the feature map is processed through the ELANBlock layer; after the ELANBlock layer is processed, S2-5 and S2-7 are executed at the same time;

[0092] S2-5: Process the feature map through the OD-MP module to improve the fineness of the feature map, and then process the feature map through the ELANBlock layer to execute S2-6;

[0093] S2-6: Process the feature map through the OD-MP module to improve the fineness of the feature map, and then process the feature map through the ELANBlock layer to execute S2-7;

[0094] When the OD-MP module is processing, the input feature map is sent to two processing branches at the same time. One of the processing branches first adjusts the channel through a 1×1 convolution operation, and then performs a 3×3 convolution operation with a stride of 2 through the ODConv module for downsampling.

[0095] The other processing branch first downsamples through the maximum pooling operation, and then the ODConv module performs a 1×1 convolution operation to modify the number of channels;

[0096] Finally, the data of the two processing branches are added to achieve super downsampling;

[0097] The ODConv module replaces the ordinary 1×1 convolution of the MP-2 module. Using ODConv's multi-dimensional attention mechanism, the model can learn attention across multiple dimensions of the convolution kernel space. ODConv is also used to replace the ordinary 3×3 convolution in the MP-2 module, further reducing the computational complexity of the model and alleviating the memory access burden. By integrating this module, we can improve the ability to extract features of dark and low-quality fastener components and further improve the detection accuracy of the model.

[0098] S2-7: Processed by the EMA module to capture the local important information of the feature map and sent to the original head detection layer;

[0099] S2-7-1: Let the input feature map be X = [X0, X1, ..., XG-1], where Xi∈R C×H×W, through the EMA module, the input feature map is divided into G sub-features based on the number of channels, and G<<C;, to learn different semantic information, C represents the number of channels of the feature map, G<<C, indicating that the number of sub-feature maps is much smaller than the channel C; execute S2-7-2, S2-7-3, S2-7-9 at the same time;

[0100] S2-7-2: Perform a horizontal global pooling operation on the input data, encode the channel in the X-axis direction, and execute S2-7-4;

[0101] S2-7-3: Perform vertical global pooling operation on the input data and encode channels in the Y-axis direction;

[0102] S2-7-4: The two sets of data are first connected and subjected to a 1×1 convolution operation, and then activated by the softmax activation function, and then the sigmoid function is used to represent the 2D binary distribution of the linear convolution;

[0103] S2-7-5: Process through GroupNorm and execute S2-7-6 and S2-7-7 at the same time;

[0104] S2-7-6: first activate through softmax, and send the data to the Matmul module, and execute S2-7-8;

[0105] S2-7-7: perform global equal pooling operation and send the data to the Matmul module to execute S2-7-12;

[0106] S2-7-8: Execute S2-7-13 by processing the two sets of data Matmul;

[0107] S2-7-9: Perform a 3×3 convolution operation on the input data, and execute S2-7-10 and S2-7-11 at the same time;

[0108] S2-7-10: Activate through softmax and send the data to the Matmul module to execute S2-7-12;

[0109] S2-7-11: perform global pooling operation and send the data to the Matmul module to execute S2-7-8;

[0110] S2-7-12: Execute S2-7-13 by processing the two sets of data Matmul;

[0111] S2-7-13: Fuse the two sets of data, and after fusion, perform nonlinear mapping on the input attention scores through the Sigmoid function and output;

[0112] When the EMA module is processing, to not sacrifice generality, let G << C to ensure that the obtained attention weights are used to improve the representation of sub - features in all attention clusters. Next, EMA adopts three paths to retrieve the attention weights of the grouped feature maps, where there are 1×1 convolutions in the first two paths and a 3×3 convolution is used in the third path.

[0113] In the 1×1 path, two global average pooling operations are applied to encode the channels in the X and Y spatial directions. The encoded features from the two operations are concatenated without dimensionality reduction in the subsequent 1×1 convolution. The result of the 1×1 convolution is split into two vectors, and two Sigmoid non - linear functions are used to represent the 2D binary distribution of the linear convolution.

[0114] In the 3×3 convolution path, multi - scale feature descriptors are captured. Finally, channel attention is achieved through multiplication to promote cross - channel interaction. The output of the path is encoded with global spatial information using 2D global average pooling, and the output is transformed into the corresponding dimensional shape, i.e., R11×C / / G×R2C / / G×H×W. The global average pooling operation is performed through the following formula:

[0115]

[0116] where, Z c is the pooling result on channel c, H and W are the height and width of the input feature map respectively. xc(i,j) represents the value of channel c at position (i,j) in the input feature map. Finally, a non - linear softmax function is applied to fit the linear transformation. The output of the three branches is calculated by combining two spatial attention weight values, and the Sigmoid activation function is used to highlight the context pixels of each pixel, so that the final output has the same dimension as the input feature Figure X ∈RC×H×W.

[0117] This paper selects to integrate the EMA module at the ends of the three outputs of the Neck, effectively establishing short - term and long - term dependencies. This helps the model capture the spatial and temporal information of the target at different scales, enhancing the model's target recognition and localization capabilities. In addition, the attention mechanism in the EMA module helps the Neck focus on important feature channels, enabling it to better adapt to targets at different scales and low resolutions. Through multi - dimensional interaction, EMA enhances the representation ability of the feature pyramid and improves the robustness of the model.

[0118] S3: Perform the regression and classification process on the fused and optimized feature maps through the original head detection layer.

[0119] This technical solution aims to improve the accuracy of detecting anomalies in railway fastener assemblies under challenging conditions. The algorithm introduced in this study is designed specifically for detecting anomalies in railway fastener assemblies and is applicable to objects with high feature similarity and suboptimal image quality. Our proposed EFOV (Multi-branch Parallel and Enhanced Field of View) module is implemented through dilated convolutional layers with different dilation factors, simulating the high sensitivity of the human eye to the central region. This enables the network to better distinguish fastener assemblies with similar shapes. In addition, we introduce OD_MP and EMA (Efficient Multi-scale Attention) modules to enhance feature extraction capabilities. We comprehensively evaluate the performance and design characteristics of the model, revealing its significant advantages in accuracy, real-time and robustness, making it suitable for deployment on mobile devices. Future research will focus on addressing challenges such as adverse weather conditions, temperature fluctuations and humidity that occur in real environments, ensuring reliable detection and analysis of railway fastener assemblies under various environmental conditions.

[0120] In order to better illustrate this technical solution, the following examples are given:

[0121] In order to realize the image acquisition of high-speed railway fasteners, the image acquisition system used in this technical solution is as follows Figure 8 As shown. The system consists of a data acquisition module, a trigger module and an industrial computer, all of which are installed on the inspection vehicle. The data acquisition module includes an industrial camera, a structured light laser and an inertial navigation unit. The module uses a camera to capture the image of the structured light projected on the object, and obtains the laser reflection intensity and the object contour information. In order to improve the accuracy and reliability of the data, an inertial navigation unit is introduced to calibrate the collected data. The trigger module includes an encoder and a signal trigger distributor. The encoder generates a signal every time the vehicle moves forward a fixed distance, and transmits the signal to the distributor to trigger the camera to shoot. The encoder generates 10 signals every time it moves forward 1 mm, and collects data every 10 signals according to the inspection requirements. The industrial computer is responsible for controlling the start and stop of the inspection process, storing the collected data, and performing post-processing and analysis.

[0122] The acquired data is stored in DAT format with an XML data description file. By reading the data description file, images of rail fasteners of various sizes can be acquired. In order to obtain better inspection results, some parts of the rails are pre-configured with defects, displacement, fracture and deflection of rail fasteners.

[0123] We screened a set of images taken continuously from railway vehicles. These images have been stitched to create a composite image containing various types of fasteners. There are 1,288 images in total. Analysis of the dataset revealed several problems: images taken when railway vehicles were moving quickly had significant noise, uneven lighting, and image blur. In addition, the sample labels in the railway fastener dataset often exhibit imbalance, with the "normal" class accounting for more than 50% of the overall label distribution. To address these challenges, we applied data augmentation techniques to the dataset, including horizontal flipping, random brightness adjustment, random occlusion, and motion blur algorithms. Fig. 9 The types of data augmentation are shown. By applying the above augmentation algorithms, we obtained a dataset containing 3870 images, including real photos. We named the final dataset as 'GKA'. Subsequently, we divided the dataset and used 75% for training, 12.5% ​​for validation, and another 12.5% ​​for testing. The statistical comparison of the fastener categories before and after data augmentation is shown in the following table:

[0124] Table 1

[0125]

[0126]

[0127] For GKA annotation, we always follow the YOLO format. In order to thoroughly and fairly evaluate the performance of the model in this article, we use precision (PR), recall (RE), and mAP as evaluation metrics. Precision (PR) measures the proportion of true positive categories among all samples predicted as positive categories, providing an indicator of model accuracy. The calculation formula is as follows:

[0128]

[0129] Among them, TP represents the true positive category (i.e., the correctly predicted positive sample), and FP represents the false positive category (i.e., the incorrectly predicted positive sample).

[0130] Recall (RE) measures the proportion of actual positive categories that the model successfully predicts as positive categories. This metric is related to the model's coverage of positive categories, that is, how many positive categories the model captures without missing, and its calculation formula is as follows:

[0131]

[0132] Among them, FN represents the false negative category (i.e., the correctly predicted negative samples).

[0133] Average precision (AP) refers to the area under the precision-recall curve, while mAP represents the mean of AP for all categories and is calculated using the following formula:

[0134]

[0135] In the field of object detection, mAP@0.5 and mAP@0.5:0.95 are two key indicators for evaluating the performance of object detection models, which measure the accuracy of the model under different conditions. When evaluating the performance of object detection, the intersection over union (IoU) between the predicted bounding box and the true bounding box is often used as a criterion. mAP@0.5 calculates mAP and considers detection results with IoU greater than or equal to 0.5 as correct, while mAP@0.5:0.95 considers a series of IoU thresholds from 0.5 to 0.95 to calculate the average precision of the model.

[0136] FLOPs refers to the number of floating point operations and is used to measure the computational complexity of the model. Params represents the sum of all parameters in the training model and is an indicator of the model size. The calculation formula for FLOPs is as follows:

[0137] FLOPs = 2C out (C in K 2 +1)×H×W

[0138] Where H×W represents the size of the input feature map. in Indicates the number of channels of the input feature map, expressed as a number. K indicates the size of the convolution kernel.

[0139] The Params calculation formula is as follows:

[0140] Params=C out (H×W×C in +1)

[0141] Among them, C out Indicates the number of channels of the output feature map.

[0142] To verify the effectiveness and superiority of our proposed variant YOLO detection method, we conducted comparative experiments on the current relatively lightweight YOLO series models on the GKA dataset. To ensure a fair comparison, we trained all models from scratch without any pre-training and tested their runtime performance on RTX3060Ti. We trained and evaluated YOLOv4, YOLOv5, YOLOX, and YOLOv7 using the GKA dataset, and compared their experimental results with those of our proposed model. The comparison results are shown in the following table:

[0143] Table 2

[0144]

[0145] By comparing the results in the above table, although the number of parameters of our proposed model is higher than that of models such as YOLOv5 and YOLOX, it is significantly lower than that of YOLOv4. In addition, our model outperforms other models in terms of mAP@0.5 and mAP@0.5:0.95. At the same time, our proposed variant YOLO model achieves an inference time of 27.3ms, which can be considered to be at the advanced level. YOLOv4 shows the lowest overall performance on the GKA dataset. YOLOv5, the most widely used network at present, still has room for improvement in detection accuracy and speed compared with the network proposed in this paper. YOLOX is an anchor-free object detection model that performs well in overall performance, but has certain limitations in various aspects compared with the variant YOLO. The only bright spot is the number of parameters. The results are shown in Fig.10 As shown in the figure, we can observe the comparative performance of these models. Our model shows superior performance in terms of metrics such as precision, recall, and loss.

[0146] To further evaluate the robustness of the model, we used the NEU-DET dataset to perform an accuracy evaluation and compared our model with contemporary mainstream object detection models. NEU-DET is a public dataset for surface defect detection, containing 1,800 grayscale images with six different types of typical surface defects. We used 75% of the dataset for training and 25% for validation.

[0147] Although models such as Sparse R-CNN, Deformable DETR, and YOLOv7 have a comparable number of parameters to our model, they perform worse in terms of accuracy, as shown in the following table:

[0148] Table 3: Comparison results with other algorithms on the NEU-DET dataset

[0149]

[0150] Relatively speaking, YOLOv5 with a smaller number of parameters has the worst result. Our algorithm significantly outperforms Deformable DETR in terms of FLOPs. It is worth emphasizing that our proposed model is significantly higher in mAP@0.5 and mAP@0.5:0.95 compared to Sparse R-CNN, Deformable DETR, YOLOv5, and YOLOX. In addition, compared to YOLOv7, although the number of parameters and model calculations increased by 5.8 and 2.9GFLOPs respectively, our model improves mAP@0.5 and mAP@0.5:0.95 indicators by 1.8% and 1.3%, respectively, showing excellent performance.

[0151] In this paper, we introduce a new object detection model, namely variant YOLO. To verify the effectiveness of the model on the GKA dataset, we use YOLOv7 as the baseline. According to the method described in the paper, we made iterative improvements, successively introducing the enhanced EFOV module, OD_MP module and attention-enhanced EMA, and the final aggregate model is the model we proposed. The comparison results of the ablation study results of the solution network are shown in the following table:

[0152] Table 4

[0153]

[0154] As can be seen in the table above, after integrating the proposed EFOV structure into the base model, mAP@0.5 increased by 1.1%, mAP@0.5:0.95 increased by 0.8%, and the number of parameters and GFLOPs increased slightly. This is because the EFOV module replaces the original ELAN, introduces additional channels, and requires some computational overhead. Subsequently, the introduction of the proposed OD_MP module led to a further increase of 0.7% in mAP@0.5 and 0.5% in mAP@0.5:0.95, with a slight decrease in the number of parameters and a slight increase in GFLOPs compared to when the EFOV module was added. This shows that the module improves accuracy while keeping the model lightweight. The introduction of EMA further improved mAP@0.5 by 0.4% and mAP@0.5:0.95 by 0.1%, with a slight increase in the number of parameters and GFLOPs accordingly. Compared to the baseline YOLOv7, our proposed model achieves 2.2% improvement on mAP@0.5 and 1.4% improvement on mAP@0.5:0.95 with almost no increase in the number of parameters and GFLOPs.

[0155] At the same time, we have recorded the mAP of various types of railway fastener detection in different models, as shown in the following table:

[0156] Table 5

[0157]

[0158] It can be seen that the mAP of normal, tilted, broken, displaced, and defect types are all improved in our proposed model. In particular, the mAP of tilted and displaced types is relatively low in the baseline model. However, with the introduction of the EFOV module, the mAP increases significantly. One main reason is that the states of tilted and displaced types are similar, making them relatively difficult to distinguish in the original model. The introduction of the improved module enables the model to identify them more clearly. Fig.11The effects of the standard model YOLOv7 and variant YOLO on the detection results of railway fasteners are compared. The left, middle, and right sides of the figure show the input of the original railway fastener image, the detection results of YOLOv7, and the detection results of variant YOLO, respectively.

Claims

1. A railway fastener anomaly detection method based on variant YOLO, characterized by: The steps include: S1: Input image, extract features through the backbone network, and generate at least three feature maps of different resolutions; S2: Multiple feature maps are fused and optimized through a feature fusion network; S3: The fused and optimized feature map is regressed and classified through the original head detection layer; Step S1 also includes: S1-1: The image data processed by the convolution layer, ELANBlock layer and MP-1 layer is processed by EFOV to form a feature map of the first resolution size, and then processed by the MP-1 layer again; S1-2: The feature map processed by S1-1 is processed again by EFOV to form a feature map of the second resolution, and is processed again by MP-1 layer; S1-3: The feature map processed by S1-2 is processed again by EFOV to form a feature map of the third definition size; Step S1-1 also includes: S1-1-1: When EFOV is processing, the image data is first subjected to a 1x1 convolution operation, and S1-1-2, S1-1-3, S1-1-4, and S1-1-5 are executed simultaneously; S1-1-2: Execute S1-1-6; S1-1-3: Perform a 3x3 convolution operation on the image data and execute S1-1-6; S1-1-4: Perform two 3x3 convolution operations on the image data and execute S1-1-6; S1-1-5: Perform four 3x3 convolution operations on the image data and execute S1-1-7; S1-1-6: The image data processed by S1-1-2, S1-1-3 and S1-1-4 are fused and connected through the residual branch. After fusion and connection, a 1x1 convolution operation is performed and S1-1-7 is executed; S1-1-7: Connect the image data processed by S1-1-5 and S1-1-6, perform 1x1 convolution operation, and modify the number of channels; Step S2 also includes: S2-1: Process the feature map with the highest clarity through SPPCSPC. After processing, execute S2-2 and S2-6 at the same time; S2-2: The feature map with the highest clarity after SPPCSPC processing is upsampled through the upsample module; S2-3: Connect the feature map after upsampling with the feature map of the second definition. After the connection is completed, process the feature map through the ELANBlock layer. After the ELANBlock layer is processed, execute S2-4 and S2-5 at the same time. S2-4: Upsample the feature map again through the upsample module, and connect the upsampled feature map with the feature map with the lowest definition. After the connection is completed, the feature map is processed through the ELANBlock layer; after the ELANBlock layer is processed, S2-5 and S2-7 are executed at the same time; S2-5: Process the feature map through the OD-MP module to improve the fineness of the feature map, and then process the feature map through the ELANBlock layer to execute S2-6 and S2-7; S2-6: Process the feature map through the OD-MP module to improve the fineness of the feature map, and then process the feature map through the ELANBlock layer to execute S2-7; S2-7: Processed by the EMA module to capture the local important information of the feature map and sent to the original head detection layer.

2. The railway fastener anomaly detection method based on variant YOLO according to claim 1, characterized in that: Steps S2-5 and S2-6 also include: When the OD-MP module is processing, the input feature map is sent to two processing branches at the same time. One of the processing branches first adjusts the channel through a 1×1 convolution operation, and then performs a 3×3 convolution operation with a stride of 2 through the ODConv module for downsampling; The other processing branch first downsamples through the maximum pooling operation, and then the ODConv module performs a 1×1 convolution operation to modify the number of channels; Finally, the data of the two processing branches are added to achieve super downsampling.

3. The railway fastener anomaly detection method based on variant YOLO according to claim 1, characterized in that: Step S2-7 also includes: S2-7-1: Let the input feature map be X = [X0, X1, ..., XG-1], where Xi∈R C×H×W , the input feature map is divided into G sub-features based on the number of channels through the EMA module, and G<<C, C represents the number of channels of the feature map; S2-7-2, S2-7-3, and S2-7-9 are executed at the same time; S2-7-2: Perform a horizontal global pooling operation on the input data, encode the channel in the X-axis direction, and execute S2-7-4; S2-7-3: Perform vertical global pooling operation on the input data and encode channels in the Y-axis direction; S2-7-4: The two sets of data are first connected and subjected to a 1×1 convolution operation, and then activated by the softmax activation function, and then the sigmoid function is used to represent the 2D binary distribution of the linear convolution; S2-7-5: Process through GroupNorm and execute S2-7-6 and S2-7-7 at the same time; S2-7-6: first activate through softmax, and send the data to the Matmul module, and execute S2-7-8; S2-7-7: perform global equal pooling operation and send the data to the Matmul module to execute S2-7-12; S2-7-8: Execute S2-7-13 by processing the two sets of data Matmul; S2-7-9: Perform a 3×3 convolution operation on the input data, and execute S2-7-10 and S2-7-11 at the same time; S2-7-10: Activate through softmax and send the data to the Matmul module to execute S2-7-12; S2-7-11: perform global pooling operation and send the data to the Matmul module to execute S2-7-8; S2-7-12: Execute S2-7-13 by processing the two sets of data Matmul; S2-7-13: Fuse the two sets of data, and after fusion, perform nonlinear mapping on the input attention score through the Sigmoid function and output it.

4. The railway fastener anomaly detection method based on variant YOLO according to claim 3, characterized in that: When performing a global average pooling operation, the global average pooling operation is performed using the following formula: Among them, Z c It is the pooling result on channel c, H and W are the height and width of the input feature map respectively, and xc(i,j) represents the value of channel c at position (i,j) in the input feature map.

Images