Railway track fastener defect detection method, system and device and medium

Abstract

The invention discloses a railway track fastener defect detection method, system and device and a medium, and relates to the technical field of defect detection, and the method comprises the steps: obtaining a target image; the target image is an image containing a to-be-detected railway track fastener; inputting the target image into a railway track fastener defect detection model to obtain a target defect detection result; the railway track fastener defect detection model is obtained by training an initial network, and the initial network is constructed based on a wavelet transform convolution base block, a wavelet feature upgrading network, a cross-stage partial parallel expansion convolution network and a channel gating attention down-sampling network; the target defect detection result is a target image marked with a prediction bounding box and a prediction category of defects contained in the to-be-detected railway track fastener. According to the invention, real-time detection of the defects of the railway track fastener can be realized, and the detection precision is improved.

Description

Technical Field

[0001] This application relates to the technical field of defect detection, and particularly to a method, system, device and medium for detecting defects in railway track fasteners. Background Art

[0002] Railway track fasteners, as the core components of railway infrastructure, undertake the key functions of fixing rails, absorbing vibrations and maintaining track stability. However, during long-term service, environmental erosion, train loads and human factors are likely to cause defects such as loosening, fracture, displacement or missing of fasteners, seriously threatening the stability of the track system and the safety of train operation. Therefore, real-time and accurate detection of the fastener state has both engineering value and safety significance.

[0003] Traditional track fastener detection relies on manual inspection. Maintenance personnel need to visually inspect the fastener state along the track, manually record the abnormalities after discovery and arrange for maintenance. Although this method has a certain degree of intuitiveness, its limitations are also very obvious: First, the efficiency of manual detection is low, making it difficult to meet the real-time monitoring requirements of large-scale railway systems; Second, human factors are likely to lead to inconsistent detection results, which are greatly affected by subjective judgment; Third, the working intensity of long-term detection is high, posing safety risks. With the rapid expansion of the railway network, traditional manual detection can no longer meet the modern track maintenance requirements in terms of efficiency and accuracy.

[0004] In recent years, the development of computer vision technology has provided new solutions for track fastener detection. Early image processing-based methods used techniques such as edge detection and morphological processing to extract the fastener area, and achieved fastener state recognition through template matching or feature analysis. Hong et al. proposed a high-speed railway fastener detection method based on the least significant region and center symmetric local binary pattern, improving the detection performance through accurate fastener positioning and low-dimensional feature extraction. However, there are still limitations in adapting to complex environments and detecting partially missing fasteners. R. Manikandan et al. proposed a machine vision-based method to automatically detect and classify missing fasteners in track images using a support vector machine classifier. However, the complex track environment and the variability of fastener states (such as light changes, stain interference, morphological differences) are likely to cause such methods to fail, with insufficient robustness and generalization ability. These methods more or less have problems such as complex image processing and overly redundant feature extraction, and are difficult to adapt to changes in behavior.

[0005] With the development of deep learning, object detection methods based on convolutional neural networks have been widely applied in the field of track fastener detection. Among them, object detection algorithms are mainly divided into two categories: two-stage and one-stage. Two-stage algorithms require two-step object recognition based on candidate boxes. One-stage algorithms can directly predict the location and category of objects without generating candidate regions. For example, the YOLO (You Only Look Once) series is widely used in fastener defect detection tasks due to its end-to-end detection ability and high real-time performance.

[0006] However, the YOLO series of algorithms still face challenges in detecting small fastener defects. They must first generate a large number of candidate frames and then apply the non-maximum suppression algorithm to eliminate inappropriate frames, which results in high computational consumption.

[0007] In contrast, the Transformer-based fastener defect method provides a more concise and efficient approach. The Transformer has a powerful self-attention mechanism that can more accurately classify fastener defects in complex environments. However, this powerful mechanism comes at a high computational cost. Common Transformer-based object detection models include DETR (DEtection TRansformer), Deformable DETR, and DINO. Although these models are superior to traditional YOLO algorithms in terms of accuracy, they are lacking in real-time detection performance. In 2023, Zhao et al. proposed the first real-time end-to-end detection transformer RT-DETR, which enhances multi-scale feature processing through intra-scale interaction and cross-scale fusion. Zhang et al. proposed the TSD-DETR model, which improves the detection performance of small objects by constructing a multi-scale feature extraction module and introducing an efficient multi-scale attention mechanism. Song et al. improved the RT-DETR model and proposed a super-resolution convolution module for enhancing image details, while introducing channel attention in the self-attention mechanism to improve the model's attention to fastener defects. Experimental results show that this method improves the detection accuracy while achieving better utilization of computational resources. In addition, for special scenarios such as occlusion and complex background interference in railway fastener detection, Bai et al. proposed a detection method based on TSR-Net (deep learning-based super-resolution technology). This method integrates a vision transformer, an inverted residual block, and a self-supervised transform attention mechanism to enhance the model's detection ability in complex scenarios. Experimental results show that this method has high accuracy and robustness in detecting fasteners occluded by foreign objects. However, its computational cost is relatively high, which limits applications with strict real-time requirements.

[0008] In summary, there is a need for a railway track fastener defect detection method that can improve the accuracy of fastener defect detection while ensuring efficient real-time detection. Summary of the Invention

[0009] The purpose of this application is to provide a railway track fastener defect detection method, system, device and medium to solve the problem that the traditional railway track fastener defect detection method cannot improve the accuracy of fastener defect detection while ensuring efficient real-time detection.

[0010] To achieve the above purpose, the following solutions are provided in this application.

[0011] In the first aspect, this application provides a railway track fastener defect detection method, including: Obtain a target image; the target image is an image containing the railway track fastener to be detected; Input the target image into a railway track fastener defect detection model to obtain a target defect detection result; the railway track fastener defect detection model is obtained by training an initial network, and the initial network is constructed based on a wavelet transform convolutional basic block, a wavelet feature upgrade network, a cross-stage partial parallel dilated convolutional network, and a channel gating attention downsampling network; the target defect detection result is a target image with a predicted bounding box and predicted category of the defect contained in the railway track fastener to be detected marked.

[0012] In one embodiment, the determination process of the railway track fastener defect detection model includes: Obtain multiple sample images and corresponding true defect detection results; the sample images are images containing sample railway track fasteners, and the true defect detection results are sample images with true bounding boxes and true categories of the defects contained in the sample railway track fasteners marked; Construct the initial network; Use each sample image as input and the corresponding true defect detection result as output to train the initial network to obtain the railway track fastener defect detection model.

[0013] In one embodiment, the initial network includes: a backbone network, a hybrid encoder, and a Transform decoder; The backbone network includes: a first convolutional block, a second convolutional block, a third convolutional block, a first max pooling layer, a first wavelet transform convolutional basic block, a second wavelet transform convolutional basic block, a third wavelet transform convolutional basic block, a fourth wavelet transform convolutional basic block, and a fourth convolutional block connected in sequence; The hybrid encoder includes: a fifth convolutional block, a sixth convolutional block, an attention-based intra-scale feature interaction module, and a seventh convolutional block, a first wavelet feature upsampling module, a first cross-stage partially parallel dilated convolutional module, an eighth convolutional block, a second wavelet feature upsampling module, a second cross-stage partially parallel dilated convolutional module, a first channel gating attention downsampling module, a first adder, a third cross-stage partially parallel dilated convolutional module, a second channel gating attention downsampling module, a second adder, and a fourth cross-stage partially parallel dilated convolutional module connected in sequence; the second wavelet transform convolutional basic block is connected to the fifth convolutional block, the third wavelet transform convolutional basic block is connected to the sixth convolutional block, the attention-based intra-scale feature interaction module is connected to the seventh convolutional block, the seventh convolutional block is connected to the second adder, and the eighth convolutional block is connected to the first adder; The second cross-stage partially parallel dilated convolutional module, the third cross-stage partially parallel dilated convolutional module, and the fourth cross-stage partially parallel dilated convolutional module are all connected to the Transform decoder.

[0014] In one embodiment, both the first wavelet feature upsampling module and the second wavelet feature upsampling module are wavelet feature upsampling networks, and the wavelet feature upsampling network includes: a wavelet transform unit, a residual block, a third adder, and a wavelet inverse transform unit; The wavelet transform unit is respectively connected to the residual block and the third adder; the third adder and the residual block are respectively connected to the wavelet inverse transform unit.

[0015] In one embodiment, the first cross-stage partially parallel dilated convolutional module, the second cross-stage partially parallel dilated convolutional module, the third cross-stage partially parallel dilated convolutional module, and the fourth cross-stage partially parallel dilated convolutional module are all cross-stage partially parallel dilated convolutional networks, and the cross-stage partially parallel dilated convolutional network includes: a ninth convolutional block, a tenth convolutional block, a parallel dilated convolutional unit, a fourth adder, and an eleventh convolutional block; the parallel dilated convolutional unit includes: a twelfth convolutional block, a thirteenth convolutional block, a fourteenth convolutional block, a fifth adder, and a fifteenth convolutional block; The ninth convolutional block is connected to the fourth adder, the tenth convolutional block is respectively connected to the twelfth convolutional block, the thirteenth convolutional block, and the fourteenth convolutional block, the twelfth convolutional block, the thirteenth convolutional block, and the fourteenth convolutional block are all connected to the fifth adder, the fifth adder is connected to the fifteenth convolutional block, the fifteenth convolutional block is connected to the fourth adder, and the fourth adder is connected to the eleventh convolutional block.

[0016] In one embodiment, both the first channel gated attention downsampling module and the second channel gated attention downsampling module are channel gated attention downsampling networks, and the channel gated attention downsampling network includes: a global average pooling layer, a sixteenth convolutional block, a second max pooling layer, a channel gated unit, a sixth adder, a seventeenth convolutional block, a multiplier, and an eighteenth convolutional block; the channel gated unit includes: a nineteenth convolutional block and a Hardsigmoid activation function; The global average pooling layer is connected to the nineteenth convolutional block, the nineteenth convolutional block is connected to the Hardsigmoid activation function, the second max pooling layer is connected to the seventeenth convolutional block, both the sixteenth convolutional block and the seventeenth convolutional block are connected to the sixth adder, both the Hardsigmoid activation function and the sixth adder are connected to the multiplier, and the multiplier is connected to the eighteenth convolutional block.

[0017] In one embodiment, the target image is input into the railway track fastener defect detection model to obtain a target defect detection result, including: The target image is input into the backbone network in the railway track fastener defect detection model to obtain a first backbone output feature map, a second backbone output feature map, and a third backbone output feature map of the railway track fastener to be detected; The first backbone output feature map, the second backbone output feature map, and the third backbone output feature map are all input into the hybrid encoder in the railway track fastener defect detection model to obtain a first encoded output feature map, a second encoded output feature map, and a third encoded output feature map of the railway track fastener to be detected; The first encoded output feature map, the second encoded output feature map, and the third encoded output feature map are all input into the Transform decoder in the railway track fastener defect detection model to obtain the target defect detection result.

[0018] In a second aspect, the present application provides a railway track fastener defect detection system to implement the railway track fastener defect detection method described in any one of the above, and the railway track fastener defect detection system includes: An image acquisition module, configured to acquire a target image; the target image is an image including a railway track fastener to be detected; A defect detection module, configured to input the target image into the railway track fastener defect detection model to obtain a target defect detection result; the railway track fastener defect detection model is obtained by training an initial network; the target defect detection result is a target image marked with a predicted bounding box and a predicted category of the defect included in the railway track fastener to be detected.

[0019] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the railway track fastener defect detection method described in any one of the above.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the railway track fastener defect detection method described in any one of the above.

[0021] According to the specific embodiments provided by the present application, the following technical effects are disclosed in the present application: The present application discloses a railway track fastener defect detection method, system, device and medium. First, a target image is obtained; the target image is an image containing the railway track fastener to be detected; then, the target image is input into a railway track fastener defect detection model to obtain a target defect detection result; the target defect detection result is a target image with a predicted bounding box and predicted category of the defect contained in the railway track fastener to be detected marked. The railway track fastener defect detection model in the present application is obtained by training an initial network, and the initial network is constructed based on a wavelet transform convolutional basic block, a wavelet feature upgrade network, a cross-stage partial parallel dilated convolutional network, and a channel gating attention downsampling network. The wavelet transform convolutional basic block is introduced to fuse high-frequency and low-frequency features, enhance the feature extraction ability, and at the same time reduce the computational complexity; the cross-stage partial parallel dilated convolutional network and the channel gating attention downsampling network are used to enhance effective features and suppress redundant features; by applying the wavelet feature upgrade network to optimize the feature fusion network, strengthen the fusion of high-frequency and low-frequency information, improve the sensitivity of the railway track fastener defect detection model to fastener defects, and ensure efficient real-time detection while improving the accuracy of railway track fastener defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0023] Figure 1 It is an application environment diagram of a railway track fastener defect detection method in an embodiment of the present application.

[0024] Figure 2 It is a schematic flowchart of a railway track fastener defect detection method provided by an embodiment of the present application.

[0025] Figure 3It is a schematic diagram of the initial network structure.

[0026] Figure 4 It is a schematic diagram of the wavelet feature upgrade network structure.

[0027] Figure 5 It is a schematic diagram of the cross-stage partially parallel dilated convolutional network structure.

[0028] Figure 6 It is a schematic diagram of the channel gated attention downsampling network structure.

[0029] Figure 7 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0031] The purpose of the present application is to provide a method, system, device and medium for detecting defects in railway track fasteners, aiming to improve the detection accuracy of railway track fastener defects while ensuring efficient real-time detection.

[0032] To make the above objects, features and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0033] The method for detecting defects in railway track fasteners provided in the embodiments of the present application can be applied to an application environment as shown in Figure 1 In the figure. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set separately, integrated on the server 104, placed in the cloud or on other servers. The terminal 102 can send the target image to the server 104. After receiving the target image, for the target image, the server 104 inputs the target image into the railway track fastener defect detection model to obtain the target defect detection result. The server 104 can feedback the obtained target defect detection result to the terminal 102. In addition, in some embodiments, the method for detecting defects in railway track fasteners can also be implemented by the server 104 or the terminal 102 alone. For example, the terminal 102 can directly perform the detection of railway track fastener defects on the target image, or the server 104 can obtain the target image from the data storage system and perform the detection of railway track fastener defects on the target image.

[0034] In an exemplary embodiment, as Figure 2 shown, a method for detecting defects in railway track fasteners is provided, including Step 1 - Step 2.

[0035] Step 1: Obtain a target image; the target image is an image containing the railway track fastener to be detected.

[0036] Step 2: Input the target image into the railway track fastener defect detection model to obtain a target defect detection result.

[0037] Among them, the railway track fastener defect detection model is obtained by training an initial network, and the initial network is constructed based on a wavelet transform convolutional basic block, a wavelet feature upgrade network, a cross-stage partial parallel dilated convolutional network, and a channel gating attention downsampling network; the target defect detection result is a target image with a predicted bounding box and a predicted category of the defect contained in the railway track fastener to be detected marked.

[0038] As an alternative implementation, the determination process of the railway track fastener defect detection model includes the following steps.

[0039] Obtain multiple sample images and corresponding true defect detection results; the sample images are images containing sample railway track fasteners, and the true defect detection results are sample images with true bounding boxes and true categories of the defects contained in the sample railway track fasteners marked.

[0040] Construct an initial network.

[0041] Use each sample image as input and the corresponding true defect detection result as output to train the initial network to obtain a railway track fastener defect detection model.

[0042] During training, the accuracy, average accuracy, and mean average precision evaluation metrics of the railway track fastener defect detection model are also calculated using a test set, and the formulas are as follows.

[0043] .

[0044] .

[0045] .

[0046] Among them, is the accuracy; is the number of correctly detected positive samples; is the number of incorrectly detected positive samples; is the average accuracy; is the P-R curve calculation function; is the recall rate; is the mean average precision; is the total number of categories in the test set; is the average precision of the

[0047] As an alternative implementation, as shown in Figure 3 , the initial network includes: a backbone network, a hybrid encoder, and a Transformer decoder.

[0048] The backbone network includes: a first convolutional block, a second convolutional block, a third convolutional block, a first max pooling layer (Max-Pool), a first wavelet transform convolutional basic block, a second wavelet transform convolutional basic block, a third wavelet transform convolutional basic block, a fourth wavelet transform convolutional basic block, and a fourth convolutional block connected in sequence.

[0049] Specifically, the first convolutional block, the second convolutional block, and the third convolutional block are all convolutional blocks (ConvN ) with a kernel size of 3×3. The fourth convolutional block is a convolutional block (ConvN ) with a kernel size of 1×1.

[0050] The first wavelet transform convolutional basic block, the second wavelet transform convolutional basic block, the third wavelet transform convolutional basic block, and the fourth wavelet transform convolutional basic block are all wavelet transform convolutional basic blocks (BasciBlock WTCnov).

[0051] The hybrid encoder includes: a fifth convolutional block, a sixth convolutional block, an attention-based intrascale feature interaction module (Attention-based Intrascale Feature Interaction, AIFI), and a seventh convolutional block, a first wavelet feature upgrade module, a first cross-stage partial parallel dilated convolutional module, an eighth convolutional block, a second wavelet feature upgrade module, a second cross-stage partial parallel dilated convolutional module, a first channel gated attention downsampling module, a first adder, a third cross-stage partial parallel dilated convolutional module, a second channel gated attention downsampling module, a second adder, and a fourth cross-stage partial parallel dilated convolutional module connected in sequence; the second wavelet transform convolutional basic block is connected to the fifth convolutional block, the third wavelet transform convolutional basic block is connected to the sixth convolutional block, the attention-based intrascale feature interaction module is connected to the seventh convolutional block, the seventh convolutional block is connected to the second adder, and the eighth convolutional block is connected to the first adder.

[0052] Specifically, the fifth convolutional block, the sixth convolutional block, the seventh convolutional block, and the eighth convolutional block are all convolutional blocks with a kernel size of 1×1.

[0053] The second cross-stage partial parallel dilated convolution module, the third cross-stage partial parallel dilated convolution module, and the fourth cross-stage partial parallel dilated convolution module are all connected to the Transform decoder.

[0054] As an optional implementation, both the first wavelet feature upgrade module and the second wavelet feature upgrade module are wavelet feature upgrade networks (WFU), as Figure 4 shown, the wavelet feature upgrade network includes: a wavelet transform unit, a residual block, a third adder, and a wavelet inverse transform unit.

[0055] The wavelet transform unit is respectively connected to the residual block and the third adder; the third adder and the residual block are respectively connected to the wavelet inverse transform unit.

[0056] As an optional implementation, the first cross-stage partial parallel dilated convolution module, the second cross-stage partial parallel dilated convolution module, the third cross-stage partial parallel dilated convolution module, and the fourth cross-stage partial parallel dilated convolution module are all cross-stage partial parallel dilated convolution networks (CSPPDC), as Figure 5 shown, the cross-stage partial parallel dilated convolution network includes: a ninth convolution block, a tenth convolution block, a parallel dilated convolution unit, a fourth adder, and an eleventh convolution block; the parallel dilated convolution unit includes: a twelfth convolution block, a thirteenth convolution block, a fourteenth convolution block, a fifth adder, and a fifteenth convolution block.

[0057] The ninth convolution block is connected to the fourth adder, the tenth convolution block is respectively connected to the twelfth convolution block, the thirteenth convolution block, and the fourteenth convolution block, the twelfth convolution block, the thirteenth convolution block, and the fourteenth convolution block are all connected to the fifth adder, the fifth adder is connected to the fifteenth convolution block, the fifteenth convolution block is connected to the fourth adder, and the fourth adder is connected to the eleventh convolution block.

[0058] Specifically, the ninth convolution block, the tenth convolution block, and the fifteenth convolution block are all convolution blocks with a convolution kernel size of 1×1. The twelfth convolution block, the thirteenth convolution block, and the fourteenth convolution are all convolution blocks with a convolution kernel size of 3×3.

[0059] As an optional implementation, both the first channel gated attention downsampling module and the second channel gated attention downsampling module are channel gated attention downsampling networks (CGAD), as Figure 6As shown in the figure, the channel gating attention downsampling network includes: a global average pooling layer, a sixteenth convolutional block, a second max pooling layer, a channel gating unit, a sixth adder, a seventeenth convolutional block, a multiplier, and an eighteenth convolutional block; the channel gating unit includes: a nineteenth convolutional block and a Hardsigmoid activation function.

[0060] The global average pooling layer is connected to the nineteenth convolutional block, the nineteenth convolutional block is connected to the Hardsigmoid activation function, the second max pooling layer is connected to the seventeenth convolutional block, both the sixteenth convolutional block and the seventeenth convolutional block are connected to the sixth adder, both the Hardsigmoid activation function and the sixth adder are connected to the multiplier, and the multiplier is connected to the eighteenth convolutional block.

[0061] Specifically, the sixteenth convolutional block is a convolutional block with a convolution kernel size of 3×3. The seventeenth convolutional block, the eighteenth convolutional block, and the nineteenth convolutional block are all convolutional blocks with a convolution kernel size of 1×1.

[0062] As an alternative implementation, step 2 includes steps 21 - 23.

[0063] Step 21: Input the target image into the backbone network in the railway track fastener defect detection model to obtain the first backbone output feature map, the second backbone output feature map, and the third backbone output feature map of the railway track fastener to be detected.

[0064] Specifically, step 21 includes steps 211 - 219.

[0065] Step 211: Input the target image into the first convolutional block in the backbone network. The first convolutional block downsamples the resolution of the target image from 640×640 to 320×320, extracts primary features such as edges / textures, and obtains the first convolutional output feature map ; The expression is: .

[0066] Among them, is the Rectified Linear Unit activation (ReLU) function, which is used to introduce the non - linear mapping of the neural network; is the batch normalization operation, which is used to accelerate the network convergence and improve the stability; is the convolution operation with a convolution kernel size of 3×3 and a stride of 2; is the target image.

[0067] Step 212: Input the first convolutional output feature map Input it into the second convolutional block of the backbone network to enhance the feature expression ability and retain the local details of the railway track fasteners to be detected, obtaining the second convolutional output feature map ; The expression is: .

[0068] Among them, is a convolutional operation with a convolutional kernel size of 3×3 and a stride of 1.

[0069] Step 213: Input the second convolutional output feature map into the third convolutional block of the backbone network to expand the feature channels to 64 and capture higher-dimensional semantic information, obtaining the third convolutional output feature map ; The expression is: .

[0070] Step 214: Input the third convolutional output feature map into the first max-pooling layer of the backbone network. After max-pooling operation, downsample it to 160×160 to retain the key structural features, obtaining the first max-pooling output feature map ; The expression is: .

[0071] Among them, is a max-pooling operation with a pooling kernel size of 3×3 and a stride of 2, which is used for downsampling and retaining the maximum response value in the local features.

[0072] Step 215: Input the first max-pooling output feature map into the first wavelet transform convolutional basic block of the backbone network, obtaining the first wavelet transform convolutional output feature map . Specifically, it includes Step 2151 - Step 2153.

[0073] Step 2151: Use wavelet transform to decompose the first max-pooling output feature map into a low-frequency component and three high-frequency components 、 、 , obtaining the decomposed first max-pooling output feature map ; The expression is: .

[0074] Among them, is the wavelet transform operation.

[0075] Step 2152: Perform inverse wavelet transform reconstruction on the decomposed first max-pooling output feature map to reconstruct it into the first spatial domain feature map ; The expression is: .

[0076] Among them, is the inverse wavelet transform operation.

[0077] Step 2153: Add the first spatial domain feature map and the first max-pooling output feature map to obtain the first wavelet transform convolution output feature map ; The expression is: .

[0078] Step 216: Input the first wavelet transform convolution output feature map into the second wavelet transform convolution basic block of the backbone network to obtain the first backbone output feature map ; Specifically, it includes steps 2161 - 2164.

[0079] Step 2161: Downsample the resolution of the first wavelet transform convolution output feature map from 160×160 to 80×80 through a 3×3 convolution with a stride of 2, and expand the number of channels to 128 to obtain the second wavelet transform convolution intermediate feature map ; The expression is: .

[0080] Step 2162: Use wavelet transform to decompose the second wavelet transform convolution intermediate feature map into a low-frequency component and three high-frequency components , , to obtain the decomposed second wavelet transform convolution intermediate feature map ; The expression is: .

[0081] Step 2163: Perform inverse wavelet transform reconstruction on the decomposed second wavelet transform convolution intermediate feature map to reconstruct it into the second spatial domain feature map ; The expression is: .

[0082] Step 2164: Add the second spatial domain feature map and the first wavelet transform convolution output feature map to obtain the first backbone output feature map ; The expression is: 。

[0083] Step 217: Input the first backbone output feature map into the third wavelet transform convolutional basic block of the backbone network to obtain the second backbone output feature map ; Specifically, it includes steps 2171 - 2174.

[0084] Step 2171: Downsample the resolution of the first backbone output feature map from 80×80 to 40×40 with a stride of 2 in a 3×3 convolution, and expand the number of channels to 256 to obtain the intermediate feature map of the third wavelet transform convolution ; The expression is:[[]] 。

[0085] Step 2172: Use wavelet transform to decompose the intermediate feature map of the third wavelet transform convolution into a low-frequency component and three high-frequency components 、 、 to obtain the decomposed intermediate feature map of the third wavelet transform convolution ; The expression is:[[]] 。

[0086] Step 2173: Perform inverse wavelet transform reconstruction on the decomposed intermediate feature map of the third wavelet transform convolution to reconstruct it into the third spatial domain feature map ; The expression is:[[]] 。

[0087] Step 2174: Add the third spatial domain feature map to the first backbone output feature map to obtain the second backbone output feature map ; The expression is:[[]] 。

[0088] Step 218: Input the second backbone output feature map into the fourth wavelet transform convolutional basic block of the backbone network to obtain the output feature map of the fourth wavelet transform convolution ; Specifically, it includes steps 2181 - 2184.

[0089] Step 2181: Downsample the resolution of the second backbone output feature map from 40×40 to 20×20 with a stride of 2 in a 3×3 convolution, and expand the number of channels to 512 to obtain the intermediate feature map of the fourth wavelet transform convolution ; The expression is: 。

[0090] Step 2182: Use wavelet transform to decompose the fourth wavelet transform convolution intermediate feature map into a low-frequency component and three high-frequency components 、 、 , obtaining the decomposed fourth wavelet transform convolution intermediate feature map ; The expression is: 。

[0091] Step 2183: Perform inverse wavelet transform reconstruction on the decomposed fourth wavelet transform convolution intermediate feature map to reconstruct it into the fourth spatial domain feature map ; The expression is: 。

[0092] Step 2184: Add the fourth spatial domain feature map to the second backbone output feature map to obtain the fourth wavelet transform convolution output feature map ; The expression is: 。

[0093] Step 219: Input the fourth wavelet transform convolution output feature map into the fourth convolution block of the backbone network. Perform input projection through 1×1 convolution to compress the number of channels of the fourth wavelet transform convolution output feature map from 512 to 256, obtaining the third backbone output feature map ; The expression is: 。

[0094] Among them, is the 1×1 convolution operation.

[0095] Step 22: Input the first backbone output feature map, the second backbone output feature map, and the third backbone output feature map into the hybrid encoder in the railway track fastener defect detection model to obtain the first encoded output feature map, the second encoded output feature map, and the third encoded output feature map of the railway track fastener to be detected.

[0096] Specifically, Step 22 includes Step 2201 - Step 2213.

[0097] Step 2201: Input the third backbone output feature map Input it into the attention-based intra-scale feature interaction module of the hybrid encoder to obtain the feature interaction output feature map ; Specifically, it includes steps 22011 - 22016.

[0098] Step 22011: Flatten the third backbone output feature map into a sequence form to obtain the flattened feature ; The expression is: .

[0099] Among them, is the flattening operation, which is used to convert the multi-dimensional feature map into a one-dimensional vector for subsequent fully connected or classification processing.

[0100] Step 22012: Generate a two-dimensional sine-cosine position encoding based on the spatial coordinates (row and column indices) of the third backbone output feature map , and add the two-dimensional sine-cosine position encoding to the flattened feature to obtain the added feature map ; The expression is: .

[0101] Step 22013: Use the 8-head attention mechanism to capture global dependencies, perform attention calculation, and obtain the attention feature ; The expression is: .

[0102] Among them, is the sofemax activation function; and come from the added feature map , and are used to calculate the attention weights; is the transpose; comes from the flattened feature , is used to store the actual feature values.

[0103] Step 22014: Perform residual connection and layer normalization on the attention feature and the added feature map to obtain the normalized feature map ; The expression is: .

[0104] Among them, is the layer normalization operation, which is used to normalize across feature dimensions to improve training stability and robustness; It is a random inactivation mechanism module, which is used to randomly discard some neurons during the training process to reduce overfitting.

[0105] Step 22015: The normalized feature map , after being processed by the feedforward network, enhances the non-linearity through two fully connected layers to obtain the non-linear feature map ; The expression is: .

[0106] Among them, is the Gaussian error linear unit activation function; is the weight matrix of the first fully connected layer; is the weight matrix of the second fully connected layer.

[0107] Step 22016: Perform residual connection and layer normalization on the non-linear feature map and the normalized feature map to obtain the feature interaction output feature map ; The expression is: .

[0108] Step 2202: Input the first backbone output feature map , the second backbone output feature map and the feature interaction output feature map into the fifth convolution block, the sixth convolution block and the seventh convolution block of the hybrid encoder respectively, and unify the output feature channels to 256 through 1×1 convolution to obtain the unified first backbone output feature map, the unified second backbone output feature map and the unified feature interaction output feature map.

[0109] Step 2203: Input the unified second backbone output feature map and the unified feature interaction output feature map into the first wavelet feature upgrade module in the hybrid encoder to obtain the first upsampled feature.

[0110] Step 2204: Input the unified first backbone output feature map into the first cross-stage partial parallel dilated convolution module of the hybrid encoder to obtain the feature map output by the first cross-stage partial parallel dilated convolution module.

[0111] Step 2205: Input the feature map output by the first cross-stage partial parallel dilated convolution module into the eighth convolution block in the hybrid encoder to obtain the feature map output by the eighth convolution block.

[0112] Step 2206: Input the unified first backbone output feature map and the feature map output by the eighth convolutional block into the second wavelet feature upgrade module of the hybrid encoder to obtain the feature map output by the second wavelet feature upgrade module.

[0113] Step 2207: Input the feature map output by the second wavelet feature upgrade module into the second cross-stage partial parallel dilated convolution module of the hybrid encoder to obtain the first encoded output feature map.

[0114] Step 2208: Input the first encoded output feature map into the first channel gated attention downsampling module of the hybrid encoder to obtain the feature map output by the first channel gated attention downsampling module.

[0115] Step 2209: Input the feature map output by the first channel gated attention downsampling module and the feature map output by the eighth convolutional block into the first adder of the hybrid encoder to obtain the feature map output by the first adder.

[0116] Step 2210: Input the feature map output by the first adder into the Si'an cross-stage partial parallel dilated convolution module of the hybrid encoder to obtain the second encoded output feature map.

[0117] Step 2211: Input the second encoded output feature map into the second channel gated attention downsampling module of the hybrid encoder to obtain the feature map output by the second channel gated attention downsampling module.

[0118] Step 2212: Input the feature map output by the second channel gated attention downsampling module and the unified feature interaction output feature map into the second adder of the hybrid encoder to obtain the feature map output by the second adder.

[0119] Step 2213: Input the feature map output by the second adder into the third cross-stage partial parallel dilated convolution module of the hybrid encoder to obtain the third encoded output feature map.

[0120] Among them, any two feature maps and are input into the wavelet feature upgrade network, and the processing process of the wavelet feature upgrade network for the feature maps and includes S11 - S15.

[0121] S11: Through the wavelet transform unit, use the Haar wavelet transform to decompose : .

[0122] Among them, are the four subbands of the decomposed ; is a low-frequency sub-band; , and are three high-frequency sub-bands, corresponding to horizontal, vertical, and diagonal details respectively; is the Haar wavelet transform. The low-frequency sub-band contains most of the image energy and structural information, while the high-frequency sub-bands contain rich detail information such as edges and textures.

[0123] S12: Add the three high-frequency sub-bands , , together, and further enhance using a residual block to obtain the enhanced feature map : .

[0124] Among them, is the residual block. Using the residual block for further enhancement to highlight the image detail information. The design of the residual block effectively avoids the problem of gradient disappearance and enhances the feature representation ability.

[0125] S13: Through the third adder, splice the low-frequency sub-band with the feature map to generate the fused feature map : .

[0126] Among them, is the splicing operation.

[0127] S14: Reduce the dimension of the fused feature map to 256 channels through 1×1 convolution, and the output is the dimension-reduced fused feature map : .

[0128] The role of channel transformation is to effectively fuse features of different scales, thereby extracting more representative feature representations. The channel transformation module consists of two convolutional layers, which are used for feature dimension reduction and expansion respectively to achieve effective interaction between features of different scales.

[0129] S15: Through the inverse wavelet transform unit, use the inverse Haar wavelet transform, based on the wavelet feature upgrade network and the dimension-reduced fused feature map , to obtain the final upsampled feature map output by the wavelet feature upgrade network: .

[0130] Among them, is the inverse Haar wavelet transform.

[0131] WFU can effectively fuse features at different scales and enhance the detailed information of the image. Compared with traditional upsampling methods, WFU not only avoids the aliasing problem caused by the direct fusion of high-frequency and low-frequency components, but also better preserves the structural and texture information of the fasteners through the multi-resolution characteristics of wavelet transform.

[0132] Among them, any feature map is input into the cross-stage partially parallel dilated convolutional network, and the processing process of the cross-stage partially parallel dilated convolutional network for the feature map includes S21 - S24.

[0133] S21: Compress the number of channels of the feature map to 1 / 2 of the original through 1×1 convolution to obtain the compressed feature map : .

[0134] S22: Extract multi-scale context information through three independent 3×3 convolutions with different dilation rates (1, 2, 3) to obtain the multi-scale feature map : .

[0135] Among them, is the operation of extracting multi-scale information; is the 3×3 convolution with a dilation rate of 1; is the channel concatenation; is the 3×3 convolution with a dilation rate of 2; is the 3×3 convolution with a dilation rate of 3.

[0136] S23: Directly keep the number of channels of the feature map through 1×1 convolution to obtain the maintained feature map : .

[0137] S24: Combine the multi-scale feature map and the maintained feature map , and then perform another 1×1 convolution to generate a refined feature map, that is, the feature map output by the cross-stage partially parallel dilated convolutional network: .

[0138] Among them, the feature map output by the cross-stage partially parallel dilated convolutional network is input into the channel gated attention downsampling network, and the processing process of the channel gated attention downsampling network for the feature map includes S31 - S38.

[0139] S31: Feature map After global average pooling is performed to capture its overall context representation, the feature map after global average pooling is obtained : .

[0140] Among them, is the global average pooling process

[0141] S32: By using 1×1 convolution and the Hardsigmid activation function, importance weights are assigned to each channel to generate a channel attention map : .

[0142] S33: The feature map is input into the sixteenth convolutional block for 3×3 convolution to obtain the feature map output by the sixteenth convolutional block : .

[0143] S34: The feature map is input into the second max-pooling layer for max-pooling to obtain the feature map output by the second max-pooling layer : .

[0144] Among them, is the max-pooling operation with a pooling kernel size of 2×2

[0145] S35: Using the seventeenth convolutional block, the feature map is processed to obtain the feature map output by the seventeenth convolutional block

[0146] S36: Using the sixth adder, the feature map output by the sixteenth convolutional block and the feature map output by the seventeenth convolutional block are added together to obtain the feature map output by the sixth adder

[0147] S37: Using a multiplier, the channel attention map and the feature map output by the sixth adder are multiplied to obtain the feature map output by the multiplier

[0148] S38: Using the eighteenth convolutional block, the feature map output by the multiplier is processed to obtain the final feature map after feature weighting and fusion, that is, the feature map output by the channel gated attention downsampling network .

[0149] The channel-gated attention downsampling network can create richer feature representations. It not only enhances the diversity of the extracted features, but also strengthens the expression of important features, while minimizing irrelevant information.

[0150] Step 23: Input the first encoded output feature map, the second encoded output feature map, and the third encoded output feature map into the Transform decoder in the railway track fastener defect detection model to obtain the target defect detection result.

[0151] Specifically, the Transform decoder uses the self-attention and cross-attention mechanisms, combines the first encoded output feature map, the second encoded output feature map, and the third encoded output feature map, and generates the target defect detection result through the detection head.

[0152] In an exemplary embodiment, a railway track fastener defect detection system is provided to implement the railway track fastener defect detection method. The railway track fastener defect detection system includes the following modules.

[0153] An image acquisition module for acquiring a target image; the target image is an image containing the railway track fastener to be detected.

[0154] A defect detection module for inputting the target image into the railway track fastener defect detection model to obtain the target defect detection result; the railway track fastener defect detection model is obtained by training an initial network; the target defect detection result is a target image with a predicted bounding box and a predicted category of the defect contained in the railway track fastener to be detected marked.

[0155] In an exemplary embodiment, a computer device is provided, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the computer program to implement the railway track fastener defect detection method.

[0156] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the railway track fastener defect detection method is implemented.

[0157] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the railway track fastener defect detection method is implemented.

[0158] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for detecting defects in railway track fasteners.

[0159] Those skilled in the art can understand that Figure 7 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0160] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0161] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable connectors, data processing connectors based on quantum computing, etc., without limitation.

[0162] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0163] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0164] In this article, specific examples are used to illustrate the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for detecting defects in railway track fasteners, characterized in that, The railway track fastener defect detection method includes: Obtaining a target image; the target image is an image containing the railway track fastener to be detected; Inputting the target image into a railway track fastener defect detection model to obtain a target defect detection result; the railway track fastener defect detection model is obtained by training an initial network, and the initial network is constructed based on a wavelet transform convolution basic block, a wavelet feature upgrade network, a cross-stage partial parallel dilated convolution network, and a channel gating attention downsampling network; the target defect detection result is a target image with a predicted bounding box and a predicted category of the defect contained in the railway track fastener to be detected marked.

2. The railway track fastener defect detection method according to claim 1, wherein The determination process of the railway track fastener defect detection model includes: Obtaining multiple sample images and corresponding true defect detection results; the sample images are images containing sample railway track fasteners, and the true defect detection results are sample images with true bounding boxes and true categories of the defects contained in the sample railway track fasteners marked. Constructing the initial network; Training the initial network with each sample image as the input and the corresponding true defect detection result as the output to obtain the railway track fastener defect detection model.

3. The railway track fastener defect detection method according to claim 2, wherein The initial network includes: a backbone network, a hybrid encoder, and a Transform decoder; The backbone network includes: a first convolutional block, a second convolutional block, a third convolutional block, a first max pooling layer, a first wavelet transform convolution basic block, a second wavelet transform convolution basic block, a third wavelet transform convolution basic block, a fourth wavelet transform convolution basic block, and a fourth convolutional block connected in sequence; The hybrid encoder includes: a fifth convolutional block, a sixth convolutional block, an attention-based scale-in feature interaction module, and a seventh convolutional block, a first wavelet feature upgrade module, a first cross-stage partial parallel dilated convolution module, an eighth convolutional block, a second wavelet feature upgrade module, a second cross-stage partial parallel dilated convolution module, a first channel gating attention downsampling module, a first adder, a third cross-stage partial parallel dilated convolution module, a second channel gating attention downsampling module, a second adder, and a fourth cross-stage partial parallel dilated convolution module connected in sequence; the second wavelet transform convolution basic block is connected to the fifth convolutional block, the third wavelet transform convolution basic block is connected to the sixth convolutional block, the attention-based scale-in feature interaction module is connected to the seventh convolutional block, the seventh convolutional block is connected to the second adder, and the eighth convolutional block is connected to the first adder; The second cross-stage partial parallel dilated convolution module, the third cross-stage partial parallel dilated convolution module, and the fourth cross-stage partial parallel dilated convolution module are all connected to the Transform decoder.

4. The railway track fastener defect detection method according to claim 3, characterized in that The first wavelet feature upgrade module and the second wavelet feature upgrade module are both wavelet feature upgrade networks, and the wavelet feature upgrade network includes: a wavelet transform unit, a residual block, a third adder, and a wavelet inverse transform unit; The wavelet transform unit is respectively connected to the residual block and the third adder; the third adder and the residual block are respectively connected to the inverse wavelet transform unit.

5. The railway track fastener defect detection method according to claim 4, wherein The first cross-stage partially parallel dilated convolution module, the second cross-stage partially parallel dilated convolution module, the third cross-stage partially parallel dilated convolution module, and the fourth cross-stage partially parallel dilated convolution module are all cross-stage partially parallel dilated convolution networks. The cross-stage partially parallel dilated convolution network includes: a ninth convolution block, a tenth convolution block, a parallel dilated convolution unit, a fourth adder, and an eleventh convolution block; the parallel dilated convolution unit includes: a twelfth convolution block, a thirteenth convolution block, a fourteenth convolution block, a fifth adder, and a fifteenth convolution block; The ninth convolution block is connected to the fourth adder, the tenth convolution block is respectively connected to the twelfth convolution block, the thirteenth convolution block, and the fourteenth convolution block, the twelfth convolution block, the thirteenth convolution block, and the fourteenth convolution block are all connected to the fifth adder, the fifth adder is connected to the fifteenth convolution block, the fifteenth convolution block is connected to the fourth adder, and the fourth adder is connected to the eleventh convolution block.

6. The railway track fastener defect detection method according to claim 5, characterized in that, The first channel gated attention downsampling module and the second channel gated attention downsampling module are both channel gated attention downsampling networks. The channel gated attention downsampling network includes: a global average pooling layer, a sixteenth convolution block, a second max pooling layer, a channel gated unit, a sixth adder, a seventeenth convolution block, a multiplier, and an eighteenth convolution block; the channel gated unit includes: a nineteenth convolution block and a Hardsigmoid activation function; The global average pooling layer is connected to the nineteenth convolution block, the nineteenth convolution block is connected to the Hardsigmoid activation function, the second max pooling layer is connected to the seventeenth convolution block, the sixteenth convolution block and the seventeenth convolution block are both connected to the sixth adder, the Hardsigmoid activation function and the sixth adder are both connected to the multiplier, and the multiplier is connected to the eighteenth convolution block.

7. The railway track fastener defect detection method according to claim 6, characterized in that Input the target image into the railway track fastener defect detection model to obtain the target defect detection result, including: Input the target image into the backbone network of the railway track fastener defect detection model to obtain the first backbone output feature map, the second backbone output feature map, and the third backbone output feature map of the railway track fastener to be detected; Input the first backbone output feature map, the second backbone output feature map, and the third backbone output feature map into the hybrid encoder of the railway track fastener defect detection model to obtain the first encoded output feature map, the second encoded output feature map, and the third encoded output feature map of the railway track fastener to be detected; Input the first encoded output feature map, the second encoded output feature map, and the third encoded output feature map into the Transform decoder of the railway track fastener defect detection model to obtain the target defect detection result.

8. A railway track fastener defect detection system for implementing the railway track fastener defect detection method according to any one of claims 1-7, characterized in that, The railway track fastener defect detection system includes: An image acquisition module for acquiring a target image; the target image is an image containing the railway track fastener to be detected; A defect detection module for inputting the target image into a railway track fastener defect detection model to obtain a target defect detection result; the railway track fastener defect detection model is obtained by training an initial network; the target defect detection result is a target image marked with a predicted bounding box and a predicted category of the defect contained in the railway track fastener to be detected.

9. A computer device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the railway track fastener defect detection method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the railway track fastener defect detection method according to any one of claims 1-7.

Images