A method for detecting the breakage of anti-loosening wire on railway carriage floor

Abstract

A method for detecting a broken wire fault in the bottom plate of a railway EMU, the present invention relates to a method for detecting a broken wire fault in the bottom plate of a railway EMU. The purpose of the present invention is to solve the problem that the existing convolution-based deep learning detection network has a poor recognition effect for fault detection and is prone to missed detection. The process is: collecting images and establishing a data set; constructing a bottom plate broken wire fault detection network, the network including: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head; training the bottom plate broken wire fault detection network to obtain a trained bottom plate broken wire fault detection network: detecting a broken wire fault in the bottom plate of a railway EMU based on the trained bottom plate broken wire fault detection network. The present invention is used in the field of railway EMU detection.

Description

Technical Field

[0001] The invention relates to a method for detecting a breakage fault of an anti-loosening iron wire on a railway motor vehicle bottom plate. Background Art

[0002] Traditional fault detection methods based on manual diagram review are time-consuming and labor-intensive, with high labor costs. Furthermore, due to fatigue and carelessness among inspectors, missed detections and false positives can occur. Railway train fault detection methods using deep learning can effectively reduce detection costs and improve efficiency. However, due to the large number of bottom plate anti-loosening wires, their thin shape, and the lack of distinct features after breaking, existing convolution-based deep learning detection networks offer poor fault detection performance, making missed detections a common occurrence. Summary of the Invention

[0003] The purpose of the present invention is to solve the problem that the existing fault detection based on convolution deep learning detection network is poor in recognition effect and prone to missed detection due to the large number of bottom plate anti-loosening wires distributed, thin shape, and unclear characteristics after breaking. A method for detecting the breakage of anti-loosening wires on the bottom plate of railway EMUs is proposed.

[0004] A method for detecting a broken wire fault in a railway train floor anti-loosening wire is as follows:

[0005] Step 1: Collect images and build a data set;

[0006] Step 2: Construct a bottom plate anti-loose wire breakage fault detection network, train the bottom plate anti-loose wire breakage fault detection network, and obtain a trained bottom plate anti-loose wire breakage fault detection network; the specific process is:

[0007] Step 21: The bottom plate anti-loose wire broken fault detection network includes: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head;

[0008] The specific process is:

[0009] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0010] Each stage includes: Patch Embedding, position encoding, branch block, concat, normalization Norm layer, FFN multi-layer perceptron;

[0011] The branch block includes branch a and branch b;

[0012] Branch a includes:

[0013] PVT Transformer Encoder encoding, FFN multi-layer perceptron, one-dimensional convolution Conv1D, first Relu activation function and SSM module;

[0014] Branch b includes: two-dimensional convolution, second Relu activation function, two-dimensional convolution, third Relu activation function, normalized Norm layer, FFN multi-layer perceptron;

[0015] The multi-scale feature fusion Transformer encoder includes: 1×1 two-dimensional convolution, concat splicing, and improved Transformer encoder;

[0016] The improved Transformer decoder structure includes: block-based multi-head attention mechanism, normalized Norm layer, and multi-layer perceptron (FFN);

[0017] The detection head is a multi-layer perceptron FFN;

[0018] Step 22: training a bottom plate anti-loosening wire breakage fault detection network to obtain a trained bottom plate anti-loosening wire breakage fault detection network;

[0019] Step 3: Detect the railway train floor anti-loosening wire breakage fault based on the trained floor anti-loosening wire breakage fault detection network.

[0020] Preferably, in step 1, images are collected and a data set is established; the specific process is:

[0021] Step 1: Set up high-definition imaging equipment around the railway to obtain images of passing trains and collect images of the bottom plate taken by the bottom camera;

[0022] Step 12: amplify the collected baseplate image to obtain an amplified baseplate image;

[0023] Steps 1 and 3

[0024] Mark the image of the amplified base plate image where the base plate anti-loosening wire is normal as a base plate anti-loosening wire normal class, and generate a marking file;

[0025] Mark the image of the base plate with the broken anti-loosening wire in the amplified base plate image as a base plate anti-loosening wire broken class, and generate a marking file;

[0026] Step 14: Photoshop the image of the normal bottom plate anti-loosening wire into an image of the broken bottom plate anti-loosening wire, mark it as the broken bottom plate anti-loosening wire category, and generate a marking file;

[0027] The tag file records information such as the image name, size, path, location of the anti-loosening wire, and type of the anti-loosening wire; the size is the size of the image, and the path is the path of the image.

[0028] Preferably, in step 22, the bottom plate anti-loosening wire breakage fault detection network is trained to obtain a trained bottom plate anti-loosening wire breakage fault detection network; the specific process is:

[0029] Step 221: Input the image in the data set established in step 1 into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features;

[0030] Step 222: Input the multi-scale features output by the improved PVT feature extraction backbone network into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features;

[0031] Step 223: The multi-scale feature fusion Transformer encoder output features and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs features;

[0032] Step 224: Input the output features of the improved Transformer decoder into the detection head, and the detection head outputs the output target category and target position;

[0033] Step 225: Use the Adam optimization method, set the initial learning rate to 0.001, and repeat steps 221 to 225 until convergence to obtain a trained bottom plate anti-loose wire breakage fault detection network.

[0034] Preferably, in step 221, the image in the data set established in step 1 is input into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features;

[0035] The specific process is:

[0036] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0037] The input image F0 of size H0×W0×C0 is input to stage 1, stage 1 outputs feature map F1 which is input to stage 2, stage 2 outputs feature map F2 which is input to stage 3, stage 3 outputs feature map F3 which is input to stage 4, and stage 4 outputs feature map F4;

[0038] The specific processing process of each stage i in the stages 1, 2, 3 and 4 is as follows:

[0039] At stage i, set the size of stage i-1 to H i-1 ×W i-1 ×C i-1 The feature map F i-1Divided into P i ×P i Patch Embedding is performed on each block, and the sequence processed by Patch Embedding is deformed and reshaped to obtain a sequence of size The characteristic sequence of Add a position code of the same size to each pixel in the feature sequence to obtain the feature sequence A; i≥1;

[0040] The feature sequence A is input into branch a and branch b respectively. Branch a and branch b respectively output the feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel. After splicing, they pass through the normalization Norm layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F. i ; Feature map F i Size

[0041] Among them, H i-1 is the feature map F i-1 High, W i-1 is the feature map F i-1 Width, C i-1 is the feature map F i-1 The number of channels, P i ×P i To transform the feature map F i-1 The size of the divided blocks;

[0042] Preferably, the feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output the feature sequence encoding results, and the feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel. After splicing, they are sequentially passed through the normalization Norm layer and the FFN multi-layer perceptron, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is:

[0043] The specific processing process of branch a is:

[0044] The feature sequence A is encoded by PVT Transformer Encoder to extract global features. The size of the encoded result is The size is The encoding result passes through a FFN multi-layer perceptron, which outputs a feature sequence Output feature sequence of FFN multi-layer perceptron After a one-dimensional convolution Conv1D, the first Relu activation function and the SSM module, the SSM module outputs a feature sequence as the feature sequence encoding result output by branch a, with a size of

[0045] The specific processing process of branch b is:

[0046] The feature sequence A is reshaped to obtain a size of H i-1 ×W i-1 ×C i-1 Feature map of

[0047] Set the size to H i-1 ×W i-1 ×C i-1 The feature map of is sequentially subjected to two-dimensional convolution, the second Relu activation function, two-dimensional convolution, the third Relu activation function, and the third Relu activation function outputs the feature map; the output feature map is deformed and reshaped to obtain a size of The characteristic sequence of

[0048] For size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn. The FFN multilayer perceptron outputs the feature sequence as the feature sequence encoding result output by branch b, with a size of

[0049] The feature sequence encoding results output by branch a and branch b are concat-joined by channel, and then pass through the normalization layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is:

[0050] The feature sequence encoding results obtained by branch a and branch b are spliced ​​by channel to obtain a size of The feature sequence of size The feature sequence is input into the normalized Norm layer and the FFN multilayer perceptron in turn, and the FFN multilayer perceptron outputs the coded feature sequence; the FFN multilayer perceptron output coded feature sequence is deformed and reshaped to obtain The feature map F i .

[0051] Preferably, in step 222, the multi-scale features output by the improved PVT feature extraction backbone network are input into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; the specific process is:

[0052] The feature map F2 undergoes a 1×1 two-dimensional convolution to generate the feature map P2;

[0053] The feature map F3 undergoes a 1×1 two-dimensional convolution to generate the feature map P3;

[0054] The feature map F4 undergoes a 1×1 two-dimensional convolution to generate the feature map P4;

[0055] Fuse the feature map P3 and the feature map P4 to obtain the feature map P5;

[0056] Fuse the feature map P2 and the feature map P5 to obtain the feature map P6;

[0057] Fuse the feature maps P3, P5, and P6 to obtain the feature map P7;

[0058] Fuse the feature map P4 and the feature map P7 to obtain the feature map P8;

[0059] Concatenate feature maps P6, P7, and P8 by channel to obtain feature map P9;

[0060] The feature map P9 is input into a 1×1 two-dimensional convolution, and the 1×1 two-dimensional convolution outputs a multi-scale fusion feature sequence;

[0061] The multi-scale fusion feature sequence is input into the improved Transformer encoder, and the improved Transformer encoder outputs the encoded features, which are used as the multi-scale feature fusion Transformer encoder output features.

[0062] Preferably, the feature map P3 and the feature map P4 are fused to obtain the feature map P5;

[0063] The specific process is:

[0064] First, upsample P4 to the size of P3;

[0065] P4, which is upsampled to the same size as P3, is processed by Patch Embedding and converted into a two-dimensional feature sequence corresponding to P4;

[0066] Perform Patch Embedding processing on P3 and convert it into a two-dimensional feature sequence corresponding to P3;

[0067] Add the two-dimensional feature sequence corresponding to P4 to a position code of the same size pixel by pixel to obtain the feature sequence B after adding the position code;

[0068] Add the two-dimensional feature sequence corresponding to P3 to a position code of the same size pixel by pixel to obtain the feature sequence C after adding the position code;

[0069] The feature sequence B and the feature sequence C are spliced ​​together according to the channel dimension. The spliced ​​feature sequence passes through the improved Transformer encoder, and the improved Transformer encoder outputs the encoded feature D;

[0070] The encoded feature sequence D output by the improved Transformer encoder is sequentially input into the normalized Norm layer and the FFN multi-layer perceptron, and the FFN multi-layer perceptron outputs the feature sequence E;

[0071] The FFN multi-layer perceptron output feature sequence E is deformed and reshaped to obtain a feature map. The width and height of the feature map are the same as those of P3. A 1×1 two-dimensional convolution is performed on the feature map to obtain the feature map P5.

[0072] Preferably, the specific processing process of the improved Transformer encoder is:

[0073] 1) Combine multi-scale fusion features with weights W V Multiply to get vector V;

[0074] 2) Add a position code of the same size pixel by pixel in the multi-scale fusion feature;

[0075] The multi-scale fusion features after adding position encoding are respectively combined with the weight W Q 、W K Multiply to get vectors Q and K;

[0076] 3) Perform SimAM attention mechanism operations on the obtained vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0077] 4) Input the results of the three SimAM attention mechanisms into the block-wise multi-head attention mechanism respectively, add the output results of the block-wise multi-head attention mechanism and the multi-scale fusion features pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0078] 5) Input the normalized Norm layer output features into the multi-layer perceptron FFN, add the multi-layer perceptron FFN output feature sequence to the normalized Norm layer output features in 4) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0079] 6) Repeat 1) to 5) N times to obtain the improved Transformer encoder output features;

[0080] The specific processing process of the block-based multi-head attention mechanism is as follows:

[0081] (1) Change the first row of each SimAM attention mechanism result to the last row and the first column to the last column in the three SimAM attention mechanism results to obtain the converted three SimAM attention mechanism results;

[0082] (2) The converted three SimAM attention mechanism results are added pixel by pixel with the three SimAM attention mechanism results in (1), and then sent to the multi-layer perceptron FFN for encoding. The multi-layer perceptron FFN outputs the encoded features;

[0083] (3) The encoded features are evenly divided into four blocks, each of which is input into a multi-head attention mechanism. The output features of the four multi-head attention mechanisms are spliced ​​by channel to obtain a feature map as the output features of the block multi-head attention mechanism.

[0084] Preferably, in steps 223, the multi-scale feature fusion Transformer encoder output features and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs features; the specific process is:

[0085] 1) Initialize the target query vector, and the target query vector is respectively Q 、W K 、W V Multiply to get vectors Q, K, V;

[0086] Perform SimAM attention mechanism operations on vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0087] The results of the three SimAM attention machines are input into the block-wise multi-head attention mechanism respectively, and the block-wise multi-head attention mechanism outputs the results;

[0088] 2) Add the output of the multi-head attention mechanism to the target query vector in 1) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0089] 3) Add the normalized Norm layer output features from 2) to the target query vector to obtain a new vector Q;

[0090] Add the improved Transformer encoder output features to a positional encoding of the same size pixel by pixel to obtain a new vector K;

[0091] The improved Transformer encoder output features are used as the new vector V;

[0092] 4) Input the new vector Q, new vector K, and new vector V obtained in 3) into the block-based multi-head attention mechanism, and the block-based multi-head attention mechanism outputs the result;

[0093] 5) Add the output of the multi-head attention mechanism in 4) and the output of the normalized Norm layer in 2) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0094] 6) Input the normalized Norm layer output features of 5) into the FFN multilayer perceptron, add the FFN multilayer perceptron output feature sequence and the normalized Norm layer output features in 5) pixel by pixel, input the added feature sequence into the normalized Norm layer, and normalize the Norm layer output features;

[0095] 7) Repeat 1) to 6) M times to obtain the improved Transformer decoder output features.

[0096] Preferably, in step three, the railway vehicle floor anti-loosening wire breakage fault is detected based on the trained floor anti-loosening wire breakage fault detection network; the specific process is:

[0097] Capture an image of a railway train passing through a high-definition imaging device and feed the bottom camera image into a trained detection network. If the detection network detects a broken bottom plate anti-loosening wire, the image is faulty, a fault message is uploaded, and the next image is tested. If the detection network does not detect a broken bottom plate anti-loosening wire, the image is faulty and no action is taken, continuing with the next image.

[0098] The beneficial effects of the present invention are:

[0099] The present invention adopts a Transformer-based PVT network to replace the original convolution-based backbone network in DETR, and improves the PVT network by introducing convolution and SSM branches to improve the global and local feature extraction capabilities of the network.

[0100] The present invention introduces a Transformer-based multi-scale feature pyramid into DETR and performs multi-scale feature fusion encoding, so that the network can comprehensively utilize features of different scales and improve the detection accuracy of targets of different scales in the network.

[0101] The Transformer encoder and Transformer decoder of the present invention adopt a block-based multi-head attention mechanism to reduce the model calculation amount, and introduce the SimAM attention mechanism to improve the network detection accuracy.

[0102] The DETR detection network based on Transformer in the present invention can better extract global features of images compared with the detection network based on convolution. Therefore, the present invention adopts the improved DETR detection network for fault detection, which can effectively reduce missed detections and improve fault detection effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Figure 1 This is a fault identification flow chart of the present invention;

[0104] Figure 2 Figure 1 shows the improved DETR bottom plate anti-loose wire missing fault detection network structure. a is the overall network result, b is the improved PVT feature extraction backbone network, c is the multi-scale feature fusion Transformer encoder, and d is the improved multi-scale feature fusion Transformer encoder and Transformer decoder structure.

[0105] Figure 3 Diagram of the block-based multi-head attention mechanism. DETAILED DESCRIPTION

[0106] Specific embodiment 1: This embodiment is a method for detecting a broken wire fault in a railway train floor anti-loosening wire. The specific process is as follows:

[0107] Step 1: Collect images and build a data set;

[0108] Step 2: Construct a bottom plate anti-loose wire breakage fault detection network, train the bottom plate anti-loose wire breakage fault detection network, and obtain a trained bottom plate anti-loose wire breakage fault detection network; the specific process is:

[0109] Step 21: The bottom plate anti-loose wire broken fault detection network includes: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head;

[0110] The specific process is:

[0111] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0112] Each stage includes: Patch Embedding, position encoding, branch block, concat, normalization Norm layer, FFN multi-layer perceptron;

[0113] The branch block includes branch a and branch b;

[0114] Branch a includes:

[0115] PVT Transformer Encoder encoding, FFN multi-layer perceptron, one-dimensional convolution Conv1D, first Relu activation function and SSM module;

[0116] Branch b includes: two-dimensional convolution, second Relu activation function, two-dimensional convolution, third Relu activation function, normalized Norm layer, FFN multi-layer perceptron;

[0117] The multi-scale feature fusion Transformer encoder includes: 1×1 two-dimensional convolution, concat splicing, and improved Transformer encoder;

[0118] The improved Transformer decoder structure includes: block-based multi-head attention mechanism, normalized Norm layer, and multi-layer perceptron (FFN);

[0119] The detection head is a multi-layer perceptron FFN;

[0120] Step 22: training a bottom plate anti-loosening wire breakage fault detection network to obtain a trained bottom plate anti-loosening wire breakage fault detection network;

[0121] Step 3: Detect the railway train floor anti-loosening wire breakage fault based on the trained floor anti-loosening wire breakage fault detection network.

[0122] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that in step 1, images are collected and a data set is established; the specific process is as follows:

[0123] Step 1: Set up high-definition imaging equipment around the railway to obtain images of passing trains and collect images of the bottom plate taken by the bottom camera;

[0124] Step 1 and 2: perform data amplification on the collected baseboard image, increase the anti-loosening wire morphology in the data set as much as possible, and obtain the amplified baseboard image;

[0125] Steps 1 and 3

[0126] Mark the image of the amplified base plate image where the base plate anti-loosening wire is normal as a base plate anti-loosening wire normal class, and generate a marking file;

[0127] Mark the image of the base plate with the broken anti-loosening wire in the amplified base plate image as a base plate anti-loosening wire broken class, and generate a marking file;

[0128] Mark the amplified baseplate images that are similar to the broken baseplate anti-loosening wires (some anti-loosening wires are relatively thin due to shooting or the reasons themselves, and if you don't judge carefully, they look like they are broken in the middle; some foreign objects such as weeds also look like broken anti-loosening wires) as interference images, and generate a label file;

[0129] Step 14: Photoshop the image of the normal bottom plate anti-loosening wire into an image of the broken bottom plate anti-loosening wire, mark it as the broken bottom plate anti-loosening wire category, and generate a marking file;

[0130] The tag file records the image's name, size, path, location of the anti-loosening wire, type of the anti-loosening wire, and other information; the size is the image size, and the path is the image path;

[0131] High-definition imaging equipment was installed around the railway to capture images of passing trains and collect images of the floor plate captured by cameras below for subsequent dataset construction. Images of normal floor plate anti-loosening wires and images of broken floor plate anti-loosening wires were collected as the detection dataset images. Because broken floor plate anti-loosening wire failures occur infrequently and images of these failures are difficult to collect, a broken floor plate anti-loosening wire failure was simulated in Photoshop on top of the normal floor plate anti-loosening wire images to supplement the dataset. The dataset was labeled and a label file was generated, completing the dataset collection. Labeling was performed using labelImg labeling software, which generated label files corresponding to the images. Normal floor plate anti-loosening wires were labeled as normal, broken floor plate anti-loosening wires were labeled as broken, and normal floor plate components with similar broken floor plate anti-loosening wires were labeled as interference. The label file contained information such as the image name, size, path, target location, and target category.

[0132] The baseplate anti-loosening wires are located in various locations on the EMU baseplate, with varying bends, angles, lengths, and thicknesses. The collected dataset cannot cover all wire configurations, necessitating data augmentation to maximize the number of anti-loosening wire configurations in the dataset. Data augmentation operations include affine transformation, rotation, cropping, and contrast transformation. Data augmentation can effectively reduce the probability of overfitting in the fault detection network and improve its generalization performance.

[0133] Other steps and parameters are the same as those in the first embodiment.

[0134] Specific embodiment three: This embodiment differs from specific embodiment one or two in that, in step 22, a bottom plate anti-loosening wire breakage fault detection network is trained to obtain a trained bottom plate anti-loosening wire breakage fault detection network;

[0135] The specific process is:

[0136] Step 221: Input the data set established in step 1 into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features;

[0137] Step 222: Input the multi-scale features output by the improved PVT feature extraction backbone network into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features;

[0138] Step 223: The multi-scale feature fusion Transformer encoder output features and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs features;

[0139] Step 224: Input the output features of the improved Transformer decoder into the detection head, and the detection head outputs the output target category and target position;

[0140] Step 225: Use the Adam optimization method, set the initial learning rate to 0.001, and repeat steps 221 to 225 until convergence to obtain a trained bottom plate anti-loose wire breakage fault detection network.

[0141] The network was trained using Adam optimization, with an initial learning rate of 0.001. The dataset was divided into training, validation, and test sets in a ratio of 7:2:1. The detection network was trained on images from the training set and tested on the validation set. Images from the validation set that were incorrectly identified were then added to the training set and the detection network was retrained. Finally, the network's performance was tested on the test set.

[0142] Other steps and parameters are the same as those in the first or second embodiment.

[0143] Specific embodiment 4: This embodiment differs from one of specific embodiments 1 to 3 in that, in step 221, the image in the data set established in step 1 is input into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features; the specific process is:

[0144] The PVT feature extraction backbone network adopts the Transformer structure, which enables the network to more fully extract the global features of the image. At the same time, the network implements the downsampling of the feature map through Patch Embedding, and obtains feature maps of different scales of 4 times, 8 times, 16 times, and 32 times downsampling, which enables the network to extract features of different scales in the image. The subsequent feature fusion can improve the network's detection ability for targets of different scales. However, the Transformer structure has high computational complexity and is difficult to train. This patent adds convolution branches and SSM branches to PVT, which reduces the computational complexity of the model, reduces the difficulty of training the model, and improves the network's feature extraction capabilities. The improved PVT feature extraction backbone network structure is as follows: Figure 2 As shown in b.

[0145] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0146] The input image F0 of size H0×W0×C0 is input to stage 1, stage 1 outputs feature map F1 which is input to stage 2, stage 2 outputs feature map F2 which is input to stage 3, stage 3 outputs feature map F3 which is input to stage 4, and stage 4 outputs feature map F4;

[0147] C0 is the number of image channels, which is 3 for a three-channel color image;

[0148] The specific processing process of each stage i in the stages 1, 2, 3 and 4 is as follows:

[0149] At stage i, set the size of stage i-1 to H i-1 ×W i-1 ×C i-1 The feature map F i-1 Divided into P i ×P i Patch Embedding is performed on each block, and the sequence processed by Patch Embedding is deformed and reshaped to obtain a sequence of size The characteristic sequence of Add a position code of the same size to each pixel in the feature sequence to obtain the feature sequence A; i≥1;

[0150] The feature sequence A is input into branch a and branch b respectively. Branch a and branch b respectively output the feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel. After splicing, they pass through the normalization Norm layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F. i ; Feature map Fi Size

[0151] Among them, H i-1 is the feature map F i-1 High, W i-1 is the feature map F i-1 Width, C i-1 is the feature map F i-1 The number of channels, P i ×P i To transform the feature map F i-1 The size of the divided block (width is P i , high is P i ).

[0152] The input image F0 of size H×W×3 is input to stage 1, and the output size of stage 1 is Feature map F1;

[0153] The feature map F1 is input to stage 2, and the output size of stage 2 is Feature map F2;

[0154] The feature map F2 is input to stage 3, and the output size of stage 3 is Feature map F3;

[0155] The feature map F3 is input to stage 4, and the output size of stage 4 is Feature map F4;

[0156] Among them, H is the height of the image, W is the width of the image, C1 is the number of channels of the feature map F1, C2 is the number of channels of the feature map F2, C3 is the number of channels of the feature map F3, and C4 is the number of channels of the feature map F4.

[0157] The input image F0 of size H×W×3 is processed through stages 1, 2, 3, and 4 to generate feature maps of different scales that are downsampled by 4 times, 8 times, 16 times, and 32 times, respectively.

[0158] The other steps and parameters are the same as those in the first to third embodiments.

[0159] Specific embodiment 5: The difference between this embodiment and specific embodiments 1 to 4 is that the feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​according to the channel. After splicing, they are sequentially passed through the normalization Norm layer and the FFN multi-layer perceptron. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F. i ; The specific process is:

[0160] The specific processing process of branch a is:

[0161] Branch a uses a PVTTransformer Encoder module and an SSM module;

[0162] The feature sequence A is encoded by PVTTransformer Encoder to extract global features. The size of the encoded result is The size is The encoding result is passed through an FFN multi-layer perceptron to reduce the feature sequence dimension by half, and the FFN multi-layer perceptron outputs the feature sequence Reduce the amount of calculation; output feature sequence of FFN multi-layer perceptron After a one-dimensional convolution Conv1D, the first Relu activation function and the SSM module, the SSM module outputs a feature sequence as the feature sequence encoding result output by branch a, with a size of

[0163] The specific processing process of branch b is:

[0164] The feature sequence A is reshaped to obtain a size of H i-1 ×W i-1 ×C i-1 Feature map of

[0165] Set the size to H i-1 ×W i-1 ×C i-1 The feature map of is sequentially subjected to two-dimensional convolution, the second Relu activation function, two-dimensional convolution, the third Relu activation function, and the third Relu activation function outputs the feature map; the output feature map is deformed and reshaped to obtain a size of The characteristic sequence of

[0166] For size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn. The FFN multilayer perceptron outputs the feature sequence as the feature sequence encoding result output by branch b, with a size of

[0167] The feature sequence encoding results output by branch a and branch b are concat-joined by channel, and then pass through the normalization layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i , get the feature encoding result of stage i; the specific process is:

[0168] The feature sequence encoding results obtained by branch a and branch b are concat-joined by channel to obtain a size of

[0169] The feature sequence of size The feature sequence is input into the normalized Norm layer and the FFN multilayer perceptron in turn, and the FFN multilayer perceptron outputs the coded feature sequence; the FFN multilayer perceptron output coded feature sequence is deformed and reshaped to obtain The feature map F i , get the feature encoding result of stage i;

[0170] The SSM module in the network is a sequence feature encoding module proposed by the Mamba network that can be flexibly embedded in the network. Compared to the Transformer encoder, the SSM, while also extracting global sequence features, has lower computational complexity, better generalization, and can reduce the probability of network overfitting. Compared to convolution, the Transformer encoder and SSM are more complex and difficult to train. Adding convolutional branches to the network can effectively reduce the model's computational complexity, stabilize the network's recognition performance, and reduce the network's computational difficulty. While the Transformer encoder and SSM can extract global image features, their ability to extract local features is poor. Adding convolutional branches can better extract more refined local features that are rotationally invariant.

[0171] The broken shape of the bottom plate anti-loosening wire is not obvious and has various forms. The detection effect of the convolution-based fault detection network is not ideal. The Transformer and DETR detection network can fully extract the global features of the image, avoid the local inductive bias problem in the convolutional network, and can better generalize to unseen images, better distinguish between broken and normal wires, and more accurately identify the location of broken wires. This patent improves the DETR detection network, enhancing the network's ability to recognize small-target anti-loosening wires and enabling the network to more accurately locate the fault location. First, the Transformer-based PVT network is used to replace the original convolution-based feature extraction backbone network in the DETR network. The PVT network is also improved by introducing convolution and SSM branches to improve the network's global and local feature extraction capabilities. Secondly, a Transformer-based multi-scale feature pyramid is introduced into the DETR network to perform multi-scale feature fusion. Together with the network encoder, it forms a multi-scale feature fusion encoder to improve the network's ability to recognize multi-scale targets. The multi-head attention mechanism in the DETR network encoder and decoder is improved, and the block multi-head attention mechanism is used to reduce the network calculation amount without affecting the network accuracy. The SimAM attention mechanism is introduced to make the detection result frame generated by the network more accurate. The overall structure of this patent network is as follows Figure 2As shown in Figure a, the network consists of an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder and a detection head.

[0172] The other steps and parameters are the same as those in the first to fourth embodiments.

[0173] Specific embodiment six: This embodiment differs from specific embodiments one to five in that, in step 222, the multi-scale features output by the improved PVT feature extraction backbone network are input into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; the specific process is as follows:

[0174] The feature map F2 undergoes a 1×1 two-dimensional convolution to generate the feature map P2;

[0175] The feature map F3 undergoes a 1×1 two-dimensional convolution to generate the feature map P3;

[0176] The feature map F4 undergoes a 1×1 two-dimensional convolution to generate the feature map P4;

[0177] Fuse the feature map P3 and the feature map P4 to obtain the feature map P5;

[0178] Fuse the feature map P2 and the feature map P5 to obtain the feature map P6;

[0179] Fuse the feature maps P3, P5, and P6 to obtain the feature map P7;

[0180] Fuse the feature map P4 and the feature map P7 to obtain the feature map P8;

[0181] Concatenate feature maps P6, P7, and P8 by channel to obtain feature map P9.

[0182] The feature map P9 is input into a 1×1 two-dimensional convolution, and the 1×1 two-dimensional convolution outputs a multi-scale fusion feature sequence;

[0183] The multi-scale fusion feature sequence is input into the improved Transformer encoder, and the improved Transformer encoder outputs the encoded features, which are used as the multi-scale feature fusion Transformer encoder output features.

[0184] The other steps and parameters are the same as those in the first to fifth embodiments.

[0185] Specific embodiment seven: This embodiment differs from any one of specific embodiments one to six in that the feature map P3 and the feature map P4 are fused to obtain the feature map P5; the specific process is as follows:

[0186] First, upsample P4 to the size of P3;

[0187] P4, which is upsampled to the same size as P3, is processed by Patch Embedding and converted into a two-dimensional feature sequence corresponding to P4;

[0188] Perform Patch Embedding processing on P3 and convert it into a two-dimensional feature sequence corresponding to P3;

[0189] Add the two-dimensional feature sequence corresponding to P4 to a position code of the same size pixel by pixel to obtain the feature sequence B after adding the position code;

[0190] Add the two-dimensional feature sequence corresponding to P3 to a position code of the same size pixel by pixel to obtain the feature sequence C after adding the position code;

[0191] The feature sequence B and the feature sequence C are spliced ​​together according to the channel dimension. The spliced ​​feature sequence passes through the improved Transformer encoder, and the improved Transformer encoder outputs the encoded feature D;

[0192] The encoded feature sequence D output by the improved Transformer encoder is sequentially input into the normalized Norm layer and the FFN multi-layer perceptron, and the FFN multi-layer perceptron outputs the feature sequence E;

[0193] The feature sequence E output by the FFN multi-layer perceptron is deformed and reshaped to obtain a feature map. The width and height of the feature map are the same as those of P3. A 1×1 two-dimensional convolution is performed on the feature map to obtain the feature map P5 after the fusion of the two scale feature maps.

[0194] The other steps and parameters are the same as those in the first to sixth embodiments.

[0195] Specific embodiment 8: This embodiment differs from specific embodiments 1 to 7 in that the specific processing process of the improved Transformer encoder is as follows:

[0196] 1) Combine multi-scale fusion features with weights W V Multiply to get vector V;

[0197] 2) Add a position code of the same size pixel by pixel in the multi-scale fusion feature;

[0198] The multi-scale fusion features after adding position encoding are respectively combined with the weight W Q 、W K Multiply to get vectors Q and K;

[0199] 3) Perform SimAM attention mechanism operations on the obtained vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0200] 4) Input the results of the three SimAM attention mechanisms into the block-wise multi-head attention mechanism respectively, add the output results of the block-wise multi-head attention mechanism and the multi-scale fusion features pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0201] 5) Input the normalized Norm layer output features into the multi-layer perceptron FFN, add the multi-layer perceptron FFN output feature sequence to the normalized Norm layer output features in 4) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0202] 6) Repeat 1) to 5) N times to obtain the improved Transformer encoder output features, 1 <N<10;

[0203] The specific processing process of the block-based multi-head attention mechanism is as follows:

[0204] (1) Change the first row of each SimAM attention mechanism result to the last row and the first column to the last column in the three SimAM attention mechanism results to obtain the converted three SimAM attention mechanism results;

[0205] (2) The converted three SimAM attention mechanism results are added pixel by pixel with the three SimAM attention mechanism results in (1), and then sent to the multi-layer perceptron FFN for encoding. The multi-layer perceptron FFN outputs the encoded features;

[0206] (3) The encoded features are evenly divided into four blocks, each block is input into a multi-head attention mechanism, and the output features of the four multi-head attention mechanisms are concat-synthesized by channel to obtain a feature map as the output features of the block-wise multi-head attention mechanism.

[0207] The base plate anti-loosening wire targets are of different sizes and positions. The use of multi-scale feature fusion can more effectively detect base plate anti-loosening wire targets of different scales. This patent uses the 8-fold, 16-fold, and 32-fold down-sampling feature maps F2, F3, and F4 of the improved PVT feature extraction backbone network for feature fusion to construct a multi-scale feature fusion encoder. The multi-scale feature fusion encoder is as follows: Figure 2 As shown in Figure c, feature maps F2, F3, and F4 undergo 1×1 two-dimensional convolution to become feature maps P2, P3, and P4, respectively. BiFPN feature pyramid feature fusion is then performed to obtain multi-scale fused features, completing multi-scale feature fusion. The improved encoder is then used to obtain the encoded features for category and position.

[0208] The feature fusion method in the multi-scale feature fusion encoder differs from the BiFPN feature pyramid and utilizes an improved Transformer encoder encoding method for feature map fusion. Due to the Transformer encoder's global feature extraction capabilities, using the Transformer encoder instead of the convolution method in BiFPN for feature fusion can more fully integrate features at different scales, thereby improving network accuracy. For example, to fuse P3 and P4, P4 is first upsampled to the size of P3. Patch Embedding is then performed on each to convert it into a two-dimensional feature sequence. These are then pixel-wise added to the positional encoding, and the two feature sequences are concatenated together along the channel dimension. After feature encoding using the improved Transformer encoder, normalization and FFN multi-layer perceptron encoding are performed. The feature sequence is then reshaped into a feature map with the same width and height as P3. Finally, a 1x1 two-dimensional convolution is performed to obtain the fused features of the two scale feature maps. The encoding structure of the improved Transformer encoder is described in the next section.

[0209] Block multi-head attention mechanism and SimAM attention mechanism

[0210] Figure 2 Where d is the multi-scale feature fusion Transformer encoder and improved Transformer decoder structure used in the network, which is similar to the Transformer encoder and Transformer decoder structure in the DETR network, except that the input encoder features are changed to multi-scale fusion features. The multi-head attention mechanism in the encoder and decoder is changed to a block-based multi-head attention mechanism, which reduces the amount of calculation while ensuring the encoding effect of the feature sequence. The SimAM attention mechanism is introduced to improve the network detection accuracy. The Transformer encoder that performs feature fusion in the multi-scale feature fusion encoder also adopts the same improvement measures. The original multi-head attention mechanism can be expressed by formula (1). Where X represents the input multi-scale fusion feature sequence, W Q 、W K 、W V represents the weight matrix, Attention represents the Transformer attention mechanism, and softmax is the activation function. To normalize the weight, MultiHead represents the Transformer multi-head attention mechanism, Concat represents the sequence feature concatenation operation, head i i∈(1,…,h) indicates that the multi-head attention mechanism is divided into several heads, that is, the Transformer attention mechanism is divided into several segments. is the weight matrix that segments Q, K, and V.

[0211] The original Transformer attention mechanism, Attention(Q,K,V), requires matrix multiplication. The size of the input multi-scale fusion feature sequence is very large, and the amount of calculation is large when performing matrix multiplication. Therefore, the Transformer multi-head attention mechanism divides Q, K, and V into segments and performs the Transformer attention mechanism, and then concats the obtained encoding results to reduce the amount of calculation, but the amount of calculation is still not negligible. Therefore, this patent adopts a block multi-head attention mechanism to further reduce the amount of calculation. The process can be described as follows: Figure 3 As shown in the figure, ① the first row of the multi-scale fusion feature is shifted to the last row, and the first column to the last column, and then pixel-by-pixel added to the original multi-scale fusion feature. The feature is then fed into the FFN multi-layer perceptron for encoding. ② The encoded feature is evenly divided into four blocks, and each block is subjected to the multi-head attention mechanism. ③ The encoded features are concatenated to obtain the final encoded multi-scale fusion feature. The reason for performing step ① instead of directly performing steps ②③ is that the multi-scale fusion feature row and column transformation allows features in different blocks to be fused in step ③. Without step ①, the features will only be fused within the four blocks divided in step ②. Step ① strengthens the interaction between information and improves network accuracy.

[0212]

[0213] At the same time, this patent performs SimAM attention mechanism operation on the Q, K, V of the multi-head attention mechanism of the input encoder block and the target query vector of the decoder, so that it focuses on the key features. The SimAM attention mechanism is an attention mechanism proposed by the YOLOV8 network. The specific implementation method is as shown in formula (2). Where X is the input. σ 2 is the variance of the input. μ is the mean of the input. ε is a small constant to avoid the denominator being 0. 2 , μ, and ε to calculate the two parameters α and β, and then calculate the SimAM attention machine result. SimAM is a simple and effective attention mechanism that significantly improves the network's feature representation ability through adaptive weighting without significantly increasing computational overhead.

[0214]

[0215] The other steps and parameters are the same as those in the first to seventh embodiments.

[0216] Specific embodiment 9: This embodiment differs from any one of specific embodiments 1 to 8 in that in step 223, the multi-scale feature fusion Transformer encoder output features and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs features; the specific process is as follows:

[0217] 1) Initialize the target query vector, and the target query vector is respectively Q 、W K 、W V Multiply to get vectors Q, K, V;

[0218] Perform SimAM attention mechanism operations on vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0219] The results of the three SimAM attention machines are input into the block-wise multi-head attention mechanism respectively, and the block-wise multi-head attention mechanism outputs the results;

[0220] 2) Add the output of the multi-head attention mechanism to the target query vector in 1) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0221] 3) Add the normalized Norm layer output features from 2) to the target query vector to obtain a new vector Q;

[0222] Add the improved Transformer encoder output features to a positional encoding of the same size pixel by pixel to obtain a new vector K;

[0223] The improved Transformer encoder output features are used as the new vector V;

[0224] 4) Input the new vector Q, new vector K, and new vector V obtained in 3) into the block-based multi-head attention mechanism, and the block-based multi-head attention mechanism outputs the result;

[0225] 5) Add the output of the multi-head attention mechanism in 4) and the output of the normalized Norm layer in 2) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0226] 6) Input the normalized Norm layer output features of 5) into the FFN multilayer perceptron, add the FFN multilayer perceptron output feature sequence and the normalized Norm layer output features in 5) pixel by pixel, input the added feature sequence into the normalized Norm layer, and normalize the Norm layer output features;

[0227] 7) Repeat 1) to 6) M times to obtain the improved Transformer decoder output features, 1 <M<10。

[0228] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0229] Specific embodiment 10: This embodiment differs from any one of specific embodiments 1 to 9 in that, in step 3, the railway train floor anti-loosening wire breakage fault is detected based on the trained floor anti-loosening wire breakage fault detection network; the specific process is as follows:

[0230] Capture an image of a railway train passing through a high-definition imaging device and feed the bottom camera image into a trained detection network. If the detection network detects a broken bottom plate anti-loosening wire, the image is faulty, a fault message is uploaded, and the next image is tested. If the detection network does not detect a broken bottom plate anti-loosening wire, the image is faulty and no action is taken, continuing with the next image.

[0231] The inspection personnel will further process the fault location based on the uploaded fault message and the principle of manual prior knowledge.

[0232] The present invention installs high-definition imaging equipment around the railway to capture images of the train as it passes. The images of the floor taken by the bottom camera are fed into the floor anti-loosening wire breakage detection network for floor anti-loosening wire breakage fault detection. If a floor anti-loosening wire breakage fault is detected in the image, an alarm message is uploaded, and the railway train inspection personnel will take appropriate measures to address the fault location based on the alarm message.

[0233] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0234] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting a broken wire fault in a railway train floor, characterized by: The specific process of the method is: Step 1: Collect images and build a data set; Step 2: Construct a bottom plate anti-loosening wire breakage fault detection network, train the bottom plate anti-loosening wire breakage fault detection network, and obtain a trained bottom plate anti-loosening wire breakage fault detection network; The specific process is: Step 21: The bottom plate anti-loose wire broken fault detection network includes: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head; The specific process is: The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4; Each stage includes: Patch Embedding, position encoding, branch block, concat, normalization Norm layer, FFN multi-layer perceptron; The branch block includes branch a and branch b; Branch a includes: PVT Transformer Encoder encoding, FFN multi-layer perceptron, one-dimensional convolution Conv1D, the first Relu activation function and SSM module; Branch b includes: two-dimensional convolution, second Relu activation function, two-dimensional convolution, third Relu activation function, normalized Norm layer, FFN multi-layer perceptron; The multi-scale feature fusion Transformer encoder includes: 1×1 two-dimensional convolution, concat splicing, and improved Transformer encoder; The improved Transformer decoder structure includes: block-based multi-head attention mechanism, normalized Norm layer, and multi-layer perceptron (FFN); The detection head is a multi-layer perceptron FFN; Step 22: training a bottom plate anti-loosening wire breakage fault detection network to obtain a trained bottom plate anti-loosening wire breakage fault detection network; Step 3: Detect the railway train floor anti-loosening wire breakage fault based on the trained floor anti-loosening wire breakage fault detection network.

2. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 1, characterized in that: In step 1, images are collected and a data set is established. The specific process is as follows: Step 1: Set up high-definition imaging equipment around the railway to obtain images of passing trains and collect images of the bottom plate taken by the bottom camera; Step 12: amplify the collected baseplate image to obtain an amplified baseplate image; Steps 1 and 3 Mark the image of the amplified base plate image where the base plate anti-loosening wire is normal as a base plate anti-loosening wire normal class, and generate a marking file; Mark the image of the base plate with the broken anti-loosening wire in the amplified base plate image as a base plate anti-loosening wire broken class, and generate a marking file; Step 14: Photoshop the image of the normal bottom plate anti-loosening wire into an image of the broken bottom plate anti-loosening wire, mark it as the broken bottom plate anti-loosening wire category, and generate a marking file; The tag file records the image's name, size, path, location of the anti-loosening wire, type of the anti-loosening wire, and other information; size is the size of the image, and path is the path to the image.

3. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 2, characterized in that: In step 22, the bottom plate anti-loosening wire breakage fault detection network is trained to obtain a trained bottom plate anti-loosening wire breakage fault detection network; the specific process is: Step 221: Input the image in the data set established in step 1 into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features; Step 222: Input the multi-scale features output by the improved PVT feature extraction backbone network into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; Step 223: The multi-scale feature fusion Transformer encoder output features and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs features; Step 224: Input the output features of the improved Transformer decoder into the detection head, and the detection head outputs the target category and target position; Step 225: Use the Adam optimization method, set the initial learning rate to 0.001, and repeat steps 221 to 225 until convergence to obtain a trained bottom plate anti-loose wire breakage fault detection network.

4. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 3, characterized in that: In step 221, the data set established in step 1 is input into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features; The specific process is: The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4; The input image F0 of size H0×W0×C0 is input to stage 1, stage 1 outputs feature map F1 which is input to stage 2, stage 2 outputs feature map F2 which is input to stage 3, stage 3 outputs feature map F3 which is input to stage 4, and stage 4 outputs feature map F4; The specific processing process of each stage i in the stages 1, 2, 3 and 4 is as follows: At stage i, set the size of stage i-1 to H i-1 ×W i-1 ×C i-1 The feature map F i-1 Divided into P i ×P i Patch Embedding is performed on each block, and the sequence processed by Patch Embedding is deformed and reshaped to obtain a sequence of size The characteristic sequence of Add a position code of the same size to each pixel in the feature sequence to obtain the feature sequence A; i≥1； The feature sequence A is input into branch a and branch b respectively. Branch a and branch b respectively output the feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel. After splicing, they pass through the normalization Norm layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F. i ; Feature map F i Size Among them, H i-1 is the feature map F i-1 High, W i-1 is the feature map F i-1 Width, C i-1 is the feature map F i-1 The number of channels, P i ×P i To transform the feature map F i-1 The size of the divided blocks.

5. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 4, characterized in that: The feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output feature sequence encoding results, and the feature sequence encoding results output by branch a and branch b are concat spliced ​​according to the channel. After splicing, they are sequentially passed through the normalization Norm layer and the FFN multi-layer perceptron, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is: The specific processing process of branch a is: The feature sequence A is encoded by PVTTransformer Encoder to extract global features. The size of the encoded result is The size is The encoding result passes through a FFN multi-layer perceptron, which outputs a feature sequence of size The feature sequence output by the FFN multi-layer perceptron passes through a one-dimensional convolution Conv1D, the first Relu activation function and the SSM module in sequence. The feature sequence output by the SSM module is used as the feature sequence encoding result output by branch a, and the size is The specific processing process of branch b is: The feature sequence A is reshaped to obtain a size of H i-1 ×W i-1 ×C i-1 Feature map of Set the size to H i-1 ×W i-1 ×C i-1 The feature map of is sequentially subjected to two-dimensional convolution, the second Relu activation function, two-dimensional convolution, the third Relu activation function, and the third Relu activation function outputs the feature map; the output feature map is deformed and reshaped to obtain a size of The characteristic sequence of For size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn. The FFN multilayer perceptron outputs the feature sequence as the feature sequence encoding result output by branch b, with a size of The feature sequence encoding results output by branch a and branch b are concat-joined by channel, and then passed through the normalization layer and FFN multi-layer perceptron in sequence, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is: The feature sequence encoding results obtained by branch a and branch b are spliced ​​by channel to obtain a size of The feature sequence of size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn, and the FFN multilayer perceptron outputs the coded feature sequence; the FFN multilayer perceptron output coded feature sequence is deformed and reshaped to obtain a size of The feature map F i .

6. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 5, characterized in that: In step 222, the multi-scale features output by the improved PVT feature extraction backbone network are input into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; the specific process is: The feature map F2 undergoes a 1×1 two-dimensional convolution to generate the feature map P2; The feature map F3 undergoes a 1×1 two-dimensional convolution to generate the feature map P3; The feature map F4 undergoes a 1×1 two-dimensional convolution to generate the feature map P4; Fuse the feature map P3 and the feature map P4 to obtain the feature map P5; Fuse the feature map P2 and the feature map P5 to obtain the feature map P6; Fuse the feature maps P3, P5, and P6 to obtain the feature map P7; Fuse the feature map P4 and the feature map P7 to obtain the feature map P8; Concatenate feature maps P6, P7, and P8 by channel to obtain feature map P9; The feature map P9 is input into a 1×1 two-dimensional convolution, and the 1×1 two-dimensional convolution outputs a multi-scale fusion feature sequence; The multi-scale fusion feature sequence is input into the improved Transformer encoder, and the improved Transformer encoder outputs the encoded features, which are used as the multi-scale feature fusion Transformer encoder output features.

7. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 6, characterized in that: The feature map P3 and the feature map P4 are fused to obtain the feature map P5; the specific process is: First, upsample P4 to the size of P3; P4, which is upsampled to the same size as P3, is processed by PatchEmbedding and converted into a two-dimensional feature sequence corresponding to P4; Perform Patch Embedding processing on P3 and convert it into a two-dimensional feature sequence corresponding to P3; Add the two-dimensional feature sequence corresponding to P4 to a position code of the same size pixel by pixel to obtain the feature sequence B after adding the position code; Add the two-dimensional feature sequence corresponding to P3 to a position code of the same size pixel by pixel to obtain the feature sequence C after adding the position code; The feature sequence B and the feature sequence C are spliced ​​together according to the channel dimension. The spliced ​​feature sequence passes through the improved Transformer encoder, and the improved Transformer encoder outputs the encoded feature D; The encoded feature sequence D output by the improved Transformer encoder is sequentially input into the normalized Norm layer and the FFN multi-layer perceptron, and the FFN multi-layer perceptron outputs the feature sequence E; The FFN multi-layer perceptron output feature sequence E is deformed and reshaped to obtain a feature map. The width and height of the feature map are the same as those of P3. A 1×1 two-dimensional convolution is performed on the feature map to obtain the feature map P5.

8. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 7, characterized in that: The specific processing process of the improved Transformer encoder is as follows: 1) Combine multi-scale fusion features with weights W V Multiply to get vector V; 2) Add a position code of the same size pixel by pixel in the multi-scale fusion feature; The multi-scale fusion features after adding position encoding are respectively combined with the weight W Q 、W K Multiply to get vectors Q and K; 3) Perform SimAM attention mechanism operations on the obtained vectors Q, K, and V respectively to obtain three SimAM attention machine results; 4) Input the results of the three SimAM attention mechanisms into the block-wise multi-head attention mechanism respectively, add the output results of the block-wise multi-head attention mechanism and the multi-scale fusion features pixel by pixel, and input the added feature sequence into the normalized Norm layer; 5) Input the normalized Norm layer output features into the multi-layer perceptron FFN, add the multi-layer perceptron FFN output feature sequence to the normalized Norm layer output features in 4) pixel by pixel, and input the added feature sequence into the normalized Norm layer; 6) Repeat 1) to 5) N times to obtain the improved Transformer encoder output features; The specific processing process of the block-based multi-head attention mechanism is as follows: (1) Change the first row of each SimAM attention mechanism result to the last row and the first column to the last column in the three SimAM attention mechanism results to obtain the converted three SimAM attention mechanism results; (2) The converted three SimAM attention mechanism results are added pixel by pixel with the three SimAM attention mechanism results in (1), and then sent to the multi-layer perceptron FFN for encoding. The multi-layer perceptron FFN outputs the encoded features; (3) The encoded features are evenly divided into four blocks, each of which is input into a multi-head attention mechanism. The output features of the four multi-head attention mechanisms are spliced ​​by channel to obtain a feature map as the output features of the block multi-head attention mechanism.

9. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 8, characterized in that: In step 223, the multi-scale feature fusion Transformer encoder output features and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs features. The specific process is as follows: 1) Initialize the target query vector, and the target query vector is respectively Q 、W K 、W V Multiply to get vectors Q, K, V; Perform SimAM attention mechanism operations on vectors Q, K, and V respectively to obtain three SimAM attention machine results; The results of the three SimAM attention machines are input into the block-wise multi-head attention mechanism respectively, and the block-wise multi-head attention mechanism outputs the results; 2) Add the output of the multi-head attention mechanism to the target query vector in 1) pixel by pixel, and input the added feature sequence into the normalized Norm layer; 3) Add the normalized Norm layer output features from 2) to the target query vector to obtain a new vector Q; Add the improved Transformer encoder output features to a positional encoding of the same size pixel by pixel to obtain a new vector K; The improved Transformer encoder output features are used as the new vector V; 4) Input the new vector Q, new vector K, and new vector V obtained in 3) into the block-based multi-head attention mechanism, and the block-based multi-head attention mechanism outputs the result; 5) Add the output of the multi-head attention mechanism in 4) and the output of the normalized Norm layer in 2) pixel by pixel, and input the added feature sequence into the normalized Norm layer; 6) Input the normalized Norm layer output features of 5) into the FFN multilayer perceptron, add the FFN multilayer perceptron output feature sequence and the normalized Norm layer output features in 5) pixel by pixel, input the added feature sequence into the normalized Norm layer, and normalize the Norm layer output features; 7) Repeat 1) to 6) M times to obtain the improved Transformer decoder output features.

10. A method for detecting a broken wire fault in a railway train floor anti-loosening wire according to claim 9, characterized in that: In step 3, the railway train floor anti-loosening wire breakage fault is detected based on the trained floor anti-loosening wire breakage fault detection network; the specific process is: Capture an image of a railway train passing through a high-definition imaging device and feed the bottom camera image into a trained detection network. If the detection network detects a broken bottom plate anti-loosening wire, the image is faulty, a fault message is uploaded, and the next image is inspected. If the detection network does not detect a broken bottom plate anti-loosening wire, the image is faulty and no action is taken, continuing with the next image.

Images