Railway bullet train bottom plate anti-loose iron wire fracture fault detection method

Abstract

The invention relates to a railway motor car bottom plate anti-loose iron wire fracture fault detection method. The invention relates to a railway motor car bottom plate anti-loose iron wire fracture fault detection method. The objective of the invention is to solve the problem that an existing deep learning detection network based on convolution is poor in fault detection recognition effect and easy to cause missing detection. The method comprises the steps of collecting images and establishing a data set; the method comprises the following steps: constructing a bottom plate anti-loose iron wire breakage fault detection network, wherein the network comprises an improved PVT feature extraction backbone network, a multi-scale feature fusion Transform encoder, an improved Transform decoder structure and a detection head; training the bottom plate anti-loose iron wire fracture fault detection network to obtain a trained bottom plate anti-loose iron wire fracture fault detection network; and detecting the railway bullet train bottom plate anti-loose iron wire fracture fault based on the trained bottom plate anti-loose iron wire fracture fault detection network. The method is applied to the field of railway bullet train detection.

Description

Technical Field

[0001] The invention relates to a method for detecting a breakage fault of an anti-loosening iron wire on a railway motor vehicle bottom plate. Background Art

[0002] The traditional manual fault detection method of checking the map is time-consuming and labor-intensive, with high labor costs, and due to fatigue and carelessness of the inspectors, missed detection and false detection may occur. The railway EMU fault detection method using deep learning can effectively reduce the detection cost and improve the efficiency of detection. However, due to the large number of bottom plate anti-loosening wires, their thin shapes, and unclear features after breaking, the existing convolution-based deep learning detection network has poor recognition effect for fault detection, and is prone to missed detection. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that the existing bottom plate anti-loosening wires are distributed in large numbers, have thin shapes, and have unclear characteristics after breaking, resulting in poor recognition effect of fault detection based on convolutional deep learning detection networks and easy missed detection. A method for detecting the breakage of anti-loosening wires on the bottom plate of railway EMUs is proposed.

[0004] A method for detecting a broken wire fault in a railway motor vehicle bottom plate is as follows:

[0005] Step 1: Collect images and build a data set;

[0006] Step 2: Construct a bottom plate anti-loose wire break fault detection network, train the bottom plate anti-loose wire break fault detection network, and obtain a trained bottom plate anti-loose wire break fault detection network; the specific process is:

[0007] Step 21. The bottom plate anti-loose wire breakage fault detection network includes: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head;

[0008] The specific process is:

[0009] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0010] Each stage includes: Patch Embedding, position encoding, branch block, concat, normalization Norm layer, FFN multi-layer perceptron;

[0011] The branch block includes branch a and branch b;

[0012] Branch a includes:

[0013] PVT Transformer Encoder encoding, FFN multi-layer perceptron, one-dimensional convolution Conv1D, first Relu activation function and SSM module;

[0014] Branch b includes: two-dimensional convolution, second Relu activation function, two-dimensional convolution, third Relu activation function, normalized Norm layer, FFN multi-layer perceptron;

[0015] The multi-scale feature fusion Transformer encoder includes: 1×1 two-dimensional convolution, concat splicing, and improved Transformer encoder;

[0016] The improved Transformer decoder structure includes: block-based multi-head attention mechanism, normalized Norm layer, and multi-layer perceptron FFN;

[0017] The detection head is a multi-layer perceptron FFN;

[0018] Step 22: training a bottom plate anti-loose wire break fault detection network to obtain a trained bottom plate anti-loose wire break fault detection network;

[0019] Step 3: Detect the railway train bottom plate anti-loose wire breakage fault based on the trained bottom plate anti-loose wire breakage fault detection network.

[0020] Preferably, in step 1, images are collected to establish a data set; the specific process is:

[0021] Step 1: Set up high-definition imaging equipment around the railway to obtain images of passing trains and collect bottom plate images taken by bottom cameras;

[0022] Step 12: amplify the collected base plate image to obtain an amplified base plate image;

[0023] Step 1-3

[0024] Mark the image in the amplified bottom plate image where the bottom plate anti-loosening wire is normal as a bottom plate anti-loosening wire normal class, and generate a marking file;

[0025] Mark the image of the bottom plate anti-loose wire broken in the amplified bottom plate image as the bottom plate anti-loose wire broken class, and generate a marking file;

[0026] Step 14: Photoshop the normal image of the bottom plate anti-loose wire into an image of the bottom plate anti-loose wire broken, mark it as the bottom plate anti-loose wire broken class, and generate a marking file;

[0027] The tag file records the image's name, size, path, location of the anti-loosening wire, type of the anti-loosening wire, and other information; the size is the size of the image, and the path is the path of the image.

[0028] Preferably, in step 22, the bottom plate anti-loose wire break fault detection network is trained to obtain a trained bottom plate anti-loose wire break fault detection network; the specific process is:

[0029] Step 221: input the image in the data set established in step 1 into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features;

[0030] Step 222: input the multi-scale features output by the improved PVT feature extraction backbone network into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features;

[0031] Step 223: The multi-scale feature fusion Transformer encoder output feature and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs the feature;

[0032] Step 224: Input the output features of the improved Transformer decoder into the detection head, and the detection head outputs the output target category and target position;

[0033] Step 225: Adopt the Adam optimization method, set the initial learning rate to 0.001, and repeat steps 221 to 225 until convergence to obtain a trained bottom plate anti-loose wire break fault detection network.

[0034] Preferably, in step 221, the image in the data set established in step 1 is input into an improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features;

[0035] The specific process is:

[0036] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0037] The input image F0 of size H0×W0×C0 is input to stage 1, stage 1 outputs feature map F1 which is input to stage 2, stage 2 outputs feature map F2 which is input to stage 3, stage 3 outputs feature map F3 which is input to stage 4, and stage 4 outputs feature map F4;

[0038] The specific processing process of each stage i in the stage 1, stage 2, stage 3, and stage 4 is as follows:

[0039] At stage i, set the size of stage i-1 to H i-1 ×W i-1 ×C i-1 The feature map F i-1Divide into P i ×P i The size of the block is , and each block is processed by Patch Embedding. The sequence after Patch Embedding is reshaped to obtain a size of The characteristic sequence of size Add a position code of the same size to each pixel in the feature sequence to obtain the feature sequence A; i≥1;

[0040] The feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output the feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel, and then pass through the normalization Norm layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; Feature map F i Size

[0041] Among them, H i-1 is the feature map F i-1 High, W i-1 is the feature map F i-1 Width, C i-1 is the feature map F i-1 The number of channels, P i ×P i To transform the feature map F i-1 The size of the divided blocks;

[0042] Preferably, the feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output feature sequence encoding results, and the feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel, and then pass through the normalization Norm layer and FFN multi-layer perceptron in turn, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is:

[0043] The specific processing process of branch a is:

[0044] The feature sequence A is encoded by PVT Transformer Encoder for global feature extraction, and the size of the encoding result is The size is The encoding result passes through a FFN multi-layer perceptron, which outputs a feature sequence Output feature sequence of FFN multi-layer perceptron After a one-dimensional convolution Conv1D, the first Relu activation function and the SSM module, the SSM module outputs a feature sequence as the feature sequence encoding result output by branch a, with a size of

[0045] The specific processing process of branch b is:

[0046] The feature sequence A is reshaped to obtain a size of H i-1 ×W i-1 ×C i-1 The feature map of

[0047] Set the size to H i-1 ×W i-1 ×C i-1 The feature map of is sequentially subjected to two-dimensional convolution, the second Relu activation function, two-dimensional convolution, and the third Relu activation function. The third Relu activation function outputs the feature map; the output feature map is deformed and reshaped to obtain a size of The characteristic sequence of

[0048] For size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn. The FFN multilayer perceptron outputs the feature sequence as the feature sequence encoding result output by branch b, with a size of

[0049] The feature sequence encoding results output by branch a and branch b are concatenated by channel, and then passed through the normalization layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are reshaped to obtain the feature map F i ; The specific process is:

[0050] The feature sequence encoding results obtained by branch a and branch b are concatenated by channel to obtain a size of The feature sequence of size The feature sequence is input into the normalized Norm layer and the FFN multilayer perceptron in turn, and the FFN multilayer perceptron outputs the coded feature sequence; the FFN multilayer perceptron output coded feature sequence is deformed and reshaped to obtain The feature map F i .

[0051] Preferably, in step 222, the multi-scale features output by the improved PVT feature extraction backbone network are input into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; the specific process is:

[0052] The feature map F2 undergoes a 1×1 two-dimensional convolution to generate the feature map P2;

[0053] Feature map F3 undergoes 1×1 two-dimensional convolution to generate feature map P3;

[0054] Feature map F4 undergoes 1×1 two-dimensional convolution to generate feature map P4;

[0055] The feature map P3 and the feature map P4 are fused to obtain the feature map P5;

[0056] The feature map P2 and the feature map P5 are fused to obtain the feature map P6;

[0057] The feature map P3, the feature map P5 and the feature map P6 are fused to obtain the feature map P7;

[0058] The feature map P4 and the feature map P7 are fused to obtain the feature map P8;

[0059] Concatenate feature maps P6, P7, and P8 by channel to obtain feature map P9;

[0060] The feature map P9 is input into a 1×1 two-dimensional convolution, and the 1×1 two-dimensional convolution outputs a multi-scale fusion feature sequence;

[0061] The multi-scale fusion feature sequence is input into the improved Transformer encoder, and the improved Transformer encoder outputs the encoded features, which are used as the multi-scale feature fusion Transformer encoder output features.

[0062] Preferably, the feature map P3 and the feature map P4 are fused to obtain the feature map P5;

[0063] The specific process is:

[0064] First, upsample P4 to the size of P3;

[0065] P4, which is upsampled to the same size as P3, is processed by Patch Embedding and converted into a two-dimensional feature sequence corresponding to P4;

[0066] Perform Patch Embedding on P3 and convert it into a two-dimensional feature sequence corresponding to P3;

[0067] Add the two-dimensional feature sequence corresponding to P4 to a position code of the same size pixel by pixel to obtain the feature sequence B after adding the position code;

[0068] Add the two-dimensional feature sequence corresponding to P3 to a position code of the same size pixel by pixel to obtain the feature sequence C after adding the position code;

[0069] The feature sequence B and the feature sequence C are concatenated together according to the channel dimension. The concatenated feature sequence is passed through an improved Transformer encoder, and the improved Transformer encoder outputs the encoded feature D.

[0070] The encoded feature sequence D output by the improved Transformer encoder is sequentially input into the normalized Norm layer and the FFN multi-layer perceptron, and the FFN multi-layer perceptron outputs the feature sequence E;

[0071] The feature sequence E output by the FFN multi-layer perceptron is reshaped to obtain a feature map whose width and height are the same as those of P3. A 1×1 two-dimensional convolution is performed on the feature map to obtain the feature map P5.

[0072] Preferably, the specific processing process of the improved Transformer encoder is:

[0073] 1) Combine multi-scale fusion features with weights W V Multiply to get vector V;

[0074] 2) Add a position code of the same size pixel by pixel in the multi-scale fusion feature;

[0075] The multi-scale fusion features after adding position encoding are respectively combined with the weights W Q , W K Multiply to get vectors Q and K;

[0076] 3) Perform the SimAM attention mechanism operation on the obtained vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0077] 4) Input the results of the three SimAM attention mechanisms into the block-based multi-head attention mechanism respectively, add the output results of the block-based multi-head attention mechanism and the multi-scale fusion features pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0078] 5) Input the normalized Norm layer output features into the multi-layer perceptron FFN, add the multi-layer perceptron FFN output feature sequence to the normalized Norm layer output features in 4) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0079] 6) Repeat 1) to 5) N times to obtain the improved Transformer encoder output features;

[0080] The specific processing process of the block-based multi-head attention mechanism is as follows:

[0081] (1) Change the first row of each SimAM attention mechanism result to the last row, and the first column to the last column in the three SimAM attention mechanism results to obtain the converted three SimAM attention mechanism results;

[0082] (2) The converted three SimAM attention mechanism results are added pixel by pixel with the three SimAM attention mechanism results in (1), and then sent to the multi-layer perceptron FFN for encoding. The multi-layer perceptron FFN outputs the encoded features.

[0083] (3) The encoded features are evenly divided into four blocks, each of which is input into a multi-head attention mechanism. The output features of the four multi-head attention mechanisms are concatenated by channel to obtain a feature map as the output features of the divided multi-head attention mechanism.

[0084] Preferably, in step 223, the multi-scale feature fusion Transformer encoder output feature and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs the feature; the specific process is:

[0085] 1) Initialize the target query vector, which is respectively related to the weight W Q , W K , W V Multiply to get vectors Q, K, V;

[0086] Perform the SimAM attention mechanism operation on vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0087] The results of the three SimAM attention machines are input into the block-based multi-head attention mechanism respectively, and the block-based multi-head attention mechanism outputs the results;

[0088] 2) Add the output of the multi-head attention mechanism of the block to the target query vector in 1) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0089] 3) Add the normalized Norm layer output features from 2) to the target query vector to obtain a new vector Q;

[0090] Add the improved Transformer encoder output features to a positional encoding of the same size pixel by pixel to obtain a new vector K;

[0091] The improved Transformer encoder output features are used as the new vector V;

[0092] 4) Input the new vector Q, new vector K, and new vector V obtained in 3) into the block-based multi-head attention mechanism, and the block-based multi-head attention mechanism outputs the result;

[0093] 5) Add the output results of the multi-head attention mechanism of the block in 4) and the output results of the normalized Norm layer in 2) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0094] 6) Input the normalized Norm layer output features of 5) into the FFN multilayer perceptron, add the FFN multilayer perceptron output feature sequence and the normalized Norm layer output features in 5) pixel by pixel, input the added feature sequence into the normalized Norm layer, and normalize the Norm layer output features;

[0095] 7) Repeat 1) to 6) M times to obtain the improved Transformer decoder output features.

[0096] Preferably, in step three, the railway vehicle bottom plate anti-loose wire breakage fault is detected based on the trained bottom plate anti-loose wire breakage fault detection network; the specific process is:

[0097] Obtain the train passing image after the high-definition imaging device, and send the bottom camera image to the trained detection network. If the detection network detects that the bottom plate anti-loose wire is broken, it proves that the image has a fault, uploads the fault message, and continues to detect the next image. If the detection network does not detect that the bottom plate anti-loose wire is broken, it proves that the image does not have a fault, and no operation is performed, and continues to detect the next image.

[0098] The beneficial effects of the present invention are:

[0099] The present invention adopts a Transformer-based PVT network to replace the original convolution-based backbone network in DETR, and improves the PVT network by introducing convolution and SSM branches to improve the global and local feature extraction capabilities of the network.

[0100] The present invention introduces a Transformer-based multi-scale feature pyramid in DETR and performs multi-scale feature fusion encoding, so that the network can comprehensively utilize features of different scales and improve the detection accuracy of targets of different scales in the network.

[0101] The Transformer encoder and Transformer decoder of the present invention adopt a block-based multi-head attention mechanism to reduce the amount of model calculation, and introduce a SimAM attention mechanism to improve network detection accuracy.

[0102] Compared with the detection network based on convolution, the DETR detection network based on Transformer in the present invention can better extract the global features of the image. Therefore, the present invention adopts the improved DETR detection network for fault detection, which can effectively reduce missed detection and improve the fault detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Figure 1 This is a fault identification flow chart of the present invention;

[0104] Figure 2 This is the improved DETR bottom plate anti-loose wire missing fault detection network structure diagram, a is the overall network result, b is the improved PVT feature extraction backbone network, c is the multi-scale feature fusion Transformer encoder, d is the improved multi-scale feature fusion Transformer encoder and Transformer decoder structure;

[0105] Figure 3 Diagram of the block-based multi-head attention mechanism. DETAILED DESCRIPTION

[0106] Specific implementation method 1: In this implementation method, a method for detecting a broken wire fault in a railway vehicle bottom plate is specifically performed as follows:

[0107] Step 1: Collect images and build a data set;

[0108] Step 2: Construct a bottom plate anti-loose wire break fault detection network, train the bottom plate anti-loose wire break fault detection network, and obtain a trained bottom plate anti-loose wire break fault detection network; the specific process is:

[0109] Step 21. The bottom plate anti-loose wire breakage fault detection network includes: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head;

[0110] The specific process is:

[0111] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0112] Each stage includes: Patch Embedding, position encoding, branch block, concat, normalization Norm layer, FFN multi-layer perceptron;

[0113] The branch block includes branch a and branch b;

[0114] Branch a includes:

[0115] PVT Transformer Encoder encoding, FFN multi-layer perceptron, one-dimensional convolution Conv1D, first Relu activation function and SSM module;

[0116] Branch b includes: two-dimensional convolution, second Relu activation function, two-dimensional convolution, third Relu activation function, normalized Norm layer, FFN multi-layer perceptron;

[0117] The multi-scale feature fusion Transformer encoder includes: 1×1 two-dimensional convolution, concat splicing, and improved Transformer encoder;

[0118] The improved Transformer decoder structure includes: block-based multi-head attention mechanism, normalized Norm layer, and multi-layer perceptron FFN;

[0119] The detection head is a multi-layer perceptron FFN;

[0120] Step 22: training a bottom plate anti-loose wire break fault detection network to obtain a trained bottom plate anti-loose wire break fault detection network;

[0121] Step 3: Detect the railway train bottom plate anti-loose wire breakage fault based on the trained bottom plate anti-loose wire breakage fault detection network.

[0122] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that in step 1, images are collected and a data set is established; the specific process is as follows:

[0123] Step 1: Set up high-definition imaging equipment around the railway to obtain images of passing trains and collect bottom plate images taken by bottom cameras;

[0124] Step 1 and 2: perform data amplification on the collected base plate image, increase the anti-loosening wire morphology in the data set as much as possible, and obtain the amplified base plate image;

[0125] Step 1-3

[0126] Mark the image in the amplified bottom plate image where the bottom plate anti-loosening wire is normal as a bottom plate anti-loosening wire normal class, and generate a marking file;

[0127] Mark the image of the bottom plate anti-loose wire broken in the amplified bottom plate image as the bottom plate anti-loose wire broken class, and generate a marking file;

[0128] Mark the images with normal anti-loose wires in the amplified base plate images that are similar to the broken state of the base plate anti-loose wires (some anti-loose wires are relatively thin due to shooting or their own reasons, and if not carefully judged, they look like they are broken in the middle; some foreign objects such as weeds also look like the anti-loose wires are broken) as interference class, and generate a marking file;

[0129] Step 14: Photoshop the normal image of the bottom plate anti-loose wire into an image of the bottom plate anti-loose wire broken, mark it as the bottom plate anti-loose wire broken class, and generate a marking file;

[0130] The tag file records the image name, size, path, location of the anti-loosening wire, type of the anti-loosening wire, etc. The size is the size of the image, and the path is the path of the image.

[0131] High-definition imaging equipment is set up around the railway to obtain images of passing trains after the railway trains pass through, and the images of the bottom plate taken by the bottom camera are collected for subsequent data set establishment. The images of the normal bottom plate anti-loose wire and the images of the broken bottom plate anti-loose wire are collected as the detection data set images. Since the frequency of the anti-loose wire broken fault is low and the fault image is difficult to collect, it is necessary to imitate the real fault on the normal image of the bottom plate anti-loose wire to PS the bottom plate anti-loose wire broken fault as a supplement to the data set. The data set is marked, the marking file is generated, and the collection and production of the data set are completed. The marking process is implemented using the labelImg marking software to generate the marking file corresponding to the image, in which the normal bottom plate anti-loose wire target is marked as the bottom plate anti-loose wire normal class, the target with the broken bottom plate anti-loose wire is marked as the bottom plate anti-loose wire broken class, and the normal bottom plate parts similar to the bottom plate anti-loose wire broken state are marked as interference classes. The marking file records the image name, size, path, target location, target category and other information.

[0132] The position of the bottom plate anti-loosening wire on the bottom plate of the EMU is not fixed, and the bending shape, angle, length, and thickness are not fixed. The collected data set cannot cover all wire shapes. It is necessary to augment the data set to increase the anti-loosening wire shapes in the data set as much as possible. The data augmentation operation includes affine transformation, rotation, cropping, contrast transformation and other operations. The data augmentation operation can effectively reduce the probability of overfitting of the fault detection network and improve the generalization performance of the fault detection network.

[0133] The other steps and parameters are the same as those in the first embodiment.

[0134] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that in step 22, the bottom plate anti-loose wire break fault detection network is trained to obtain a trained bottom plate anti-loose wire break fault detection network;

[0135] The specific process is:

[0136] Step 221: input the data set established in step 1 into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features;

[0137] Step 222: input the multi-scale features output by the improved PVT feature extraction backbone network into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features;

[0138] Step 223: The multi-scale feature fusion Transformer encoder output feature and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs the feature;

[0139] Step 224: Input the output features of the improved Transformer decoder into the detection head, and the detection head outputs the output target category and target position;

[0140] Step 225: Adopt the Adam optimization method, set the initial learning rate to 0.001, and repeat steps 221 to 225 until convergence to obtain a trained bottom plate anti-loose wire break fault detection network.

[0141] The network was trained using Adam optimization, and the initial learning rate was set to 0.001. The data set was divided into training set, validation set, and test set in a ratio of 7:2:1. The detection network was trained using training set images, tested on the validation set, and then the images with incorrect recognition in the validation set were added to the training set to retrain the detection network. Finally, the performance of the network was tested on the test set.

[0142] The other steps and parameters are the same as those in the first or second embodiment.

[0143] Specific implementation method 4: This implementation method is different from specific implementation methods 1 to 3 in that in step 221, the image in the data set established in step 1 is input into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features; the specific process is:

[0144] The PVT feature extraction backbone network adopts the Transformer structure, which enables the network to more fully extract the global features of the image. At the same time, the network implements the downsampling of feature maps through Patch Embedding, and obtains feature maps of different scales with 4 times, 8 times, 16 times, and 32 times downsampling, so that the network can extract features of different scales in the image. Subsequent feature fusion can improve the network's detection ability for targets of different scales. However, the Transformer structure has high computational complexity and is difficult to train. This patent adds convolution branches and SSM branches to PVT, which reduces the amount of computation of the model, reduces the difficulty of training the model, and improves the network's ability to extract features. The improved PVT feature extraction backbone network structure is as follows: Figure 2 As shown in b.

[0145] The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4;

[0146] The input image F0 of size H0×W0×C0 is input to stage 1, stage 1 outputs feature map F1 which is input to stage 2, stage 2 outputs feature map F2 which is input to stage 3, stage 3 outputs feature map F3 which is input to stage 4, and stage 4 outputs feature map F4;

[0147] C0 is the number of image channels, it is a three-channel color image so it is 3;

[0148] The specific processing process of each stage i in the stage 1, stage 2, stage 3, and stage 4 is as follows:

[0149] At stage i, set the size of stage i-1 to H i-1 ×W i-1 ×C i-1 The feature map F i-1 Divide into P i ×P i The size of the block is , and each block is processed by Patch Embedding. The sequence after Patch Embedding is reshaped to obtain a size of The characteristic sequence of size Add a position code of the same size to each pixel in the feature sequence to obtain the feature sequence A; i≥1;

[0150] The feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output the feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel, and then pass through the normalization Norm layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; Feature map Fi Size

[0151] Among them, H i-1 is the feature map F i-1 High, W i-1 is the feature map F i-1 Width, C i-1 is the feature map F i-1 The number of channels, P i ×P i To transform the feature map F i-1 The size of the divided block (width is P i , high is P i ).

[0152] The input image F0 of size H×W×3 is input to stage 1, and the output size of stage 1 is Feature map F1;

[0153] The feature map F1 is input to stage 2, and the output size of stage 2 is Feature map F2;

[0154] The feature map F2 is input to stage 3, and the output size of stage 3 is Feature map F3;

[0155] The feature map F3 is input to stage 4, and the output size of stage 4 is Feature map F4;

[0156] Among them, H is the height of the image, W is the width of the image, C1 is the number of channels of the feature map F1, C2 is the number of channels of the feature map F2, C3 is the number of channels of the feature map F3, and C4 is the number of channels of the feature map F4.

[0157] The input image F0 of size H×W×3 is processed through stages 1, 2, 3, and 4 to generate feature maps of different scales that are downsampled by 4 times, 8 times, 16 times, and 32 times, respectively.

[0158] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0159] Specific implementation method five: This implementation method is different from the specific implementation methods one to four in that the feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output feature sequence encoding results, and the feature sequence encoding results output by branch a and branch b are concat spliced ​​according to channels, and after splicing, they are sequentially passed through the normalization Norm layer and the FFN multi-layer perceptron, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is:

[0160] The specific processing process of branch a is:

[0161] Branch a uses a PVTTransformer Encoder encoding module and an SSM module;

[0162] The feature sequence A is encoded by PVTTransformer Encoder for global feature extraction, and the size of the encoding result is The size is The encoding result is passed through an FFN multi-layer perceptron to reduce the feature sequence dimension by half, and the FFN multi-layer perceptron outputs the feature sequence Reduce the amount of calculation; output feature sequence of FFN multi-layer perceptron After a one-dimensional convolution Conv1D, the first Relu activation function and the SSM module, the SSM module outputs a feature sequence as the feature sequence encoding result output by branch a, with a size of

[0163] The specific processing process of branch b is:

[0164] The feature sequence A is reshaped to obtain a size of H i-1 ×W i-1 ×C i-1 The feature map of

[0165] Set the size to H i-1 ×W i-1 ×C i-1 The feature map of is sequentially subjected to two-dimensional convolution, the second Relu activation function, two-dimensional convolution, and the third Relu activation function. The third Relu activation function outputs the feature map; the output feature map is deformed and reshaped to obtain a size of The characteristic sequence of

[0166] For size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn. The FFN multilayer perceptron outputs the feature sequence as the feature sequence encoding result output by branch b, with a size of

[0167] The feature sequence encoding results output by branch a and branch b are concatenated by channel, and then passed through the normalization layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are reshaped to obtain the feature map F i , and obtain the feature encoding result of stage i; the specific process is:

[0168] The feature sequence encoding results obtained by branch a and branch b are concatenated by channel to obtain a size of

[0169] The feature sequence of size The feature sequence is input into the normalized Norm layer and the FFN multilayer perceptron in turn, and the FFN multilayer perceptron outputs the coded feature sequence; the FFN multilayer perceptron output coded feature sequence is deformed and reshaped to obtain The feature map F i , get the feature encoding result of stage i;

[0170] The SSM module in the network is a sequence feature encoding module proposed by the Mamba network that can be flexibly embedded in the network. Compared with the Transformer encoder, SSM has lower computational complexity and better generalization ability while extracting the global features of the sequence, which can reduce the probability of network overfitting. Compared with convolution, the Transformer encoder and SSM have high complexity and high training difficulty. Adding convolution branches to the network can effectively reduce the computational complexity of the model, stabilize the recognition effect of the network, and reduce the computational difficulty of the network. At the same time, the Transformer encoder and SSM can extract the global features of the image, but the ability to extract local features is poor. After adding the convolution branch, more refined local features with rotation invariance can be better extracted.

[0171] The broken shape of the bottom plate anti-loose wire is not obvious and has various shapes. The detection effect of the convolution-based fault detection network is not ideal. The Transformer and DETR detection networks can fully extract the global features of the image, avoid the local induction preference problem in the convolution network, and can better generalize to unseen images, better distinguish between broken and normal wires, and more accurately identify the location of the broken wire. This patent improves the DETR detection network, enhances the network's recognition ability for small target anti-loose wires, and enables the network to locate the fault location more accurately. First, the Transformer-based PVT network is used to replace the original convolution-based feature extraction backbone network in the DETR network, and the PVT network is improved by introducing convolution and SSM branches to improve the network's global and local feature extraction capabilities. Secondly, a Transformer-based multi-scale feature pyramid is introduced in the DETR network to perform multi-scale feature fusion, and together with the network encoder, a multi-scale feature fusion encoder is formed to improve the network's recognition ability for multi-scale targets. The multi-head attention mechanism in the DETR network encoder and decoder is improved, and the block multi-head attention mechanism is used to reduce the network calculation amount without affecting the network accuracy. The SimAM attention mechanism is introduced to make the detection result frame generated by the network more accurate. The overall structure of the network in this patent is as follows Figure 2As shown in (a), the network consists of an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder and a detection head.

[0172] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0173] Specific implementation method six: This implementation method is different from specific implementation methods one to five in that in step 222, the multi-scale features output by the improved PVT feature extraction backbone network are input into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; the specific process is:

[0174] The feature map F2 undergoes a 1×1 two-dimensional convolution to generate the feature map P2;

[0175] Feature map F3 undergoes 1×1 two-dimensional convolution to generate feature map P3;

[0176] Feature map F4 undergoes 1×1 two-dimensional convolution to generate feature map P4;

[0177] The feature map P3 and the feature map P4 are fused to obtain the feature map P5;

[0178] The feature map P2 and the feature map P5 are fused to obtain the feature map P6;

[0179] The feature map P3, the feature map P5 and the feature map P6 are fused to obtain the feature map P7;

[0180] The feature map P4 and the feature map P7 are fused to obtain the feature map P8;

[0181] Concatenate feature maps P6, P7, and P8 by channel to obtain feature map P9;

[0182] The feature map P9 is input into a 1×1 two-dimensional convolution, and the 1×1 two-dimensional convolution outputs a multi-scale fusion feature sequence;

[0183] The multi-scale fusion feature sequence is input into the improved Transformer encoder, and the improved Transformer encoder outputs the encoded features, which are used as the multi-scale feature fusion Transformer encoder output features.

[0184] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0185] Specific implementation method 7: This implementation method is different from the specific implementation methods 1 to 6 in that the feature map P3 and the feature map P4 are fused to obtain the feature map P5; the specific process is:

[0186] First, upsample P4 to the size of P3;

[0187] P4, which is upsampled to the same size as P3, is processed by Patch Embedding and converted into a two-dimensional feature sequence corresponding to P4;

[0188] Perform Patch Embedding on P3 and convert it into a two-dimensional feature sequence corresponding to P3;

[0189] Add the two-dimensional feature sequence corresponding to P4 to a position code of the same size pixel by pixel to obtain the feature sequence B after adding the position code;

[0190] Add the two-dimensional feature sequence corresponding to P3 to a position code of the same size pixel by pixel to obtain the feature sequence C after adding the position code;

[0191] The feature sequence B and the feature sequence C are concatenated together according to the channel dimension. The concatenated feature sequence is passed through an improved Transformer encoder, and the improved Transformer encoder outputs the encoded feature D.

[0192] The encoded feature sequence D output by the improved Transformer encoder is sequentially input into the normalized Norm layer and the FFN multi-layer perceptron, and the FFN multi-layer perceptron outputs the feature sequence E;

[0193] The feature sequence E output by the FFN multi-layer perceptron is deformed and reshaped to obtain a feature map. The width and height of the feature map are the same as those of P3. A 1×1 two-dimensional convolution is performed on the feature map to obtain a feature map P5 after the fusion of the two scale feature maps.

[0194] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0195] Specific implementation eight: This implementation differs from specific implementations one to seven in that the specific processing process of the improved Transformer encoder is as follows:

[0196] 1) Combine multi-scale fusion features with weights W V Multiply to get vector V;

[0197] 2) Add a position code of the same size pixel by pixel in the multi-scale fusion feature;

[0198] The multi-scale fusion features after adding position encoding are respectively combined with the weights W Q , W K Multiply to get vectors Q and K;

[0199] 3) Perform the SimAM attention mechanism operation on the obtained vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0200] 4) Input the results of the three SimAM attention mechanisms into the block-based multi-head attention mechanism respectively, add the output results of the block-based multi-head attention mechanism and the multi-scale fusion features pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0201] 5) Input the normalized Norm layer output features into the multi-layer perceptron FFN, add the multi-layer perceptron FFN output feature sequence to the normalized Norm layer output features in 4) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0202] 6) Repeat 1) to 5) N times to obtain the improved Transformer encoder output features, 1 <N<10;

[0203] The specific processing process of the block-based multi-head attention mechanism is as follows:

[0204] (1) Change the first row of each SimAM attention mechanism result to the last row, and the first column to the last column in the three SimAM attention mechanism results to obtain the converted three SimAM attention mechanism results;

[0205] (2) The converted three SimAM attention mechanism results are added pixel by pixel with the three SimAM attention mechanism results in (1), and then sent to the multi-layer perceptron FFN for encoding. The multi-layer perceptron FFN outputs the encoded features.

[0206] (3) The encoded features are evenly divided into four blocks, each of which is input into a multi-head attention mechanism. The output features of the four multi-head attention mechanisms are concat-spliced ​​by channel to obtain a feature map as the output features of the divided multi-head attention mechanism.

[0207] The base plate anti-loosening wire targets are of different sizes and positions. The use of multi-scale feature fusion can more effectively detect base plate anti-loosening wire targets of different scales. This patent uses the 8-fold, 16-fold, and 32-fold down-sampling feature maps F2, F3, and F4 of the improved PVT feature extraction backbone network for feature fusion to construct a multi-scale feature fusion encoder. The multi-scale feature fusion encoder is as follows Figure 2 As shown in c. Feature maps F2, F3, and F4 are transformed into feature maps P2, P3, and P4 after 1×1 two-dimensional convolution, and then BiFPN feature pyramid feature fusion is performed to obtain multi-scale fusion features and complete multi-scale feature fusion. After that, the improved encoder is used to obtain the encoding features of the category and position.

[0208] The feature fusion method in the multi-scale feature fusion encoder is different from that in the BiFPN feature pyramid. The improved Transformer encoder encoding method is used for feature map fusion. Due to the global feature extraction capability of the Transformer encoder, the Transformer encoder is used instead of the convolution in BiFPN for feature fusion, which can more fully fuse features of different scales and thus improve network accuracy. For example, for the fusion of P3 and P4, P4 is first upsampled to the size of P3, and then Patch Embedding is performed to convert it into a two-dimensional feature sequence, which is added pixel by pixel with the position code, and the two feature sequences are spliced ​​together according to the channel dimension. After feature encoding by the improved Transformer encoder, normalization and FFN multi-layer perceptron encoding are performed, and then the feature sequence is reshaped into a feature map. The width and height of the feature map are the same as those of P3. Finally, a 1X1 two-dimensional convolution is performed to obtain the features after the fusion of the two scale feature maps. The encoding structure of the improved Transformer encoder is described in the next paragraph.

[0209] Block multi-head attention mechanism and SimAM attention mechanism

[0210] Figure 2 Where d is the multi-scale feature fusion Transformer encoder and improved Transformer decoder structure used in the network, which is similar to the Transformer encoder and Transformer decoder structure in the DETR network, except that the input encoder features are changed to multi-scale fusion features. The multi-head attention mechanism in the encoder and decoder is changed to a block-based multi-head attention mechanism, which reduces the amount of calculation while ensuring the feature sequence encoding effect. The SimAM attention mechanism is introduced to improve the network detection accuracy. The Transformer encoder that performs feature fusion in the multi-scale feature fusion encoder also adopts the same improvement measures. The original multi-head attention mechanism can be expressed by formula (1). Where X represents the input multi-scale fusion feature sequence, W Q , W K , W V represents the weight matrix, Attention represents the Transformer attention mechanism, and softmax is the activation function. is the normalized weight, MultiHead represents the Transformer multi-head attention mechanism, Concat represents the sequence feature concatenation operation, head i i∈(1,…,h) indicates that the multi-head attention mechanism is divided into several heads, that is, the Transformer attention mechanism is divided into several sections. is the weight matrix that divides Q, K, and V into segments.

[0211] The original Transformer attention mechanism, Attention(Q,K,V), requires matrix multiplication. The size of the input multi-scale fusion feature sequence is very large, and the amount of calculation is large when performing matrix multiplication. Therefore, the Transformer multi-head attention mechanism divides Q, K, and V into segments for the Transformer attention mechanism, and then concatenates the obtained encoding results to reduce the amount of calculation, but its amount of calculation is still not negligible. Therefore, this patent adopts a block multi-head attention mechanism to further reduce the amount of calculation, and the process can be described as follows: Figure 3 As shown in the figure, ① the first row of the multi-scale fusion feature is changed to the last row, and the first column is changed to the last column, and the original multi-scale fusion feature is added pixel by pixel, and then sent to the FFN multi-layer perceptron for encoding. ② The encoded features are evenly divided into four blocks and the multi-head attention mechanism is performed separately. ③ The encoded features are concat spliced ​​to obtain the final encoded multi-scale fusion features. The reason for performing step ① instead of directly performing steps ②③ is that through the multi-scale fusion feature row and column transformation, the features in different blocks can be fused in step ③. If step ① is not performed, the features will only be fused in the four blocks divided in step ②. Step ① strengthens the interaction between information and improves the network accuracy.

[0212]

[0213] At the same time, this patent performs SimAM attention mechanism operation on Q, K, V of the multi-head attention mechanism of the input encoder block and the target query vector of the decoder to make it focus on the key features. SimAM attention mechanism is an attention mechanism proposed by the YOLOV8 network. The specific implementation method is as shown in formula (2). Where X is the input. σ 2 is the variance of the input. μ is the mean of the input. ε is a small constant to avoid the denominator being 0. 2 , μ, ε are used to calculate two parameters α and β, and then the SimAM attention machine result is calculated. SimAM is a concise and effective attention mechanism that significantly improves the feature representation ability of the network through adaptive weighting without significantly increasing the computational overhead.

[0214]

[0215] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0216] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 8 in that in step 223, the multi-scale feature fusion Transformer encoder output feature and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs the feature; the specific process is:

[0217] 1) Initialize the target query vector, which is respectively related to the weight W Q , W K , W V Multiply to get vectors Q, K, V;

[0218] Perform the SimAM attention mechanism operation on vectors Q, K, and V respectively to obtain three SimAM attention machine results;

[0219] The results of the three SimAM attention machines are input into the block-based multi-head attention mechanism respectively, and the block-based multi-head attention mechanism outputs the results;

[0220] 2) Add the output of the multi-head attention mechanism of the block to the target query vector in 1) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0221] 3) Add the normalized Norm layer output features from 2) to the target query vector to obtain a new vector Q;

[0222] Add the improved Transformer encoder output features to a positional encoding of the same size pixel by pixel to obtain a new vector K;

[0223] The improved Transformer encoder output features are used as the new vector V;

[0224] 4) Input the new vector Q, new vector K, and new vector V obtained in 3) into the block-based multi-head attention mechanism, and the block-based multi-head attention mechanism outputs the result;

[0225] 5) Add the output results of the multi-head attention mechanism of the block in 4) and the output results of the normalized Norm layer in 2) pixel by pixel, and input the added feature sequence into the normalized Norm layer;

[0226] 6) Input the normalized Norm layer output features of 5) into the FFN multilayer perceptron, add the FFN multilayer perceptron output feature sequence and the normalized Norm layer output features in 5) pixel by pixel, input the added feature sequence into the normalized Norm layer, and normalize the Norm layer output features;

[0227] 7) Repeat 1) to 6) M times to obtain the improved Transformer decoder output features, 1 <M<10。

[0228] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0229] Specific implementation method ten: This implementation method is different from any one of specific implementation methods one to nine in that, in step three, the railway train bottom plate anti-loose wire breakage fault is detected based on the trained bottom plate anti-loose wire breakage fault detection network; the specific process is:

[0230] Obtain the train passing image after the high-definition imaging device, and send the bottom camera image to the trained detection network. If the detection network detects that the bottom plate anti-loose wire is broken, it proves that the image has a fault, uploads the fault message, and continues to detect the next image. If the detection network does not detect that the bottom plate anti-loose wire is broken, it proves that the image does not have a fault, and no operation is performed, and continues to detect the next image.

[0231] The inspection personnel will further process the fault location based on the uploaded fault message and the principle of manual priori.

[0232] The present invention sets up high-definition imaging equipment around the railway, and obtains the passing train image after the railway train passes. The bottom plate image taken by the bottom camera is sent to the bottom plate anti-loose wire breakage detection network to detect the bottom plate anti-loose wire breakage fault. If the bottom plate anti-loose wire breakage fault is detected in the image, an alarm message is uploaded, and the railway train inspection personnel handle the fault part accordingly according to the alarm message.

[0233] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0234] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting the breakage of anti-loosening wire on the bottom plate of a railway motor vehicle, characterized in that: The specific process of the method is: Step 1: Collect images and build a data set; Step 2: construct a bottom plate anti-loose wire break fault detection network, train the bottom plate anti-loose wire break fault detection network, and obtain a trained bottom plate anti-loose wire break fault detection network; The specific process is: Step 21. The bottom plate anti-loose wire breakage fault detection network includes: an improved PVT feature extraction backbone network, a multi-scale feature fusion Transformer encoder, an improved Transformer decoder structure, and a detection head; The specific process is: The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4; Each stage includes: Patch Embedding, position encoding, branch block, concat, normalization Norm layer, FFN multi-layer perceptron; The branch block includes branch a and branch b; Branch a includes: PVT Transformer Encoder encoding, FFN multi-layer perceptron, one-dimensional convolution Conv1D, the first Relu activation function and SSM module; Branch b includes: two-dimensional convolution, second Relu activation function, two-dimensional convolution, third Relu activation function, normalized Norm layer, FFN multi-layer perceptron; The multi-scale feature fusion Transformer encoder includes: 1×1 two-dimensional convolution, concat splicing, and improved Transformer encoder; The improved Transformer decoder structure includes: block-based multi-head attention mechanism, normalized Norm layer, and multi-layer perceptron FFN; The detection head is a multi-layer perceptron FFN; Step 22: training a bottom plate anti-loose wire break fault detection network to obtain a trained bottom plate anti-loose wire break fault detection network; Step 3: Detect the railway train bottom plate anti-loose wire breakage fault based on the trained bottom plate anti-loose wire breakage fault detection network.

2. According to claim 1, a method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire, characterized in that: In step 1, images are collected and a data set is established; the specific process is as follows: Step 1: Set up high-definition imaging equipment around the railway to obtain images of passing trains and collect bottom plate images taken by bottom cameras; Step 12: amplify the collected base plate image to obtain an amplified base plate image; Step 1-3 Mark the image in the amplified bottom plate image where the bottom plate anti-loosening wire is normal as a bottom plate anti-loosening wire normal class, and generate a marking file; Mark the image of the bottom plate anti-loose wire broken in the amplified bottom plate image as the bottom plate anti-loose wire broken class, and generate a marking file; Step 14: Photoshop the normal image of the bottom plate anti-loose wire into an image of the bottom plate anti-loose wire broken, mark it as the bottom plate anti-loose wire broken class, and generate a marking file; The tag file records the image's name, size, path, location of the anti-loosening wire, type of the anti-loosening wire, and other information; size is the size of the image, path is the path to the image.

3. According to claim 2, a method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire, characterized in that: In the step 22, the bottom plate anti-loose wire break fault detection network is trained to obtain a trained bottom plate anti-loose wire break fault detection network; the specific process is: Step 221: input the image in the data set established in step 1 into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features; Step 222: input the multi-scale features output by the improved PVT feature extraction backbone network into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; Step 223: The multi-scale feature fusion Transformer encoder output feature and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs the feature; Step 224: Input the output features of the improved Transformer decoder into the detection head, and the detection head outputs the output target category and target position; Step 225: Adopt the Adam optimization method, set the initial learning rate to 0.001, and repeat steps 221 to 225 until convergence to obtain a trained bottom plate anti-loose wire break fault detection network.

4. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire according to claim 3, characterized in that: In the step 221, the data set established in step 1 is input into the improved PVT feature extraction backbone network, and the improved PVT feature extraction backbone network outputs multi-scale features; The specific process is: The improved PVT feature extraction backbone network includes stage 1, stage 2, stage 3, and stage 4; The input image F0 of size H0×W0×C0 is input to stage 1, stage 1 outputs feature map F1 which is input to stage 2, stage 2 outputs feature map F2 which is input to stage 3, stage 3 outputs feature map F3 which is input to stage 4, and stage 4 outputs feature map F4; The specific processing process of each stage i in the stage 1, stage 2, stage 3, and stage 4 is as follows: At stage i, set the size of stage i-1 to H i-1 ×W i-1 ×C i-1 The feature map F i-1 Divide into P i ×P i The size of the block is , and each block is processed by Patch Embedding. The sequence after Patch Embedding is reshaped to obtain a size of The characteristic sequence of size Add a position code of the same size to each pixel in the feature sequence to obtain the feature sequence A; i≥1； The feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output the feature sequence encoding results. The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel, and then pass through the normalization Norm layer and FFN multi-layer perceptron in turn. The output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; Feature map F i Size Among them, H i-1 is the feature map F i-1 High, W i-1 is the feature map F i-1 Width, C i-1 is the feature map F i-1 The number of channels, P i ×P i To transform the feature map F i-1 The size of the divided blocks.

5. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening according to claim 4, characterized in that: The feature sequence A is input into branch a and branch b respectively, and branch a and branch b respectively output feature sequence encoding results, and the feature sequence encoding results output by branch a and branch b are concat spliced ​​according to channels. After splicing, they are sequentially passed through the normalization Norm layer and the FFN multi-layer perceptron, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is: The specific processing process of branch a is: The feature sequence A is encoded by PVTTransformer Encoder for global feature extraction, and the size of the encoding result is The size is The encoding result passes through a FFN multi-layer perceptron, which outputs a feature sequence Output feature sequence of FFN multi-layer perceptron After a one-dimensional convolution Conv1D, the first Relu activation function and the SSM module, the SSM module outputs a feature sequence as the feature sequence encoding result output by branch a, with a size of The specific processing process of branch b is as follows: The feature sequence A is reshaped to obtain a size of H i-1 ×W i-1 ×C i-1 The feature map of Set the size to H i-1 ×W i-1 ×C i-1 The feature map of is sequentially subjected to two-dimensional convolution, the second Relu activation function, two-dimensional convolution, and the third Relu activation function. The third Relu activation function outputs the feature map; the output feature map is deformed and reshaped to obtain a size of The characteristic sequence of For size The feature sequence of is input into the normalized Norm layer and the FFN multilayer perceptron in turn. The FFN multilayer perceptron outputs the feature sequence as the feature sequence encoding result output by branch b, with a size of The feature sequence encoding results output by branch a and branch b are concat spliced ​​by channel, and then passed through the normalization Norm layer and the FFN multi-layer perceptron in sequence, and the output features of the FFN multi-layer perceptron are deformed and reshaped to obtain the feature map F i ; The specific process is: The feature sequence encoding results obtained by branch a and branch b are concatenated by channel to obtain a size of The feature sequence of size The feature sequence is input into the normalized Norm layer and the FFN multilayer perceptron in turn, and the FFN multilayer perceptron outputs the coded feature sequence; the FFN multilayer perceptron output coded feature sequence is deformed and reshaped to obtain The feature map F i .

6. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire according to claim 5, characterized in that: In the step 222, the multi-scale features output by the improved PVT feature extraction backbone network are input into the multi-scale feature fusion Transformer encoder, and the multi-scale feature fusion Transformer encoder outputs features; the specific process is: The feature map F2 undergoes a 1×1 two-dimensional convolution to generate the feature map P2; Feature map F3 undergoes 1×1 two-dimensional convolution to generate feature map P3; Feature map F4 undergoes 1×1 two-dimensional convolution to generate feature map P4; The feature map P3 and the feature map P4 are fused to obtain the feature map P5; The feature map P2 and the feature map P5 are fused to obtain the feature map P6; The feature map P3, the feature map P5 and the feature map P6 are fused to obtain the feature map P7; The feature map P4 and the feature map P7 are fused to obtain the feature map P8; Concatenate feature maps P6, P7, and P8 by channel to obtain feature map P9; The feature map P9 is input into a 1×1 two-dimensional convolution, and the 1×1 two-dimensional convolution outputs a multi-scale fusion feature sequence; The multi-scale fusion feature sequence is input into the improved Transformer encoder, and the improved Transformer encoder outputs the encoded features, which are used as the multi-scale feature fusion Transformer encoder output features.

7. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire according to claim 6, characterized in that: The feature map P3 and the feature map P4 are fused to obtain the feature map P5; the specific process is: First, upsample P4 to the size of P3; P4, which is upsampled to the same size as P3, is processed by PatchEmbedding and converted into a two-dimensional feature sequence corresponding to P4; Perform Patch Embedding on P3 and convert it into a two-dimensional feature sequence corresponding to P3; Add the two-dimensional feature sequence corresponding to P4 to a position code of the same size pixel by pixel to obtain the feature sequence B after adding the position code; Add the two-dimensional feature sequence corresponding to P3 to a position code of the same size pixel by pixel to obtain the feature sequence C after adding the position code; The feature sequence B and the feature sequence C are concatenated together according to the channel dimension. The concatenated feature sequence is passed through an improved Transformer encoder, and the improved Transformer encoder outputs the encoded feature D. The encoded feature sequence D output by the improved Transformer encoder is sequentially input into the normalized Norm layer and the FFN multi-layer perceptron, and the FFN multi-layer perceptron outputs the feature sequence E; The feature sequence E output by the FFN multi-layer perceptron is reshaped to obtain a feature map whose width and height are the same as those of P3. A 1×1 two-dimensional convolution is performed on the feature map to obtain the feature map P5.

8. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire according to claim 7, characterized in that: The specific processing process of the improved Transformer encoder is as follows: 1) Combine multi-scale fusion features with weights W V Multiply to get vector V; 2) Add a position code of the same size pixel by pixel in the multi-scale fusion feature; The multi-scale fusion features after adding position encoding are respectively combined with the weights W Q , W K Multiply to get vectors Q and K; 3) Perform the SimAM attention mechanism operation on the obtained vectors Q, K, and V respectively to obtain three SimAM attention machine results; 4) Input the results of the three SimAM attention mechanisms into the block-based multi-head attention mechanism respectively, add the output results of the block-based multi-head attention mechanism and the multi-scale fusion features pixel by pixel, and input the added feature sequence into the normalized Norm layer; 5) Input the normalized Norm layer output features into the multi-layer perceptron FFN, add the multi-layer perceptron FFN output feature sequence to the normalized Norm layer output features in 4) pixel by pixel, and input the added feature sequence into the normalized Norm layer; 6) Repeat 1) to 5) N times to obtain the improved Transformer encoder output features; The specific processing process of the block-based multi-head attention mechanism is as follows: (1) Change the first row of each SimAM attention mechanism result to the last row, and the first column to the last column in the three SimAM attention mechanism results to obtain the converted three SimAM attention mechanism results; (2) The converted three SimAM attention mechanism results are added pixel by pixel with the three SimAM attention mechanism results in (1), and then sent to the multi-layer perceptron FFN for encoding. The multi-layer perceptron FFN outputs the encoded features. (3) The encoded features are evenly divided into four blocks, each of which is input into a multi-head attention mechanism. The output features of the four multi-head attention mechanisms are concatenated by channel to obtain a feature map as the output features of the divided multi-head attention mechanism.

9. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire according to claim 8, characterized in that: In step 223, the multi-scale feature fusion Transformer encoder output feature and the target query vector are input into the improved Transformer decoder, and the improved Transformer decoder outputs the feature; the specific process is: 1) Initialize the target query vector, which is respectively related to the weight W Q , W K , W V Multiply to get vectors Q, K, V; Perform the SimAM attention mechanism operation on vectors Q, K, and V respectively to obtain three SimAM attention machine results; The results of the three SimAM attention machines are input into the block-based multi-head attention mechanism respectively, and the block-based multi-head attention mechanism outputs the results; 2) Add the output of the multi-head attention mechanism of the block to the target query vector in 1) pixel by pixel, and input the added feature sequence into the normalized Norm layer; 3) Add the normalized Norm layer output features from 2) to the target query vector to obtain a new vector Q; Add the improved Transformer encoder output features to a positional encoding of the same size pixel by pixel to obtain a new vector K; The improved Transformer encoder output features are used as the new vector V; 4) Input the new vector Q, new vector K, and new vector V obtained in 3) into the block-based multi-head attention mechanism, and the block-based multi-head attention mechanism outputs the result; 5) Add the output results of the multi-head attention mechanism of the block in 4) and the output results of the normalized Norm layer in 2) pixel by pixel, and input the added feature sequence into the normalized Norm layer; 6) Input the normalized Norm layer output features of 5) into the FFN multilayer perceptron, add the FFN multilayer perceptron output feature sequence and the normalized Norm layer output features in 5) pixel by pixel, input the added feature sequence into the normalized Norm layer, and normalize the Norm layer output features; 7) Repeat 1) to 6) M times to obtain the improved Transformer decoder output features.

10. A method for detecting a broken wire fault in a railway vehicle floor anti-loosening wire according to claim 9, characterized in that: In the step 3, the railway vehicle bottom plate anti-loose wire breakage fault is detected based on the trained bottom plate anti-loose wire breakage fault detection network; the specific process is: Obtain the train passing image after the high-definition imaging device, and send the bottom camera image to the trained detection network. If the detection network detects that the bottom plate anti-loose wire is broken, it proves that the image has a fault, uploads the fault message, and continues to detect the next image. If the detection network does not detect that the bottom plate anti-loose wire is broken, it proves that the image does not have a fault, and no operation is performed, and continues to detect the next image.

Images