A high-speed rail surface damage detection method based on deep neural network

Abstract

The present invention discloses a method for detecting surface damage of high-speed rails based on a deep neural network, comprising the following steps: collecting rail surface damage images, annotating the collected rail surface damage images, processing the annotated rail surface damage images through data enhancement operations, obtaining a data set, dividing the data set into a training set and a validation set, and constructing a dual-branch image feature extraction network; using the training set and the validation set to train the dual-branch image feature extraction network to obtain a Mamba-based dual-branch image feature extraction network, and inputting the rail surface damage images into the Mamba-based dual-branch image feature extraction network for processing to obtain a segmented image. The present invention adopts a Mamba-based dual-branch image feature extraction network, which has the advantages of global receptive field coverage, linear complexity, and effective selection of key information of different modalities, thereby improving the efficiency and continuity of detection and reducing false alarms and missed detections.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a high-speed railway rail surface damage detection method based on a deep neural network. Background Art

[0002] As an extremely important symbol of transportation modernization, high-speed rail has the characteristics of high speed, good smoothness, and high comfort. However, with the continuous increase in high-speed rail transportation volume, under the action of long-term repeated loads, the rails in service are affected by contact stresses such as wheel-rail extrusion, impact, and wear, as well as material microstructure degradation. Their health and performance quality will continue to deteriorate, and various types of damage will be formed. Once the damage occurs, it is not easy to develop rapidly. If it is not discovered in time and safety measures are not taken, when the damage reaches a certain extent, if no warning is given in time, it will lead to major accidents such as train derailment and overturning, resulting in serious casualties and huge property losses. Therefore, in order to ensure the safe operation of high-speed rail lines, realizing intelligent, accurate, and rapid detection of rail surface damage has become a key issue that needs to be solved in the development of high-speed rail, and it has very important practical application value and research significance.

[0003] At present, the surface damage detection of high-speed rails includes eddy current detection, ultrasonic detection and machine vision detection. However, due to the extremely weak signals generated by the surface damage of high-speed rails, the use of the first two methods mentioned above is very likely to cause missed detection problems. In addition, these two detection methods are easily affected by the external environment, reducing the detection accuracy. The machine vision detection method uses CCD cameras and other equipment to collect images of rail surface damage, and with the help of image processing methods, it can achieve effective detection of damage, and has the advantages of fast speed and intuitive results. In machine vision detection, the visible light image obtained by the CCD camera has high resolution, but is affected by light and weather; the infrared camera has strong anti-interference, but low resolution. However, in the same scene, visible light images and infrared images can highlight different expression content and semantic information, and the information contained is complementary, that is, in the detection of high-speed rails, a single imaging device is limited, and the fusion of visible light and infrared images can complement information, which can achieve a more complete and accurate description and interpretation of the detection scene.

[0004] The current machine vision-based rail surface damage detection method is mainly based on deep learning. The target detection method based on deep learning has the advantages of flexible structure, automatic feature extraction, high detection accuracy, and fast detection speed, and can realize intelligent detection of targets. The target detection method based on deep learning can be divided into three types: image level, region level, and pixel level. Among them, the pixel-level target detection method can accurately obtain the category characteristics and location information of the target, which plays a vital role in evaluating the health status of high-speed rails and making maintenance decisions. Therefore, the pixel-level target detection method is the most effective method for rail surface damage detection.

[0005] In the process of pixel-level target detection, images need to be semantically segmented. Commonly used neural network models include convolutional neural networks and transformer-based networks. The advantage of convolutional neural networks is that they have a small number of parameters and a faster convergence speed, but they are limited by the local receptive field and cannot effectively obtain global information. Transformer-based networks can well obtain global information and long-term dependencies between pixels, and are more suitable for detecting targets with large changes in geometric features, but their computational complexity is large. Recently, in order to solve these problems, selective structured state space models have become increasingly popular due to their global receptive field coverage, dynamic weights, and linear complexity, and state space models have performed well in tasks involving long sequence modeling.

[0006] Therefore, the use of state-space models to fuse infrared and visible light images is of great significance for rail surface damage detection, because it can achieve global receptive field coverage and dynamic weight adjustment with linear complexity, effectively integrate the complementary information of the two modalities, and improve the accuracy and robustness of detection, especially under insufficient lighting or severe weather conditions, thereby reducing missed detection problems, enhancing scene representation and visual perception, and providing strong guarantees for the safe operation of high-speed rail lines. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a high-speed railway rail surface damage detection method based on deep neural network to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a method for detecting surface damage of high-speed rails based on a deep neural network, comprising the following steps:

[0009] Step S1: collecting rail surface damage images;

[0010] Step S2: annotating the rail surface damage image collected in step S1;

[0011] Step S3: Process the annotated rail surface damage image through data enhancement operation to obtain a data set, and divide the data set into a training set and a validation set;

[0012] Step S4: constructing a dual-branch image feature extraction network; using a training set and a validation set to train the dual-branch image feature extraction network to obtain a dual-branch image feature extraction network based on Mamba;

[0013] Step S5: inputting the rail surface damage image in step S1 into a Mamba-based dual-branch image feature extraction network for processing to obtain a segmented image;

[0014] The dual-branch image feature extraction network based on Mamba consists of an encoder and a decoder;

[0015] The encoder is composed of a first visual state space module, a second visual state space module, a third visual state space module, a fourth visual state space module, a first fusion module, a second fusion module, a third fusion module, a fourth fusion module, a fifth visual state space module, a sixth visual state space module, a seventh visual state space module and an eighth visual state space module;

[0016] The decoder consists of an upsampling X4 layer, a first visual state space decoding block, a second visual state space decoding block, a third visual state space decoding block, and block expansion.

[0017] Furthermore, in step S1, the rail surface damage image is collected, and the specific process is as follows:

[0018] A platform for collecting damage images of high-speed rail surface is built. High-speed rail surface damage images are collected through the collection platform to obtain rail surface damage images; rail surface damage images include visible light images and infrared images.

[0019] Further, in step S2, the rail surface damage image collected in step S1 is annotated, and the specific process is as follows:

[0020] The image annotation tool is used to annotate the rail surface damage image collected in step S1 in detail and mark the boundary of the damaged area.

[0021] Further, in step S3, the data set is divided into a training set and a test set. The specific process is as follows:

[0022] The annotated rail surface damage image is processed by data enhancement operation; data enhancement operation includes translation and rotation; translation is to move the pixels of the annotated rail surface damage image by 15 pixels in the vertical direction and 15 pixels in the horizontal direction; rotation is to rotate all pixels on the annotated rail surface damage image by 180 degrees and cut off the annotated rail surface damage image beyond the display area;

[0023] The rail surface damage images that are too blurred or incorrectly annotated are deleted, and the remaining annotated rail surface damage images are used as the data set, which is then divided into a training set and a validation set.

[0024] Further, in step S5, a segmented image is obtained, and the specific process is as follows:

[0025] Input the visible light image and infrared image included in the rail surface damage image in step S1 into an encoder, and the visible light image is sequentially input into a first visual state space module, a second visual state space module, a third visual state space module, and a fourth visual state space module in the encoder;

[0026] The infrared image is sequentially input into the fifth visual state space module, the sixth visual state space module, the seventh visual state space module, and the eighth visual state space module in the encoder;

[0027] The output of the first visual state space module and the output of the fifth visual state space module are input into the first fusion module; the output of the second visual state space module and the output of the sixth visual state space module are input into the second fusion module; the output of the third visual state space module and the output of the seventh visual state space module are input into the third fusion module; the output of the fourth visual state space module and the output of the eighth visual state space module are input into the fourth fusion module; the output of the fourth fusion module is input into block expansion, the output of the block expansion and the output of the third fusion module are input into the third visual state space decoding block, the output of the third visual state space decoding block and the output of the second fusion module are input into the second visual state space decoding block, the output of the second visual state space decoding block and the output of the first fusion module are input into the first visual state space decoding block, the output of the first visual state space decoding block is input into the upsampling X4 layer, and the output of the upsampling X4 layer is used as the final output of the Mamba-based dual-branch image feature extraction network, which is the segmented image.

[0028] Further, the first visual state space module, the second visual state space module, the third visual state space module, the fourth visual state space module, the fifth visual state space module, the sixth visual state space module, the seventh visual state space module and the eighth visual state space module have the same structure;

[0029] The first visual state space module is composed of a first normalization layer, a first linear transformation, a second linear transformation, a first depthwise separable convolution, a first selective scan, a second normalization layer, and a third linear transformation;

[0030] The processing flow of the first visual state space module is as follows: input the visible light image into the first normalization layer, input the output of the first normalization layer into the first linear transformation and the second linear transformation respectively, input the output of the first linear transformation into the first depthwise separable convolution and the first selective scan in sequence; multiply the output of the second linear transformation by the output of the first selective scan and input them into the second normalization layer and the third linear transformation in sequence, and add the output of the third linear transformation to the output of the first normalization layer as the final output of the first visual state space module;

[0031] The processing flow of the fifth visual state space module is as follows: the infrared image is input into the first normalization layer, the output of the first normalization layer is respectively input into the first linear transformation and the second linear transformation, the output of the first linear transformation is sequentially input into the first depthwise separable convolution and the first selective scan; the output of the second linear transformation is multiplied by the output of the first selective scan and sequentially input into the second normalization layer and the third linear transformation, and the output of the third linear transformation is added to the output of the first normalization layer as the final output of the fifth visual state space module.

[0032] Furthermore, the first fusion module, the second fusion module, the third fusion module, and the fourth fusion module have the same structure;

[0033] The first fusion module consists of a fifth linear transformation, a second depth-separable convolution, a second selective scan, a fourth normalization layer, a seventh linear transformation, a sixth linear transformation, a third depth-separable convolution, a fifth normalization layer, an eighth linear transformation, and a ninth linear transformation;

[0034] The processing flow of the first fusion module is: the output of the first visual state space module is sequentially input into the fifth linear transformation, the second depthwise separable convolution, the second selective scanning, the fourth normalization layer, and the seventh linear transformation, and the output of the seventh linear transformation is element-wise added to the input of the fifth linear transformation in the spatial dimension to obtain the first feature after the addition;

[0035] Input the output of the fifth visual state space module into the sixth linear transformation, the third depthwise separable convolution, the second selective scan, the fifth normalization layer, and the eighth linear transformation in sequence, and add the output of the eighth linear transformation and the input of the sixth linear transformation element by element in the spatial dimension to obtain the added second feature;

[0036] The first feature and the second feature are spliced ​​on the channel dimension C, the spliced ​​output is input into the ninth linear transformation, the output of the ninth linear transformation is added to the input of the fifth linear transformation and the input of the sixth linear transformation, and the added output is used as the final output of the first fusion module.

[0037] Further, the first visual state space decoding block, the second visual state space decoding block and the third visual state space decoding block all have the same structure;

[0038] The first visual state space decoding block consists of the visual state space, the third normalization layer, the fourth linear transformation, the convolution layer, the maximum pooling layer, the average pooling layer and the activation function;

[0039] The processing flow of the first visual state space decoding block is as follows: the output of the first fusion module and the output of the second visual state space decoding block are input into the visual state space, the third normalization layer, the fourth linear transformation, and the convolution layer in sequence; the output of the convolution layer is input into the maximum pooling layer and the average pooling layer respectively; the output of the maximum pooling layer and the output of the average pooling layer are added and input into the activation function; the output of the activation function is multiplied by the output of the fourth linear transformation; the multiplied output is added to the output of the visual state space as the final output of the first visual state space decoding block.

[0040] Compared with the existing technology, the present invention has the following beneficial effects:

[0041] (1) The present invention adopts a dual-branch image feature extraction network based on Mamba, which has the advantages of global receptive field coverage, linear complexity, and effective selection of key information of different modalities. By utilizing the powerful representation ability of the dual-branch image feature extraction network, it can better adapt to diverse damage types and complex background environments than traditional methods, reducing false alarms and missed detections.

[0042] (2) The present invention can simultaneously detect multiple types of rail damage, including fatigue cracks and peeling blocks, covering a wide range of damage from minor to severe, avoiding the need to use a combination of multiple single technologies. In addition, the fully automated process reduces human intervention and improves the efficiency and continuity of detection. This not only reduces the burden on staff, but also makes the detection process more standardized and objective. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the present invention.

[0044] Figure 2 This is a Mamba-based dual-branch image feature extraction network framework diagram of the present invention.

[0045] Figure 3 This is a framework diagram of the visual state space module of the present invention.

[0046] Figure 4 This is a block framework diagram of the visual state space decoding of the present invention.

[0047] Figure 5 This is a framework diagram of the fusion module of the present invention. DETAILED DESCRIPTION

[0048] like Figure 1 As shown, the present invention provides a technical solution: a high-speed rail surface damage detection method based on a deep neural network, comprising the following steps:

[0049] Step S1: collecting rail surface damage images;

[0050] Step S2: annotating the rail surface damage image collected in step S1;

[0051] Step S3: Process the annotated rail surface damage image through data enhancement operation to obtain a data set, and divide the data set into a training set and a validation set;

[0052] Step S4: constructing a dual-branch image feature extraction network; using a training set and a validation set to train the dual-branch image feature extraction network to obtain a dual-branch image feature extraction network based on Mamba;

[0053] Step S5: inputting the rail surface damage image in step S1 into a Mamba-based dual-branch image feature extraction network for processing to obtain a segmented image;

[0054] The dual-branch image feature extraction network based on Mamba consists of an encoder and a decoder;

[0055] The encoder is composed of a first visual state space module VSSM (visual state space model), a second visual state space module, a third visual state space module, a fourth visual state space module, a first fusion module, a second fusion module, a third fusion module, a fourth fusion module, a fifth visual state space module, a sixth visual state space module, a seventh visual state space module and an eighth visual state space module;

[0056] The decoder consists of an upsampling X4 layer, a first visual state space decoding block, a second visual state space decoding block, a third visual state space decoding block, and a block expansion patch expend.

[0057] Wherein, in step S1, the rail surface damage image is collected, and the specific process is as follows:

[0058] Build a high-speed rail surface damage image acquisition platform, which includes a CCD camera, an infrared camera, a light source, an acquisition and control module, and a power module;

[0059] The rail surface damage images collected by the acquisition platform are visible light images and infrared images respectively; the rail surface damage images collected are mainly rail contact fatigue cracks and peeling blocks caused by wheel-rail rolling contact.

[0060] Among them, in step S2, the rail surface damage image collected in step S1 is annotated, and the specific process is as follows:

[0061] The image annotation tool is used to annotate the rail surface damage image collected in step S1 in detail to mark the precise boundary of the damaged area.

[0062] In step S3, the data set is divided into a training set and a test set. The specific process is as follows:

[0063] The annotated rail surface damage image is processed by data enhancement operation; data enhancement operation includes translation and rotation; translation is to move the pixels of the annotated rail surface damage image by 15 pixels in the vertical direction and 15 pixels in the horizontal direction; rotation is to rotate all pixels on the annotated rail surface damage image by 180 degrees and cut off the annotated rail surface damage image beyond the display area;

[0064] The rail surface damage images that are too blurred or incorrectly annotated are deleted, and the remaining annotated rail surface damage images are used as the data set, which is then divided into a training set and a validation set with a ratio of 8:2.

[0065] like Figure 2 As shown, in step S5, the segmented image is obtained, and the specific process is:

[0066] Input the visible light image and the infrared image included in the rail surface damage image in step S1 into the encoder, and the visible light image is sequentially input into the first visual state space module, the second visual state space module, the third visual state space module, and the fourth visual state space module in the encoder, and the first visual state space module, the second visual state space module, the third visual state space module, and the fourth visual state space module respectively perform feature extraction on the visible light image to obtain the first four groups of feature maps and the number of channels;

[0067] The infrared image is sequentially input into the fifth visual state space module, the sixth visual state space module, the seventh visual state space module, and the eighth visual state space module in the encoder, and the fifth visual state space module, the sixth visual state space module, the seventh visual state space module, and the eighth visual state space module respectively extract features of the infrared image to obtain the second four groups of feature maps and the number of channels;

[0068] The first four groups of feature maps and the number of channels and the second four groups of feature maps and the number of channels are both derived from the first group of feature maps with a size of ( × ), the number of channels in the first group is C1, and the size of the feature map in the second group is ( × ), the second group of channels is C2, and the third group of feature map size is ( × ), the third group of channels is C3 and the fourth group of feature map size is ( × ), the fourth group of channels consists of C4;

[0069] The output of the first visual state space module is the first set of feature maps with a size of ( × ), the first set of channels is C1 and the output of the fifth visual state space module is the first set of feature maps with a size of ( × ), the first group of channels is C1 and input into the first fusion module;

[0070] The output of the second visual state space module is the second set of feature maps with a size of ( × ), the second set of channels is C2 and the output of the sixth visual state space module, the second set of feature map size is ( × ), the second group of channels is C2 and input into the second fusion module;

[0071] The output of the third visual state space module is the third set of feature maps with a size of ( × ), the third group of channels is C3 and the output of the seventh visual state space module, the third group of feature map size is ( × ), the third group of channels is C3 and input into the third fusion module;

[0072] The output of the fourth visual state space module is the fourth set of feature maps with a size of ( × ), the fourth group of channels is C4 and the output of the eighth visual state space module, the fourth group of feature map size is ( × ), the fourth group of channels is C4 and is input into the fourth fusion module; the output of the fourth fusion module is input into the block expansion, the output of the block expansion and the output of the third fusion module are input into the third visual state space decoding block, the output of the third visual state space decoding block and the output of the second fusion module are input into the second visual state space decoding block, the output of the second visual state space decoding block and the output of the first fusion module are input into the first visual state space decoding block, the output of the first visual state space decoding block is input into the upsampling X4 layer, and the output of the upsampling X4 layer is used as the final output of the Mamba-based dual-branch image feature extraction network, which is the segmented image.

[0073] like Figure 3 As shown, the first visual state space module, the second visual state space module, the third visual state space module, the fourth visual state space module, the fifth visual state space module, the sixth visual state space module, the seventh visual state space module and the eighth visual state space module have the same structure;

[0074] The first visual state space module is composed of a first normalization layer, a first linear transformation, a second linear transformation, a first depthwise separable convolution, a first selective scan, a second normalization layer, and a third linear transformation;

[0075] The processing flow of the first visual state space module is as follows: input the visible light image into the first normalization layer, input the output of the first normalization layer into the first linear transformation and the second linear transformation respectively, input the output of the first linear transformation into the first depthwise separable convolution and the first selective scan in sequence; multiply the output of the second linear transformation by the output of the first selective scan and input them into the second normalization layer and the third linear transformation in sequence, and add the output of the third linear transformation to the output of the first normalization layer as the final output of the first visual state space module, which is the first set of feature maps with a size of ( × ), the number of channels in the first group is C1;

[0076] The processing flow of the fifth visual state space module is as follows: the infrared image is input into the first normalization layer, the output of the first normalization layer is respectively input into the first linear transformation and the second linear transformation, and the output of the first linear transformation is sequentially input into the first depthwise separable convolution and the first selective scan; the output of the second linear transformation is multiplied by the output of the first selective scan and sequentially input into the second normalization layer and the third linear transformation, and the output of the third linear transformation is added to the output of the first normalization layer as the final output of the fifth visual state space module, which is the first set of feature maps with a size of ( × ), the number of channels in the first group is C1.

[0077] like Figure 4 As shown, the first visual state space decoding block, the second visual state space decoding block and the third visual state space decoding block all have the same structure;

[0078] The first visual state space decoding block consists of the visual state space, the third normalization layer, the fourth linear transformation, the convolution layer, the maximum pooling layer, the average pooling layer and the activation function;

[0079] The processing flow of the first visual state space decoding block is as follows: the output of the first fusion module and the output of the second visual state space decoding block are input into the visual state space, the third normalization layer, the fourth linear transformation, and the convolution layer in sequence; the output of the convolution layer is input into the maximum pooling layer and the average pooling layer respectively; the output of the maximum pooling layer and the output of the average pooling layer are added and input into the activation function; the output of the activation function is multiplied by the output of the fourth linear transformation; the multiplied output is added to the output of the visual state space as the final output of the first visual state space decoding block.

[0080] like Figure 5 As shown, the first fusion module, the second fusion module, the third fusion module, and the fourth fusion module have the same structure;

[0081] The first fusion module consists of a fifth linear transformation, a second depth-separable convolution, a second selective scan, a fourth normalization layer, a seventh linear transformation, a sixth linear transformation, a third depth-separable convolution, a fifth normalization layer, an eighth linear transformation, and a ninth linear transformation;

[0082] The processing flow of the first fusion module is: the output first set of feature maps of the first visual state space module is ( × ), the first group of channels is C1, and the fifth linear transformation, the second depth-wise separable convolution, the second selective scan, the fourth normalization layer, and the seventh linear transformation are sequentially input, and the output of the seventh linear transformation is added element by element to the input of the fifth linear transformation in the spatial dimension to obtain the first feature after the addition;

[0083] The output first set of feature maps of the fifth visual state space module is sized as ( × ), the first group of channels is C1, and the sixth linear transformation, the third depth-wise separable convolution, the second selective scan, the fifth normalization layer, and the eighth linear transformation are sequentially input, and the output of the eighth linear transformation and the input of the sixth linear transformation are added element by element in the spatial dimension to obtain the second feature after the addition;

[0084] The first feature and the second feature are spliced ​​on the channel dimension C, the spliced ​​output is input into the ninth linear transformation, the output of the ninth linear transformation is added to the input of the fifth linear transformation and the input of the sixth linear transformation, and the added output is used as the final output of the first fusion module.

[0085] From the precise collection and intelligent annotation of rail surface damage images to efficient preprocessing, and then to the careful design and training of the Mamba-based dual-branch image feature extraction network, the rapid and accurate identification of rail surface damage images is finally achieved. It is worth noting that although the above embodiments specifically show one or more preferred implementation forms of the present invention, the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field, based on the understanding of the core ideas and technical solutions of the present invention, makes various equivalent changes or improvements thereto. As long as these changes or improvements do not deviate from the essence and innovation of the technical solutions disclosed by the present invention, and do not exceed the protection scope disclosed by the present invention, they should be deemed to fall within the protection scope of the present invention. This includes but is not limited to the use of different image acquisition devices, data enhancement strategies, subtle adjustments to the network architecture, optimization of training strategies, or surface damage detection applied to other similar scenarios, etc., all of which are protected by the present invention.

[0086] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-speed rail surface damage detection method based on deep neural network, characterized in that: The following steps are involved: Step S1: collecting rail surface damage images; Step S2: annotating the rail surface damage image collected in step S1; Step S3: Process the annotated rail surface damage image through data enhancement operation to obtain a data set, and divide the data set into a training set and a validation set; Step S4: constructing a dual-branch image feature extraction network; using a training set and a validation set to train the dual-branch image feature extraction network to obtain a dual-branch image feature extraction network based on Mamba; Step S5: inputting the rail surface damage image in step S1 into a Mamba-based dual-branch image feature extraction network for processing to obtain a segmented image; The dual-branch image feature extraction network based on Mamba consists of an encoder and a decoder; The encoder is composed of a first visual state space module, a second visual state space module, a third visual state space module, a fourth visual state space module, a first fusion module, a second fusion module, a third fusion module, a fourth fusion module, a fifth visual state space module, a sixth visual state space module, a seventh visual state space module and an eighth visual state space module; The first visual state space module, the second visual state space module, the third visual state space module, the fourth visual state space module, the fifth visual state space module, the sixth visual state space module, the seventh visual state space module and the eighth visual state space module have the same structure; The first visual state space module is composed of a first normalization layer, a first linear transformation, a second linear transformation, a first depthwise separable convolution, a first selective scan, a second normalization layer, and a third linear transformation; The first fusion module, the second fusion module, the third fusion module, and the fourth fusion module have the same structure; The first fusion module consists of a fifth linear transformation, a second depth-separable convolution, a second selective scan, a fourth normalization layer, a seventh linear transformation, a sixth linear transformation, a third depth-separable convolution, a fifth normalization layer, an eighth linear transformation, and a ninth linear transformation; The decoder consists of an upsampling X4 layer, a first visual state space decoding block, a second visual state space decoding block, a third visual state space decoding block, and block expansion; The first visual state space decoding block, the second visual state space decoding block and the third visual state space decoding block all have the same structure; The first visual state space decoding block consists of the visual state space, the third normalization layer, the fourth linear transformation, the convolution layer, the maximum pooling layer, the average pooling layer and the activation function.

2. According to claim 1, a method for detecting surface damage of high-speed rail based on deep neural network is characterized in that: In step S1, the rail surface damage image is collected, and the specific process is as follows: A platform for collecting damage images of high-speed rail surface is built. High-speed rail surface damage images are collected through the collection platform to obtain rail surface damage images; rail surface damage images include visible light images and infrared images.

3. The method for detecting surface damage of high-speed rails based on a deep neural network according to claim 2 is characterized in that: In step S2, the rail surface damage image collected in step S1 is annotated. The specific process is as follows: The image annotation tool is used to annotate the rail surface damage image collected in step S1 in detail and mark the boundary of the damaged area.

4. The method for detecting surface damage of high-speed rails based on a deep neural network according to claim 3 is characterized in that: In step S3, the data set is divided into a training set and a test set. The specific process is as follows: The annotated rail surface damage image is processed by data enhancement operation; the data enhancement operation includes translation and rotation; the translation is to move the pixels of the annotated rail surface damage image by 15 pixels in the vertical direction and 15 pixels in the horizontal direction; Rotation is to rotate all pixels on the annotated rail surface damage image by 180 degrees, and cut off the annotated rail surface damage image beyond the display area; The rail surface damage images that are too blurred or incorrectly annotated are deleted, and the remaining annotated rail surface damage images are used as the data set, which is then divided into a training set and a validation set.

5. The method for detecting surface damage of high-speed rails based on a deep neural network according to claim 4 is characterized in that: In step S5, the segmented image is obtained, and the specific process is as follows: Input the visible light image and infrared image included in the rail surface damage image in step S1 into an encoder, and the visible light image is sequentially input into a first visual state space module, a second visual state space module, a third visual state space module, and a fourth visual state space module in the encoder; The infrared image is sequentially input into the fifth visual state space module, the sixth visual state space module, the seventh visual state space module, and the eighth visual state space module in the encoder; The output of the first visual state space module and the output of the fifth visual state space module are input into the first fusion module; the output of the second visual state space module and the output of the sixth visual state space module are input into the second fusion module; the output of the third visual state space module and the output of the seventh visual state space module are input into the third fusion module; the output of the fourth visual state space module and the output of the eighth visual state space module are input into the fourth fusion module; the output of the fourth fusion module is input into block expansion, the output of the block expansion and the output of the third fusion module are input into the third visual state space decoding block, the output of the third visual state space decoding block and the output of the second fusion module are input into the second visual state space decoding block, the output of the second visual state space decoding block and the output of the first fusion module are input into the first visual state space decoding block, the output of the first visual state space decoding block is input into the upsampling X4 layer, and the output of the upsampling X4 layer is used as the final output of the Mamba-based dual-branch image feature extraction network, which is the segmented image.

6. The method for detecting surface damage of high-speed rails based on a deep neural network according to claim 5, characterized in that: The processing flow of the first visual state space module is as follows: inputting the visible light image into the first normalization layer, inputting the output of the first normalization layer into the first linear transformation and the second linear transformation respectively, and inputting the output of the first linear transformation into the first depthwise separable convolution and the first selective scanning in sequence; The output of the second linear transformation is multiplied by the output of the first selective scan, and the two are sequentially input into the second normalization layer and the third linear transformation, and the output of the third linear transformation is added to the output of the first normalization layer as the final output of the first visual state space module; The fifth visual state space module processing flow is: input the infrared image into the first normalization layer, input the output of the first normalization layer into the first linear transformation and the second linear transformation respectively, and input the output of the first linear transformation into the first depthwise separable convolution and the first selective scanning in sequence; The output of the second linear transformation is multiplied by the output of the first selective scan and is sequentially input into the second normalization layer and the third linear transformation. The output of the third linear transformation is added to the output of the first normalization layer as the final output of the fifth visual state space module.

7. The method for detecting surface damage of high-speed rails based on a deep neural network according to claim 6 is characterized in that: The processing flow of the first fusion module is: the output of the first visual state space module is sequentially input into the fifth linear transformation, the second depthwise separable convolution, the second selective scanning, the fourth normalization layer, and the seventh linear transformation, and the output of the seventh linear transformation is element-wise added to the input of the fifth linear transformation in the spatial dimension to obtain the first feature after the addition; Input the output of the fifth visual state space module into the sixth linear transformation, the third depthwise separable convolution, the second selective scan, the fifth normalization layer, and the eighth linear transformation in sequence, and add the output of the eighth linear transformation and the input of the sixth linear transformation element by element in the spatial dimension to obtain the added second feature; The first feature and the second feature are spliced ​​on the channel dimension C, the spliced ​​output is input into the ninth linear transformation, the output of the ninth linear transformation is added to the input of the fifth linear transformation and the input of the sixth linear transformation, and the added output is used as the final output of the first fusion module.

8. The method for detecting surface damage of high-speed rails based on a deep neural network according to claim 7 is characterized in that: The processing flow of the first visual state space decoding block is as follows: the output of the first fusion module and the output of the second visual state space decoding block are input into the visual state space, the third normalization layer, the fourth linear transformation, and the convolution layer in sequence; the output of the convolution layer is input into the maximum pooling layer and the average pooling layer respectively; the output of the maximum pooling layer and the output of the average pooling layer are added and input into the activation function; the output of the activation function is multiplied by the output of the fourth linear transformation; the multiplied output is added to the output of the visual state space as the final output of the first visual state space decoding block.

Images