Track surface defect detection method and device based on visual Mamba, storage medium, and electronic device

Abstract

The present invention provides a method and device for detecting rail surface defects based on Visual Mamba, a storage medium, and an electronic device. The method comprises: obtaining a rail surface image to be inspected; segmenting the rail surface image into multiple non-overlapping patch images, and creating multiple embedded images based on the patch images, wherein each embedded image corresponds to a resolution; performing omnidirectional continuous scanning of the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images at all resolutions; fusing the multiple embedded images and the multi-level feature information to obtain fused features; inputting the fused features into a target rail surface defect detection network (RVMNet) model based on Visual Mamba, and outputting the defect location and defect category of the rail surface defects in the rail surface image. This method solves the technical problem in related arts of difficulty in distinguishing defects in rail surface images that are similar to the background.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for detecting track surface defects based on Visual Mamba, a storage medium, and an electronic device. Background Art

[0002] In related technologies, as urbanization progresses, the mileage, speed, and density of railway operations continue to increase, making railway safety increasingly important. Its health and stability have a profound impact on public safety and property. Due to the complex operating environment of railway infrastructure, track defects are inevitable during service, and these defects accumulate and expand over time.

[0003] Track defects are generally categorized as internal and external. Over long periods of train operation, the extrusion and excessive friction exerted on the track gradually increase, leading to the development of internal defects. Today, mature track production processes have significantly reduced internal defects. However, due to increased friction and impact forces between train wheels and the track surface, as well as a volatile external environment, track surface defects pose a threat to rail transportation. Common track surface defects include cracks, wear, and deformation. If not promptly detected and repaired, surface defects will continue to expand over time.

[0004] With the rapid development of artificial intelligence technology, target detection algorithms based on deep learning have also been widely used in the field of rail surface defect detection. Although current deep learning technology has achieved certain results in the task of rail surface defect detection, these methods still have some problems. The main ones are: (1) The light at the track inspection site is low, which leads to uneven brightness of the collected rail surface defect images and the similarity between the rail surface defects and the background, which easily leads to missed detection and false detection. The existing methods have low detection performance for rail surface defects. (2) Due to the huge passenger volume of rail transit, the time left for rail surface defect detection is usually very short. Through observation of the rail surface images, the applicant found that the rail surface defects are discretely distributed, and most rail surfaces are in a normal state. The use of complex deep learning networks will result in a waste of computing resources and time.

[0005] For the above-mentioned problems existing in related technologies, no efficient and accurate solutions have been found yet. Summary of the Invention

[0006] The present invention provides a track surface defect detection method and device based on Visual Mamba, a storage medium, and an electronic device to solve technical problems in related technologies.

[0007] According to one embodiment of the present invention, a rail surface defect detection method based on Visual Mamba is provided, comprising: acquiring a rail surface image to be detected; segmenting the rail surface image into multiple non-overlapping patch images, and creating multiple embedded images based on the patch images, wherein each embedded image corresponds to a resolution; performing omnidirectional continuous scanning on the multiple embedded images, and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; fusing the multiple embedded images and the multi-level feature information to obtain fused features; inputting the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0008] Optionally, the track surface image is segmented into multiple non-overlapping patch maps, and multiple embedded images are created based on the patch maps, including: segmenting the track surface image into multiple non-overlapping two-dimensional feature maps of the same scale to obtain patch maps; encoding the two-dimensional feature maps in the patch maps into multiple embedded images through linear projection.

[0009] Optionally, encoding the two-dimensional feature map in the patch image into multiple embedded images through linear projection includes: encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution through linear projection; downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; downsampling the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and doubling the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0010] Optionally, performing omnidirectional continuous scanning on the multiple embedded images and extracting global feature information includes: for each embedded image, performing omnidirectional continuous scanning on the embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; extracting a feature map of each of the scanning sequences; merging the feature maps of the multiple scanning sequences to obtain global feature information; and performing fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0011] Optionally, the embedded image is omnidirectionally and continuously scanned according to multiple scanning trajectories to obtain multiple scanning sequences, including: starting from the first row and first column of the embedded image, performing omnidirectional continuous scanning row by row to obtain a first scanning sequence; starting from the first row and first column of the embedded image, performing omnidirectional continuous scanning column by column to obtain a second scanning sequence; starting from the last row and last column of the embedded image, performing omnidirectional continuous scanning row by row to obtain a third scanning sequence; starting from the last row and last column of the embedded image, performing omnidirectional continuous scanning column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0012] Optionally, the multiple embedded images and the multi-level feature information are fused to obtain fused features, including: upsampling the multi-level feature information to obtain hierarchical features with the same scale as the embedded images; for each embedded image, pixel-by-pixel splicing the embedded image and the hierarchical features with the same scale to obtain spliced ​​features; and fusing the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0013] Optionally, before inputting the fusion feature into the target RVMNet model, the method further includes: collecting a set of track surface defect images; performing data enhancement on the track surface defect image set to obtain a sample set; dividing the sample set into a training set, a validation set and a test set; using the training set to train the initial RVMNet model, and calculating the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; adjusting the parameter value of the initial RVMNet model based on the classification loss and regression loss to obtain an intermediate RVMNet model; using the validation set to cross-validate the intermediate RVMNet model, and adjusting the parameter value of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; using the test set to test the target RVMNet model to obtain a qualitative index value and a quantitative index value; judging whether the qualitative index value and the quantitative index value both meet the preset conditions; if the qualitative index value and the quantitative index value both meet the preset conditions, outputting the target RVMNet model.

[0014] According to another embodiment of the present invention, a rail surface defect detection device based on Visual Mamba is provided, including: an acquisition module for acquiring a rail surface image to be detected; a segmentation module for segmenting the rail surface image into multiple non-overlapping patch images, and creating multiple embedded images based on the patch images, wherein each embedded image corresponds to a resolution; a scanning module for performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; a fusion module for fusing the multiple embedded images and the multi-level feature information to obtain fused features; a detection module for inputting the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0015] Optionally, the segmentation module includes: a segmentation unit, used to segment the track surface image into multiple non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; an encoding unit, used to encode the two-dimensional feature maps in the patch map into multiple embedded images through linear projection.

[0016] Optionally, the encoding unit includes: an encoding subunit, for encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution through linear projection; a first sampling subunit, for downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; a second sampling subunit, for downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; a third sampling subunit, for downsampling the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and doubling the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0017] Optionally, the scanning module includes: a scanning unit, used to perform omnidirectional continuous scanning on each embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; an extraction unit, used to extract a feature map of each of the scanning sequences; a merging unit, used to merge the feature maps of the multiple scanning sequences to obtain global feature information; and a pooling unit, used to perform fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0018] Optionally, the scanning unit includes: a first scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the first row and first column of the embedded image to obtain a first scanning sequence; a second scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the first row and first column of the embedded image to obtain a second scanning sequence; a third scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the last row and last column of the embedded image to obtain a third scanning sequence; a fourth scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the last row and last column of the embedded image to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0019] Optionally, the fusion module includes: a sampling unit for upsampling the multi-level feature information to obtain hierarchical features of the same scale as the embedded image; a splicing unit for performing pixel-by-pixel splicing of the embedded image and the hierarchical features of the same scale for each embedded image to obtain spliced ​​features; and a fusion unit for fusing the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0020] Optionally, the device further includes: an acquisition module for acquiring a track surface defect image set before the detection module inputs the fusion feature into the target RVMNet model; an enhancement module for performing data enhancement on the track surface defect image set to obtain a sample set; a division module for dividing the sample set into a training set, a validation set, and a test set; a training module for training an initial RVMNet model using the training set and calculating a classification loss and a regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; a first adjustment module for adjusting the prediction accuracy based on the classification loss. and regression loss to adjust the parameter values ​​of the initial RVMNet model to obtain an intermediate RVMNet model; a second adjustment module, used to cross-validate the intermediate RVMNet model with the validation set, and adjust the parameter values ​​of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; a testing module, used to test the target RVMNet model with the test set to obtain qualitative index values ​​and quantitative index values; a judgment module, used to judge whether the qualitative index values ​​and the quantitative index values ​​both meet the preset conditions; an output module, used to output the target RVMNet model if the qualitative index values ​​and the quantitative index values ​​both meet the preset conditions.

[0021] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0022] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.

[0023] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above-mentioned apparatus embodiments when run.

[0024] According to the embodiment of the present invention, a track surface image to be inspected is obtained; the track surface image is divided into multiple non-overlapping patch images, and multiple embedded images are created based on the patch images, wherein each embedded image corresponds to a resolution; the multiple embedded images are omnidirectionally and continuously scanned, and multi-level feature information is extracted, wherein the multi-level feature information includes feature information of embedded images of all resolutions; the multiple embedded images and the multi-level feature information are fused to obtain fused features; the fused features are input into a target visual mamba-based track surface defect detection network RVMNet model, and the defect position and defect category of the track surface defects in the track surface image are output, and automatic recognition and positioning of track surface defects are realized based on the visual mamba architecture, and multi-level feature information is obtained through omnidirectional continuous scanning, which solves the technical problem in the related art that defects similar to the background in the track surface image are difficult to distinguish, thereby improving the detection efficiency and accuracy of track surface defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of the present invention;

[0027] Figure 2 is a flow chart of a method for detecting track surface defects based on Visual Mamba according to an embodiment of the present invention;

[0028] Figure 3 is an overall network structure diagram in an embodiment of the present invention;

[0029] Figure 4 is a schematic diagram of omnidirectional continuous scanning in an embodiment of the present invention;

[0030] Figure 5is a schematic diagram of a track surface defect detection method based on Visual Mamba in an embodiment of the present invention;

[0031] Figure 6 This is a structural block diagram of a track surface defect detection device based on Visual Mamba according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Example 1

[0035] The method embodiment provided in the first embodiment of the present application can be executed in a server, a computer, a mobile phone, or a similar computing device. Taking running on a computer as an example, Figure 1 This is a hardware structure diagram of a computer according to an embodiment of the present application. Figure 1 As shown, the computer may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. Optionally, the computer may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. Figure 1More or fewer components than shown, or with Figure 1 Different configurations shown.

[0036] Memory 104 can be used to store computer programs, such as software programs and modules for application software, such as the computer program corresponding to a Visual Mamba-based rail surface defect detection method in an embodiment of the present invention. Processor 102 executes the computer program stored in memory 104 to perform various functional applications and data processing, thereby implementing the aforementioned method. Memory 104 may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, which can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a computer's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] In this embodiment, a track surface defect detection method based on Visual Mamba is provided. Figure 2 FIG. 1 is a flow chart of a method for detecting track surface defects based on Visual Mamba according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0039] Step S202, obtaining an image of the track surface to be inspected;

[0040] Step S204: segmenting the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution;

[0041] Step S206, performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions;

[0042] The multi-level feature information includes feature information of embedded images of all resolutions, such as deep information, shallow information, etc.

[0043] Step S208, fusing the multiple embedded images and the multi-level feature information to obtain fused features;

[0044] Step S210: input the fusion features into the target visual mamba-based rail surface defect detection network RVMNet model, and output the defect position and defect category of the rail surface defect in the rail surface image.

[0045] The RVMNet (Rail Surface Defects Detection based on VMamba) model in this embodiment is a rail surface defect detection model. The model includes a patch segmentation module, a hierarchical omnidirectional continuous scanning module, a feature fusion module, and a comprehensive detection module. Split into The size of the patch is The two-dimensional feature map is encoded into a high-dimensional embedded image through linear projection and recorded as a marker . The four network stages create a resolution of , , and The hierarchical representation of the network consists of an omnidirectional continuity scanning block and a downsampling layer, thus constructing a hierarchical omnidirectional continuity scanning module. The omnidirectional continuity scanning block enhances the spatial continuity of the marker features, while the downsampling layer captures more abstract deep features by gradually reducing the resolution of the features and increasing the number of channels. The hierarchical structure can not only extract high-resolution shallow features containing object location information, but also extract low-resolution deep features containing abstract semantic information, thereby improving the ability to detect defects. The specific operation of the omnidirectional continuity scanning block is as follows: the marker is expanded into four continuous sequences with different directions. When the end of a row (or column) is reached, the next scanned marker will be the marker in the next adjacent column (or row), rather than the marker on the opposite side. The scan is then continued in the opposite direction until the scan is completed. In this way, four continuous sequences with different directions are obtained, and the four sequences in different directions are then processed and merged to form the output. By scanning the four continuous sequences in different directions, each pixel in the image can effectively integrate the information of all other pixels in the image, thereby establishing a global receptive field in two-dimensional space and enhancing the ability to extract global features. Each network stage also contains an omnidirectional continuity scanning block and a downsampling layer. The four network stages perform 4x, 8x, 16x and 32x downsampling respectively, reducing the height and width of the marker features to reduce computational complexity, while increasing the number of channels to enhance the expression of track surface defect features. Finally, the fast spatial pyramid pooling (SPPF) receives multi-level features extracted from the hierarchical omnidirectional continuity scanning module through pooling operations at different scales. This multi-level feature representation helps the model better capture the shape, size and location information of defects, thereby improving the accuracy of target detection.

[0046] The comprehensive detection module receives the fused features from the feature fusion module, performs comprehensive detection on them, and outputs defect category and location information. For multiple prediction boxes generated from a track surface defect image, only the prediction with the highest confidence score is selected from multiple prediction results for the same prediction box. The comprehensive detection module receives the feature map output by the feature fusion module, performs classification and regression on it, and outputs defect category and location information.

[0047] The comprehensive detection module is divided into two independent steps: classification and regression. Predicting different feature representations separately can better detect defects of different forms, thereby improving the model's generalization ability. Considering that track surface defects are more obvious on fast tracks, but the area is relatively small. In contrast, on ordinary / heavy tracks, defects are mostly wear and the defect area is relatively large. Due to the high randomness of defect occurrence, this embodiment uses two independent methods, classification and regression, to classify and regress track surface defects. This enables the model to detect defects more accurately, effectively improving the robustness and generalization ability of track surface defect detection.

[0048] Through the above steps, a track surface image to be inspected is obtained; the track surface image is divided into multiple non-overlapping patch images, and multiple embedded images are created based on the patch images, wherein each embedded image corresponds to a resolution; the multiple embedded images are omnidirectionally and continuously scanned, and multi-level feature information is extracted, wherein the multi-level feature information includes feature information of embedded images of all resolutions; the multiple embedded images and the multi-level feature information are fused to obtain fused features; the fused features are input into the target visual mamba-based track surface defect detection network RVMNet model, and the defect position and defect category of the track surface defects in the track surface image are output, and automatic recognition and positioning of track surface defects are realized based on the visual mamba architecture, and multi-level feature information is obtained through omnidirectional continuous scanning, which solves the technical problem in the related art that defects similar to the background in the track surface image are difficult to distinguish, thereby improving the detection efficiency and accuracy of track surface defects.

[0049] In one implementation of this embodiment, the track surface image is segmented into multiple non-overlapping patch maps, and multiple embedded images are created based on the patch maps, including: segmenting the track surface image into multiple non-overlapping two-dimensional feature maps of the same scale to obtain patch maps; and encoding the two-dimensional feature maps in the patch maps into multiple embedded images through linear projection.

[0050] Among them, by segmenting the track surface image, multiple two-dimensional feature maps are obtained. The patch map is composed of multiple two-dimensional feature maps and is used for subsequent omnidirectional continuous scanning. The patch map can be an embedded image of the original resolution of the track surface image. By upsampling or downsampling the embedded image of the original resolution, multiple embedded images with different resolutions can be obtained.

[0051] With the rapid development of image sensor technology, low-cost image data is increasingly being used in rail surface defect detection. However, the surge in data volume makes it difficult for manual inspection methods to meet efficiency and accuracy requirements. Convolutional neural networks (CNNs) extract image features through convolution operations and have achieved remarkable results in computer vision tasks, promoting in-depth research in academia and industry. However, as CNNs increase in size and complexity, their limitations in capturing global information, which is crucial for understanding visual tasks, have become increasingly apparent. Inspired by the attention mechanism in human perception, researchers proposed the Transformer architecture, which enhances the model's ability to capture global features by introducing an attention mechanism. The Transformer architecture overcomes the limitations of CNNs due to their local receptive field, but its computational complexity is quadratic. To address this, this embodiment proposes a rail surface defect detection network based on Visual Mamba.

[0052] In the embodiment of the present invention, the patch segmentation module divides the track surface image into the following steps: Divide into multiple non-overlapping patches, H and W are the height and width of the track surface defect image, and encode the patches into high-dimensional embedded images through linear projection, which are recorded as markers. The size of the marker is , the feature dimension of each tag is , and then transform the labeled feature dimension into the dimension C that the model can handle, and C is set to 96.

[0053] In one example, encoding the two-dimensional feature map in the patch image into multiple embedded images by linear projection includes: encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution by linear projection; downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; downsampling the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and doubling the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0054] In one implementation of this embodiment, performing omnidirectional continuous scanning on the multiple embedded images and extracting global feature information includes: for each embedded image, performing omnidirectional continuous scanning on the embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; extracting a feature map of each of the scanning sequences; merging the feature maps of the multiple scanning sequences to obtain global feature information; and performing fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0055] Figure 3 This is the overall network structure diagram of the embodiment of the present invention, which includes four network stages from stage 1 to stage 4. Each stage includes an omnidirectional selective scanning block and downsampling. The hierarchical omnidirectional continuous scanning is used to extract multi-level features of the track surface image. The four constructed network stages (respectively constructing the first embedded image, the second embedded image, the third embedded image, and the fourth embedded image) all include omnidirectional continuous scanning blocks and downsampling layers. For example, the resolution of the first embedded image in stage 1 is , the feature dimension is The core component of omnidirectional continuity scanning is the state space model (SSM), which characterizes the dynamics of the system through a set of hidden variables called "states", enabling it to effectively capture the dependencies of time data and has correlation characteristics. It usually includes state equations and output equations to describe the internal state and output of the system. This module first maps the features to one dimension, and the linear steady-state system is represented by the hidden state. Mapping to N-dimensional latent states Mapping to output These systems are usually expressed as linear ordinary differential equations, which are given by:

[0056]

[0057] in, Is the current state The derivative of . is the state transition matrix that describes how the state changes over time; is the input matrix that controls how the input affects the state change; Is the output matrix, which represents the output generated according to the current state.

[0058] In one example, the embedded image is omnidirectionally and continuously scanned according to multiple scanning trajectories to obtain multiple scanning sequences, including: starting from the first row and first column of the embedded image, omnidirectionally and continuously scanning row by row to obtain a first scanning sequence; starting from the first row and first column of the embedded image, omnidirectionally and continuously scanning column by column to obtain a second scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally and continuously scanning row by row to obtain a third scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally and continuously scanning column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0059] In this example, the omnidirectional continuous scan is performed in four different trajectories. Figure 4 It is a schematic diagram of omnidirectional continuous scanning in an embodiment of the present invention. The embedded image includes 9 patch images, namely 1 to 9. When the end of a row (or column) is reached, the mark to be scanned in the next step will be the mark in the next adjacent column (or row), rather than the mark on the opposite side. Then continue scanning in the opposite direction until the scan is completed. The extended scanning sequences include: 123654789, 147852369, 987456321, 963258741. The S6 block then processes the four different sequences independently. Finally, the four different sequences are added and merged and output, so that each pixel in the image can effectively integrate the information of all other pixels in the image, thereby establishing a global receptive field in two-dimensional space and enhancing the ability to extract image features. During the continuous scanning process, the current mark can be located in one of four possible directions relative to its previous mark. This is because the model only relies on Identifying the precise spatial relationship between adjacent markers poses a challenge. A set of learnable parameters { }( By correlating these parameters with the data Sequence addition can enhance directional information during the scanning process. The direction-aware update formula can ensure that the global features of the image are accurately preserved during scanning. Its calculation formula is as follows:

[0060] ;

[0061] ;

[0062] ;

[0063] in, Used to indicate four different sequence directions, Represents the discretization , represent.

[0064] This embodiment uses downsampling to reduce the height and width of input features to reduce computational complexity, while increasing the number of channels to enhance the expressiveness of defect features. The number of channels in each stage of the model of the present invention is [C, 2C, 4C, 8C] respectively.

[0065] The computational complexity of the omnidirectional continuous scanning block in this embodiment is linear. Compared with the computational complexity of the Transformer quadratic form, it effectively reduces the number of model parameters, making RVMNet lightweight. The calculation formulas of the attention mechanism (self-attention) and SSM are as follows:

[0066] ;

[0067] ;

[0068] in, Indicates the sequence length, corresponding to the length and width of the resolution, Represents the feature dimension, N represents a fixed parameter, which is set to 16 by default. Attention mechanism and sequence length There is a quadratic relationship between SSM and sequence length The linear relationship and the reduction in computational complexity enable Visual Mamba to be extended to applications with large sequence lengths, improving the efficiency of rail surface defect detection tasks.

[0069] In this embodiment, the multiple embedded images and the multi-level feature information are fused to obtain fused features, including: upsampling the multi-level feature information to obtain hierarchical features with the same scale as the embedded images; for each embedded image, pixel-by-pixel splicing the embedded image and the hierarchical features with the same scale to obtain spliced ​​features; and fusing the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0070] In this embodiment, when performing feature fusion, Figure 3As shown, the feature fusion module includes a cross-channel feature fusion block, a splicing operation, an upsampling layer, and a deep convolution layer. The feature fusion module fuses the multi-level feature information, combining shallow and deep features to detect defects of different sizes. The process includes: first, an upsampling operation restores the spatial scale of the low-resolution feature maps in the multi-level feature information. Then, a splicing operation is performed to splice the upsampled spliced ​​feature map with the high-resolution feature map of the embedded image pixel by pixel, preserving detailed information and fusing features of different scales. The cross-channel feature fusion block uses convolution operations to fuse features from different channels, allowing the relevant information between different channels to effectively complement each other, thereby enhancing the representation of defect features. In addition, the deep convolution operation deeply extracts local information and high-level features of image features. Through multiple stages of cross-channel feature fusion blocks, splicing operations, upsampling layers, and deep convolution layers, the feature fusion module improves the spatial resolution of the feature map, ultimately outputting a fusion of multi-level features. This approach not only ensures the effective transmission of multi-scale information but also further enhances the model's expressiveness, enabling accurate detection of defects of different scales and improving the model's detection performance.

[0071] Optionally, before inputting the fusion feature into the target RVMNet model, it also includes: collecting a set of track surface defect images; performing data enhancement on the track surface defect image set to obtain a sample set; dividing the sample set into a training set, a validation set and a test set; using the training set to train the initial RVMNet model, and calculating the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; adjusting the parameter value of the initial RVMNet model based on the classification loss and regression loss to obtain an intermediate RVMNet model; using the validation set to cross-validate the intermediate RVMNet model, and adjusting the parameter value of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; using the test set to test the target RVMNet model to obtain qualitative index values ​​and quantitative index values; judging whether the qualitative index values ​​and the quantitative index values ​​both meet the preset conditions; if the qualitative index values ​​and the quantitative index values ​​both meet the preset conditions, outputting the target RVMNet model.

[0072] In one implementation scenario, Figure 5This is a schematic diagram of a rail surface defect detection method based on Visual Mamba in an embodiment of the present invention. The model training process includes: obtaining a public rail surface defect image set (rail surface defect images and corresponding sample labels); performing data augmentation on the rail surface defect images and randomly dividing them into a training set, a validation set, and a test set in a ratio of 7:2:1 to obtain the rail surface defect sample set required for the experiment; using the rail surface defect training set to train the initial model, using the validation set to cross-validate the performance of the RVMNet model, and dynamically adjusting the model parameters to ultimately obtain the RVMNet model for rail surface defect detection.

[0073] Rail surface discrete defects datasets (RSDDs) were obtained, including Type-I and Type-II RSDDs datasets. Data preprocessing was then performed on the images, augmenting them using data augmentation techniques such as horizontal flipping, rotation, Gaussian blurring, noise addition, and brightness changes. This generated a sample set, which was randomly divided into training, validation, and test sets in a ratio of 7:2:1. The training set was used to train the model, the validation set was used to verify model performance and dynamically adjust model parameters, and the test set was used to assess the model's learning effectiveness.

[0074] The initial model was trained using the track surface defect training set. The validation set was used to cross-validate the RVMNet model's performance, and the model parameters were dynamically adjusted. Ultimately, the RVMNet model for track surface defect detection was obtained. Experimental results show that RVMNet achieves detection accuracy of 93.12% and 86.78% on the Type-I RSDDs dataset (for fast track) and Type-II RSDDs dataset (for normal / heavy-load track), respectively. Recall rates reach 88.22% and 62.06%, respectively, and F1 scores reach 90.6% and 72.37%, respectively. With a parameter count of 5.98M, it achieves high-precision detection and a lightweight model.

[0075] During the training process, the model parameters are dynamically adjusted according to the loss value to obtain the optimal RVMNet model, including: calculating the confidence loss function and the bounding box loss function. The total training loss function adopts a weighted mode, where the confidence loss function is used to calculate the difference between the predicted category and the true category; the bounding box loss function is used to calculate the difference between the predicted bounding box and the true bounding box;

[0076] The confidence loss function is used to calculate the difference between the predicted category and the true category. It is usually calculated using the binary cross entropy loss. The calculation formula is as follows:

[0077] ;

[0078] in, The value of is 0 or 1, Is the output of The probability of . When it is 1, approaches 1, the value of the loss function approaches 0. On the contrary, Approaching 0, the value of the loss function is very large;

[0079] The bounding box loss function is used to calculate the difference between the predicted bounding box and the true bounding box. CIoULoss can be used to calculate the bounding box loss function. The loss value The calculation formula is as follows:

[0080] ; ;

[0081] Among them, IoU represents the ratio of the intersection and union of the predicted box and the real box; is the Euclidean distance between the center of the predicted box and the true box, is the diagonal length of the bounding box, is a small constant used to avoid division by zero errors, is the weight, w and h are the width and height of the image, is the difference in aspect ratio.

[0082] The weighted calculation results of the confidence loss function and the bounding box loss function are used as the total training loss value , the calculation formula is as follows:

[0083] ;

[0084] in Total loss function for model training, confidence loss weight Set to 0.05, the bounding box loss weight Set to 0.75;

[0085] The confidence loss function and bounding box loss function guide model optimization by measuring the gap between model predictions and actual annotations, thereby improving the performance and accuracy of the model. The confidence loss function and bounding box loss function are used to evaluate the model and adjust parameters as a guide in the model optimization process, so that the model can better fit the data.

[0086] To evaluate the performance of the model, we used common evaluation metrics: Precision, Recall, F1 score, and Params. Precision is expressed as the ratio of the number of retrieved relevant items (i.e., correctly classified samples) to the total number of retrieved items; Recall is expressed as the ratio of the number of retrieved relevant items to all relevant items; and F1 is the harmonic mean of the model, describing its stability. The specific calculation formula is as follows:

[0087]

[0088]

[0089]

[0090] Using the solution of this embodiment, a public track surface image and the sample labels corresponding to the image are obtained; the track surface defect image is input into the RVMNet model for track surface defect detection; first, the patch segmentation module divides the track surface image into patches, and the patches are encoded into high-dimensional embeddings through linear projection and recorded as labels. Then, the hierarchical omnidirectional continuous scanning module learns key information such as the category, shape, and position of the track surface defects, extracts the multi-level features of the labels, and the feature fusion module fuses the extracted multi-level features. The comprehensive detection module receives the fused feature map and outputs the category and location information of the defects. Based on the visual Mamba architecture, the automatic identification and positioning of track surface defects is realized, solving the technical problems of poor defect detection performance caused by uneven brightness of the track surface image and the difficulty in distinguishing defects from the background, thereby improving the efficiency and accuracy of track surface defect detection.

[0091] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0092] Example 2

[0093] This embodiment also provides a track surface defect detection device based on Visual Mamba. This device is used to implement the above-mentioned embodiments and preferred implementations. Details already described are omitted. As used below, the term "module" may refer to a combination of software and hardware that implements the intended functionality. While the devices described in the following embodiments are preferably implemented in software, hardware, or a combination of software and hardware, is also contemplated.

[0094] Figure 6 This is a structural block diagram of a track surface defect detection device based on visual Mamba according to an embodiment of the present invention. Figure 6 Shown, including:

[0095] An acquisition module 60 is used to acquire an image of the track surface to be detected;

[0096] a segmentation module 62 for segmenting the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution;

[0097] a scanning module 64 for performing omnidirectional continuous scanning on the plurality of embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions;

[0098] a fusion module 66, configured to fuse the multiple embedded images and the multi-level feature information to obtain fused features;

[0099] The detection module 68 is used to input the fusion features into the target visual mamba-based rail surface defect detection network RVMNet model, and output the defect location and defect category of the rail surface defect in the rail surface image.

[0100] Optionally, the segmentation module includes: a segmentation unit, used to segment the track surface image into multiple non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; an encoding unit, used to encode the two-dimensional feature maps in the patch map into multiple embedded images through linear projection.

[0101] Optionally, the encoding unit includes: an encoding subunit, for encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution through linear projection; a first sampling subunit, for downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; a second sampling subunit, for downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; a third sampling subunit, for downsampling the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and doubling the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0102] Optionally, the scanning module includes: a scanning unit, used to perform omnidirectional continuous scanning on each embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; an extraction unit, used to extract a feature map of each of the scanning sequences; a merging unit, used to merge the feature maps of the multiple scanning sequences to obtain global feature information; and a pooling unit, used to perform fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0103] Optionally, the scanning unit includes: a first scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the first row and first column of the embedded image to obtain a first scanning sequence; a second scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the first row and first column of the embedded image to obtain a second scanning sequence; a third scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the last row and last column of the embedded image to obtain a third scanning sequence; a fourth scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the last row and last column of the embedded image to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0104] Optionally, the fusion module includes: a sampling unit for upsampling the multi-level feature information to obtain hierarchical features of the same scale as the embedded image; a splicing unit for performing pixel-by-pixel splicing of the embedded image and the hierarchical features of the same scale for each embedded image to obtain spliced ​​features; and a fusion unit for fusing the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0105] Optionally, the device further includes: an acquisition module for acquiring a track surface defect image set before the detection module inputs the fusion feature into the target RVMNet model; an enhancement module for performing data enhancement on the track surface defect image set to obtain a sample set; a division module for dividing the sample set into a training set, a validation set, and a test set; a training module for training an initial RVMNet model using the training set and calculating a classification loss and a regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; a first adjustment module for adjusting the prediction accuracy based on the classification loss. and regression loss to adjust the parameter values ​​of the initial RVMNet model to obtain an intermediate RVMNet model; a second adjustment module, used to cross-validate the intermediate RVMNet model with the validation set, and adjust the parameter values ​​of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; a testing module, used to test the target RVMNet model with the test set to obtain qualitative index values ​​and quantitative index values; a judgment module, used to judge whether the qualitative index values ​​and the quantitative index values ​​both meet the preset conditions; an output module, used to output the target RVMNet model if the qualitative index values ​​and the quantitative index values ​​both meet the preset conditions.

[0106] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0107] Example 3

[0108] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0109] Optionally, in this embodiment, the storage medium may be configured to store a computer program for executing:

[0110] S1, obtaining the track surface image to be inspected;

[0111] S2, dividing the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution;

[0112] S3, performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of the embedded images at all resolutions;

[0113] S4, fusing the multiple embedded images and the multi-level feature information to obtain a fused feature;

[0114] S5, inputting the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0115] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0116] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0117] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0118] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0119] S1, obtaining the track surface image to be inspected;

[0120] S2, dividing the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution;

[0121] S3, performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of the embedded images at all resolutions;

[0122] S4, fusing the multiple embedded images and the multi-level feature information to obtain a fused feature;

[0123] S5, inputting the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0124] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0125] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0126] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0128] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0129] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, controller or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk, and other media that can store program code.

[0131] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A track surface defect detection method based on Visual Mamba, characterized in that: include: Acquiring a track surface image to be inspected; Segmenting the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; Performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; fusing the multiple embedded images and the multi-level feature information to obtain fused features; Inputting the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and outputting the defect location and defect category of the rail surface defect in the rail surface image; Among them, the multiple embedded images are omnidirectionally and continuously scanned, and the global feature information is extracted, including: for each embedded image, the embedded image is omnidirectionally and continuously scanned according to multiple scanning trajectories to obtain multiple scanning sequences, and when each scanning sequence reaches the end of a row or column, the next step is to scan the object adjacent to the current object in the next column or row, and continue scanning in the opposite direction until the scan is completed, wherein each scanning sequence corresponds to a scanning trajectory; extracting the feature map of each scanning sequence; merging the feature maps of the multiple scanning sequences to obtain global feature information; performing fast spatial pyramid pooling on the multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information. Information; wherein, the embedded image is omnidirectionally and continuously scanned according to multiple scanning trajectories to obtain multiple scanning sequences, including: starting from the first row and first column of the embedded image, omnidirectionally and continuously scanned row by row to obtain a first scanning sequence; starting from the first row and first column of the embedded image, omnidirectionally and continuously scanned column by column to obtain a second scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally and continuously scanned row by row to obtain a third scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally and continuously scanned column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

2. The method according to claim 1, characterized in that Segmenting the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, comprising: Segmenting the track surface image into a plurality of non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; The 2D feature maps in the patch image are encoded into multiple embedded images via linear projection.

3. The method according to claim 2, characterized in that Encoding the two-dimensional feature map in the patch image into multiple embedded images through linear projection includes: Encoding the two-dimensional feature map in the patch image into a first embedded image of the initial resolution by linear projection; downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; The third embedded image is downsampled to obtain a fourth embedded image of a third intermediate resolution, and the number of channels of the fourth embedded image is doubled, wherein the second intermediate resolution is twice the third intermediate resolution.

4. The method according to claim 1, wherein Fusing the multiple embedded images and the multi-level feature information to obtain fused features includes: Upsampling the multi-level feature information to obtain hierarchical features with the same scale as the embedded image; For each embedded image, the embedded image and the hierarchical features of the same scale are pixel-by-pixel spliced ​​to obtain a spliced ​​feature; The convolution operation is used to fuse the splicing features of multiple scales to obtain the fused features.

5. The method according to claim 1, wherein Before inputting the fused features into the target RVMNet model, the method further includes: Collect track surface defect image sets; Performing data enhancement on the track surface defect image set to obtain a sample set; Dividing the sample set into a training set, a validation set, and a test set; Using the training set to train an initial RVMNet model, and calculating the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; Adjusting the parameter values ​​of the initial RVMNet model based on the classification loss and the regression loss to obtain an intermediate RVMNet model; Cross-validating the intermediate RVMNet model using the validation set, and adjusting parameter values ​​of the intermediate RVMNet model based on the validation results to obtain a target RVMNet model; The target RVMNet model is tested using the test set to obtain qualitative and quantitative index values; Determine whether both the qualitative indicator value and the quantitative indicator value meet the preset conditions; If both the qualitative index value and the quantitative index value meet the preset conditions, the target RVMNet model is output.

6. A track surface defect detection device based on visual Mamba, characterized in that: include: An acquisition module, used for acquiring an image of the track surface to be detected; a segmentation module, configured to segment the track surface image into a plurality of non-overlapping patch images, and create a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; a scanning module, configured to perform omnidirectional and continuous scanning on the plurality of embedded images and extract multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; a fusion module, configured to fuse the multiple embedded images and the multi-level feature information to obtain fused features; a detection module, configured to input the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and output a defect location and defect category of the rail surface defect in the rail surface image; The scanning module includes: a scanning unit for performing omnidirectional continuous scanning on each embedded image according to multiple scanning trajectories to obtain multiple scanning sequences. When each scanning sequence reaches the end of a row or column, the next step is to scan the object adjacent to the current object in the next column or row, and the scanning is continued in the opposite direction until the scanning is completed. Each scanning sequence corresponds to a scanning trajectory; an extraction unit for extracting a feature map of each scanning sequence; a merging unit for merging the feature maps of the multiple scanning sequences to obtain global feature information; a pooling unit for performing fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information; Wherein, the scanning unit includes: a first scanning subunit, used to start from the first row and first column of the embedded image, and perform omnidirectional continuous scanning row by row to obtain a first scanning sequence; a second scanning subunit, used to start from the first row and first column of the embedded image, and perform omnidirectional continuous scanning column by column to obtain a second scanning sequence; a third scanning subunit, used to start from the last row and last column of the embedded image, and perform omnidirectional continuous scanning row by row to obtain a third scanning sequence; a fourth scanning subunit, used to start from the last row and last column of the embedded image, and perform omnidirectional continuous scanning column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

7. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the steps of the method for detecting rail surface defects in any one of claims 1 to 5 when the computer program is run.

8. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, communication interface, and memory communicate with each other via a communication bus; wherein: Memory for storing computer programs; A processor, configured to execute the steps of the method for detecting rail surface defects according to any one of claims 1 to 5 by running a program stored in a memory.

Images