Visual Mangbar-based track surface defect detection method and device, storage medium and electronic equipment

Abstract

The invention provides a track surface defect detection method and device based on vision Mangbar, a storage medium and electronic equipment, and the method comprises the steps: obtaining a to-be-detected track surface image; the track surface image is segmented into a plurality of patch diagrams which are not overlapped with one another, a plurality of embedded images are created based on the patch diagrams, and each embedded image corresponds to one resolution; the plurality of embedded images are subjected to omnidirectional continuous scanning, multi-level feature information is extracted, and the multi-level feature information comprises feature information of the embedded images of all resolutions; fusing the plurality of embedded images and the multi-level feature information to obtain fused features; and inputting the fusion features into a target track surface defect detection network RVMNet model based on visual Mangbar, and outputting the defect position and defect category of the track surface defect in the track surface image. The technical problem that the defect similar to the background in the track surface image is difficult to distinguish in the related technology is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a track surface defect detection method and device based on Visual Mamba, a storage medium, and an electronic device. Background Art

[0002] In related technologies, with the advancement of urbanization, the mileage, speed and density of railway operations are increasing, and the importance of railway safety is becoming more and more prominent. Its health and stability have a profound impact on public safety and property. Due to the complex working environment of railway infrastructure, defects will inevitably occur during the service life of the track, and they will accumulate and expand over time.

[0003] Track defects are usually divided into internal defects and external defects. During the long-term operation of the train, the extrusion pressure and excessive friction on the track gradually increase, and the internal defects will gradually expand. Nowadays, the track production process is mature and the internal defects are greatly reduced. However, due to the increase in friction and impact between the train wheels and the track surface, the volatile external environment and other factors, track surface defects have become one of the threats to rail transportation. Common track surface defects include cracks, wear and deformation. If not detected and repaired in time, surface defects will continue to expand over time.

[0004] With the rapid development of artificial intelligence technology, target detection algorithms based on deep learning have also been widely used in the field of rail surface defect detection. Although current deep learning technology has achieved certain results in rail surface defect detection tasks, these methods still have some problems. The main ones are: (1) The light at the track inspection site is low, resulting in uneven brightness of the collected track surface defect images and similarities between the track surface defects and the background, which can easily lead to missed detections and false detections. The existing methods have low detection performance for track surface defects. (2) Due to the huge passenger volume of rail transit, the time left for track surface defect detection is usually very short. Through observation of the track surface images, the applicant found that the track surface defects are discretely distributed, and most track surfaces are in a normal state. The use of complex deep learning networks will result in a waste of computing resources and time.

[0005] With respect to the above-mentioned problems existing in the related technologies, no efficient and accurate solutions have been found yet. Summary of the invention

[0006] The present invention provides a track surface defect detection method and device based on Visual Mamba, a storage medium, and an electronic device to solve the technical problems in the related art.

[0007] According to one embodiment of the present invention, a rail surface defect detection method based on Visual Mamba is provided, comprising: acquiring a rail surface image to be detected; segmenting the rail surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; performing omnidirectional continuous scanning on the plurality of embedded images, and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; fusing the plurality of embedded images and the multi-level feature information to obtain fused features; inputting the fused features into a target rail surface defect detection network RVMNet model based on Visual Mamba, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0008] Optionally, the track surface image is segmented into a plurality of non-overlapping patch maps, and a plurality of embedded images are created based on the patch maps, including: segmenting the track surface image into a plurality of non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; encoding the two-dimensional feature map in the patch map into a plurality of embedded images by linear projection.

[0009] Optionally, encoding the two-dimensional feature map in the patch image into multiple embedded images through linear projection includes: encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution through linear projection; downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; downsampling the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and doubling the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0010] Optionally, performing omnidirectional continuous scanning on the multiple embedded images and extracting global feature information includes: for each embedded image, performing omnidirectional continuous scanning on the embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; extracting a feature map of each of the scanning sequences; merging the feature maps of the multiple scanning sequences to obtain global feature information; and performing fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0011] Optionally, the embedded image is omnidirectionally continuously scanned according to multiple scanning trajectories to obtain multiple scanning sequences, including: starting from the first row and first column of the embedded image, omnidirectionally continuously scanning row by row to obtain a first scanning sequence; starting from the first row and first column of the embedded image, omnidirectionally continuously scanning column by column to obtain a second scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally continuously scanning row by row to obtain a third scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally continuously scanning column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0012] Optionally, the multiple embedded images and the multi-level feature information are fused to obtain fused features, including: upsampling the multi-level feature information to obtain hierarchical features with the same scale as the embedded images; for each embedded image, pixel-by-pixel splicing the embedded image and the hierarchical features with the same scale to obtain spliced ​​features; and fusing the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0013] Optionally, before inputting the fusion feature into the target RVMNet model, the method further includes: collecting a set of rail surface defect images; performing data enhancement on the set of rail surface defect images to obtain a sample set; dividing the sample set into a training set, a validation set and a test set; using the training set to train the initial RVMNet model, and calculating the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; adjusting the parameter value of the initial RVMNet model based on the classification loss and the regression loss to obtain an intermediate RVMNet model; using the validation set to cross-validate the intermediate RVMNet model, and adjusting the parameter value of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; using the test set to test the target RVMNet model to obtain a qualitative index value and a quantitative index value; judging whether the qualitative index value and the quantitative index value both meet the preset conditions; if the qualitative index value and the quantitative index value both meet the preset conditions, outputting the target RVMNet model.

[0014] According to another embodiment of the present invention, a rail surface defect detection device based on visual mamba is provided, including: an acquisition module, used to acquire a rail surface image to be detected; a segmentation module, used to segment the rail surface image into multiple non-overlapping patch images, and create multiple embedded images based on the patch images, wherein each embedded image corresponds to a resolution; a scanning module, used to perform omnidirectional continuous scanning on the multiple embedded images, and extract multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; a fusion module, used to fuse the multiple embedded images and the multi-level feature information to obtain fused features; a detection module, used to input the fused features into a target rail surface defect detection network RVMNet model based on visual mamba, and output the defect position and defect category of the rail surface defect in the rail surface image.

[0015] Optionally, the segmentation module includes: a segmentation unit, used to segment the track surface image into a plurality of non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; and an encoding unit, used to encode the two-dimensional feature map in the patch map into multiple embedded images through linear projection.

[0016] Optionally, the encoding unit includes: an encoding subunit, used to encode the two-dimensional feature map in the patch image into a first embedded image of an initial resolution through linear projection; a first sampling subunit, used to downsample the first embedded image to obtain a second embedded image of a first intermediate resolution, and double the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; a second sampling subunit, used to downsample the second embedded image to obtain a third embedded image of a second intermediate resolution, and double the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; a third sampling subunit, used to downsample the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and double the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0017] Optionally, the scanning module includes: a scanning unit, used to perform omnidirectional continuous scanning on each embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; an extraction unit, used to extract a feature map of each of the scanning sequences; a merging unit, used to merge the feature maps of the multiple scanning sequences to obtain global feature information; and a pooling unit, used to perform fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0018] Optionally, the scanning unit includes: a first scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the first row and first column of the embedded image to obtain a first scanning sequence; a second scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the first row and first column of the embedded image to obtain a second scanning sequence; a third scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the last row and last column of the embedded image to obtain a third scanning sequence; a fourth scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the last row and last column of the embedded image to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0019] Optionally, the fusion module includes: a sampling unit, used to upsample the multi-level feature information to obtain hierarchical features of the same scale as the embedded image; a splicing unit, used to perform pixel-by-pixel splicing of the embedded image and the hierarchical features of the same scale for each embedded image to obtain a spliced ​​feature; and a fusion unit, used to fuse the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0020] Optionally, the device further includes: an acquisition module, which is used to acquire a track surface defect image set before the detection module inputs the fusion feature into the target RVMNet model; an enhancement module, which is used to perform data enhancement on the track surface defect image set to obtain a sample set; a division module, which is used to divide the sample set into a training set, a validation set and a test set; a training module, which is used to train the initial RVMNet model using the training set and calculate the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; a first adjustment module, which is used to adjust the prediction result based on the classification loss. and regression loss to adjust the parameter values ​​of the initial RVMNet model to obtain an intermediate RVMNet model; a second adjustment module, used to cross-validate the intermediate RVMNet model with the validation set, and adjust the parameter values ​​of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; a testing module, used to test the target RVMNet model with the test set to obtain qualitative indicator values ​​and quantitative indicator values; a judgment module, used to judge whether the qualitative indicator values ​​and the quantitative indicator values ​​both meet the preset conditions; an output module, used to output the target RVMNet model if the qualitative indicator values ​​and the quantitative indicator values ​​both meet the preset conditions.

[0021] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0022] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0023] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned device embodiments when running.

[0024] According to the embodiment of the present invention, a track surface image to be detected is obtained; the track surface image is divided into a plurality of non-overlapping patch images, and a plurality of embedded images are created based on the patch images, wherein each embedded image corresponds to a resolution; the plurality of embedded images are omnidirectionally and continuously scanned, and multi-level feature information is extracted, wherein the multi-level feature information includes feature information of embedded images of all resolutions; the plurality of embedded images and the multi-level feature information are fused to obtain fused features; the fused features are input into a target visual mamba-based track surface defect detection network RVMNet model, and the defect position and defect category of the track surface defects in the track surface image are output, automatic recognition and positioning of track surface defects are realized based on the visual mamba architecture, and multi-level feature information is obtained through omnidirectional and continuous scanning, thereby solving the technical problem in the related art that defects similar to the background in the track surface image are difficult to distinguish, and improving the detection efficiency and accuracy of track surface defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present invention; Figure 2 is a flow chart of a track surface defect detection method based on Visual Mamba according to an embodiment of the present invention; Figure 3 is an overall network structure diagram in an embodiment of the present invention; Figure 4 is a schematic diagram of omnidirectional continuous scanning in an embodiment of the present invention; Figure 5 is a schematic diagram of a track surface defect detection method based on Visual Mamba in an embodiment of the present invention; Figure 6It is a structural block diagram of a track surface defect detection device based on Visual Mamba according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Example 1 The method embodiment provided in the first embodiment of the present application can be executed in a server, a computer, a mobile phone, or a similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer in an embodiment of the present application. Figure 1 As shown, the computer may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a track surface defect detection method based on visual Mamba in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a track surface defect detection method based on Visual Mamba is provided. Figure 2 is a flow chart of a track surface defect detection method based on visual Mamba according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps: Step S202, obtaining a track surface image to be detected; Step S204, dividing the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; Step S206, performing omnidirectional continuous scanning on the multiple embedded images, and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; The multi-level feature information includes feature information of embedded images of all resolutions, such as deep information, shallow information, etc.

[0032] Step S208, fusing the multiple embedded images and the multi-level feature information to obtain a fused feature; Step S210, inputting the fusion feature into the target visual mamba-based rail surface defect detection network RVMNet model, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0033] The RVMNet (Rail Surface Defects Detection based on VMamba) model in this embodiment is a rail surface defect detection model, which includes a patch segmentation module, a hierarchical omnidirectional continuous scanning module, a feature fusion module and a comprehensive detection module. Split into The size of the patch is The two-dimensional feature map is encoded into a high-dimensional embedded image through linear projection and marked as The four network stages create a resolution of , , and The hierarchical representation of each network stage contains an omnidirectional continuity scanning block and a downsampling layer, thereby constructing a hierarchical omnidirectional continuity scanning module. The omnidirectional continuity scanning block enhances the spatial continuity of the marking features, and the downsampling layer captures more abstract deep features by gradually reducing the resolution of the features and increasing the number of channels. The hierarchical structure can not only extract high-resolution shallow features containing object location information, but also extract low-resolution deep features containing abstract semantic information, thereby improving the detection capability of defects. The specific operation of the omnidirectional continuity scanning block is: the mark is expanded into four continuous sequences with different directions. When the end of a row (or column) is reached, the mark to be scanned in the next step will be the mark in the adjacent next column (or row), rather than the mark on the opposite side, and then continue scanning in the opposite direction until the scanning is completed. In this way, four continuous sequences with different directions are obtained, and then the sequences in the four different directions are processed, and then the sequences in the four different directions are merged to form an output. By scanning the continuous sequences in the four different directions, each pixel in the image can effectively integrate the information of all other pixels in the image, thereby establishing a global receptive field in two-dimensional space and enhancing the ability to extract global features. Each network stage also contains an omnidirectional continuity scanning block and a downsampling layer. The four network stages perform 4 times, 8 times, 16 times and 32 times downsampling respectively, reducing the height and width of the marker features to reduce the computational complexity, while increasing the number of channels to enhance the expression of track surface defect features. Finally, the fast spatial pyramid pooling (SPPF) receives multi-level features extracted from the hierarchical omnidirectional continuity scanning module through pooling operations of different scales. This multi-level feature representation helps the model better capture the shape, size and location information of defects, thereby improving the accuracy of target detection.

[0034] The comprehensive detection module receives the fused features from the feature fusion module, performs comprehensive detection on them, and outputs the category information and location information of the defects. For multiple prediction frames generated by a track surface defect image, only the prediction result with the highest confidence score is taken for multiple prediction results of the same prediction frame. The comprehensive detection module receives the feature map output by the feature fusion module, performs classification and regression on it, and outputs the category information and location information of the defects.

[0035] In the comprehensive detection module, there are two independent steps: classification and regression. Predicting different feature representations separately can better detect defects of different forms, thereby improving the generalization ability of the model. Considering that the surface defects of the track are more obvious on the fast track, but the area is relatively small. On the contrary, on ordinary / heavy tracks, the defects are mostly wear and the defect area is relatively large. Since the occurrence of defects is highly random, this embodiment uses two independent methods of classification and regression to classify and regress the track surface defects, so that the model can detect defects more accurately and effectively improve the robustness and generalization ability of track surface defect detection.

[0036] Through the above steps, a track surface image to be detected is obtained; the track surface image is divided into multiple non-overlapping patch images, and multiple embedded images are created based on the patch images, wherein each embedded image corresponds to a resolution; the multiple embedded images are omnidirectionally and continuously scanned, and multi-level feature information is extracted, wherein the multi-level feature information includes feature information of embedded images of all resolutions; the multiple embedded images and the multi-level feature information are fused to obtain fused features; the fused features are input into the target visual mamba-based track surface defect detection network RVMNet model, and the defect position and defect category of the track surface defects in the track surface image are output, and automatic recognition and positioning of track surface defects are realized based on the visual mamba architecture, and multi-level feature information is obtained through omnidirectional continuous scanning, which solves the technical problem in the related art that defects similar to the background in the track surface image are difficult to distinguish, thereby improving the detection efficiency and accuracy of track surface defects.

[0037] In one implementation of this embodiment, the track surface image is segmented into a plurality of non-overlapping patch maps, and a plurality of embedded images are created based on the patch maps, including: segmenting the track surface image into a plurality of non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; and encoding the two-dimensional feature map in the patch map into a plurality of embedded images by linear projection.

[0038] Among them, by segmenting the track surface image, multiple two-dimensional feature maps are obtained. The patch map is composed of multiple two-dimensional feature maps and is used for subsequent omnidirectional continuous scanning. The patch map can be an embedded image of the original resolution of the track surface image. By upsampling or downsampling the embedded image of the original resolution, multiple embedded images of different resolutions can be obtained.

[0039] With the rapid development of image sensor technology, low-cost image data is increasingly used in rail surface defect detection. However, the surge in data volume makes it difficult for manual inspection methods to meet the needs in terms of efficiency and accuracy. Convolutional neural networks (CNNs) extract image features through convolution operations and have achieved remarkable results in computer vision tasks, promoting their in-depth research in academia and industry. However, as the scale and complexity of CNNs increase, their limitations in capturing global information gradually emerge, and global information is crucial to the understanding of visual tasks. Inspired by the attention mechanism in human perception, researchers proposed the Transformer architecture, which enhances the model's ability to capture global features by introducing an attention mechanism. The Transformer architecture breaks through the limitation of CNNs to local receptive fields, but its computational complexity is quadratic. In response to this, this embodiment is based on a rail surface defect detection network based on Visual Mamba.

[0040] In the implementation of this embodiment, the process of the patch segmentation module segmenting the track surface image includes: Divide into multiple non-overlapping patches, H and W are the height and width of the track surface defect image, and encode the patch into a high-dimensional embedded image by linear projection, which is recorded as a marker. The size of the marker is , the feature dimension of each tag is , and then transform the labeled feature dimension into the dimension C that the model can handle, and C is set to 96.

[0041] In one example, encoding the two-dimensional feature map in the patch image into multiple embedded images by linear projection includes: encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution by linear projection; downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; downsampling the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and doubling the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0042] In one implementation of the present embodiment, performing omnidirectional continuous scanning on the multiple embedded images and extracting global feature information includes: for each embedded image, performing omnidirectional continuous scanning on the embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; extracting a feature map of each of the scanning sequences; merging the feature maps of the multiple scanning sequences to obtain global feature information; and performing fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0043] Figure 3 : is the overall network structure diagram of the embodiment of the present invention, including four network stages from stage 1 to stage 4, each stage includes an omnidirectional selective scanning block and downsampling, and the hierarchical omnidirectional continuous scanning is used to extract the multi-level features of the track surface image. The four constructed network stages (respectively constructing the first embedded image, the second embedded image, the third embedded image, and the fourth embedded image) all include an omnidirectional continuous scanning block and a downsampling layer. For example, the resolution of the first embedded image in stage 1 is , the feature dimension is The core component of omnidirectional continuous scanning is the state space model (SSM), which characterizes the dynamics of the system through a set of hidden variables called "states", enabling it to effectively capture the dependencies of time data and has correlation characteristics. It usually includes state equations and output equations to describe the internal state and output of the system. This module first maps the features to one dimension, and the linear steady-state system is represented by the hidden state Mapping to N-dimensional latent states Mapping to output These systems are usually expressed as linear ordinary differential equations, which are given by:

[0044] in, Is the current state The derivative of . is the state transition matrix that describes how the state changes over time; is the input matrix that controls how the input affects the state change; Is the output matrix, which represents the output generated according to the current state.

[0045] In one example, the embedded image is omnidirectionally and continuously scanned according to multiple scanning trajectories to obtain multiple scanning sequences, including: starting from the first row and first column of the embedded image, omnidirectionally and continuously scan row by row to obtain a first scanning sequence; starting from the first row and first column of the embedded image, omnidirectionally and continuously scan column by column to obtain a second scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally and continuously scan row by row to obtain a third scanning sequence; starting from the last row and last column of the embedded image, omnidirectionally and continuously scan column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0046] In this example, the omnidirectional continuous scan is performed in four different trajectories. Figure 4 It is a schematic diagram of omnidirectional continuous scanning in an embodiment of the present invention. The embedded image includes 9 patch images, 1 to 9. When the end of a row (or column) is reached, the mark to be scanned in the next step will be the mark in the next adjacent column (or row), rather than the mark on the opposite side. Then continue scanning in the opposite direction until the scan is completed. The scan sequences of the scan extension include: 123654789, 147852369, 987456321, 963258741. Then the S6 block processes the four different sequences independently. Finally, the four different sequences are added and combined and outputted, so that each pixel in the image can effectively integrate the information of all other pixels in the image, thereby establishing a global receptive field in two-dimensional space and enhancing the ability to extract image features. During the continuous scanning process, the current mark can be located in one of four possible directions relative to its previous mark. This is because the model only relies on Identifying the precise spatial relationship between adjacent markers poses a challenge. We design a set of learnable parameters { }( By correlating these parameters with the data Sequence addition can enhance directional information during scanning. The direction-aware update formula can ensure that the global features of the image are accurately preserved during scanning. The calculation formula is as follows: ; ; ; in, Used to indicate four different sequence directions, Represents the discretization , represent.

[0047] This implementation uses downsampling to reduce the height and width of the input features to reduce the computational complexity, while increasing the number of channels to enhance the expressiveness of defect features. The number of channels in each stage of the model of the present invention is [C, 2C, 4C, 8C] respectively.

[0048] The computational complexity of the omnidirectional continuous scanning block in this embodiment is linear. Compared with the computational complexity of the Transformer quadratic form, the number of model parameters is effectively reduced, making RVMNet lightweight. The calculation formulas of the attention mechanism (self-attention) and SSM are as follows: ; ; in, Indicates the sequence length, corresponding to the length and width of the resolution, represents the feature dimension, and N represents a fixed parameter, which is set to 16 by default. Attention mechanism and sequence length There is a quadratic relationship between SSM and sequence length The linear relationship and reduction in computational complexity enable Vision Mamba to be extended to applications with large sequence lengths, improving the efficiency of rail surface defect detection tasks.

[0049] In this embodiment, the multiple embedded images and the multi-level feature information are fused to obtain fused features, including: upsampling the multi-level feature information to obtain hierarchical features with the same scale as the embedded images; for each embedded image, pixel-by-pixel splicing the embedded image and the hierarchical features with the same scale to obtain spliced ​​features; and fusing the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0050] In this embodiment, when performing feature fusion, Figure 3As shown, it includes a cross-channel feature fusion block, a splicing operation, an upsampling layer and a deep convolution layer. The feature fusion module fuses the multi-level feature information and detects defects of different sizes by combining shallow and deep features. The process includes: first, the spatial size of the low-resolution feature map in the multi-level feature information is restored by an upsampling operation, and then a splicing operation is performed to splice the feature map of the upsampled spliced ​​feature with the feature map of the high-resolution embedded image pixel by pixel, thereby maintaining the detail information and fusing features of different scales. The cross-channel feature fusion block uses a convolution operation to fuse features from different channels, so that the relevant information between different channels can be effectively complementary, thereby enhancing the representation ability of defect features. In addition, the deep convolution operation deeply extracts local information and high-level features of image features. The feature fusion module improves the spatial resolution of the feature map through multi-stage cross-channel feature fusion blocks, splicing operations, upsampling layers and deep convolution layers, and finally outputs the fusion of multi-level features. This method not only ensures the effective transmission of multi-scale information, but also further enhances the expression ability of the model, realizes the accurate detection of defects of different scales, and improves the detection performance of the model.

[0051] Optionally, before inputting the fusion feature into the target RVMNet model, it also includes: collecting a set of rail surface defect images; performing data enhancement on the rail surface defect image set to obtain a sample set; dividing the sample set into a training set, a validation set and a test set; using the training set to train the initial RVMNet model, and calculating the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; adjusting the parameter value of the initial RVMNet model based on the classification loss and the regression loss to obtain an intermediate RVMNet model; using the validation set to cross-validate the intermediate RVMNet model, and adjusting the parameter value of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; using the test set to test the target RVMNet model to obtain a qualitative index value and a quantitative index value; judging whether the qualitative index value and the quantitative index value both meet the preset conditions; if the qualitative index value and the quantitative index value both meet the preset conditions, outputting the target RVMNet model.

[0052] In one implementation scenario, Figure 5It is a schematic diagram of a rail surface defect detection method based on Visual Mamba in an embodiment of the present invention. The model training process includes: obtaining a public rail surface defect image set (rail surface defect images and sample labels corresponding to the images); performing data enhancement on the rail surface defect images, and randomly dividing them into a training set, a validation set, and a test set in a ratio of 7:2:1 to obtain a rail surface defect sample set required for the experiment; using the rail surface defect training set to train the initial model, using the validation set to cross-validate the performance of the RVMNet model, and dynamically adjusting the parameters of the model to finally obtain the RVMNet model for rail surface defect detection.

[0053] Rail surface discrete defects datasets (RSDDs datasets), including Type-Ⅰ RSDDs dataset and Type-Ⅱ RSDDs dataset, are obtained. Then, the images are preprocessed and data enhancement techniques such as horizontal flipping, rotation, Gaussian blur, noise addition, brightness change, etc. are used to expand the images to obtain sample sets, which are randomly divided into training set, validation set and test set in a ratio of 7:2:1. The training set is used to train the model, the validation set verifies the performance of the model and dynamically adjusts the parameters of the model, and the test set detects the learning effect of the model.

[0054] The initial model is trained using the track surface defect training set, the performance of the RVMNet model is cross-validated using the validation set, and the parameters of the model are dynamically adjusted, and finally the RVMNet model for track surface defect detection is obtained. The experimental results show that the detection accuracy (Precision) of RVMNet on Type-Ⅰ RSDDs dataset (fast track) and Type-Ⅱ RSDDs dataset (ordinary / heavy-load track) reaches 93.12% and 86.78% respectively, the recall rate (Recall) reaches 88.22% and 62.06% respectively, and the F1 score reaches 90.6% and 72.37% respectively. The number of parameters is 5.98M, achieving high-precision detection and lightweight model.

[0055] During the training process, the model parameters are dynamically adjusted according to the loss value to obtain the optimal RVMNet model, including: calculating the confidence loss function and the bounding box loss function, and the total training loss function adopts a weighted mode, where the confidence loss function is used to calculate the difference between the predicted category and the true category; the bounding box loss function is used to calculate the difference between the predicted bounding box and the true bounding box; where The confidence loss function is used to calculate the difference between the predicted category and the true category. It is usually calculated using the binary cross entropy loss. The calculation formula is as follows: ; in, The value is 0 or 1. Is the output of The probability of When is 1, approaches 1, the value of the loss function approaches 0. On the contrary, Approaching 0, the value of the loss function is very large; The bounding box loss function is used to calculate the difference between the predicted bounding box and the true bounding box. CIoULoss can be used to calculate the bounding box loss function. The loss value The calculation formula is as follows: ; ; Among them, IoU represents the ratio of the intersection and union of the predicted box and the true box; is the Euclidean distance between the center of the predicted box and the true box, is the diagonal length of the bounding box, is a small constant used to avoid division by zero errors, is the weight, w and h are the width and height of the image, is the difference in aspect ratio.

[0056] The weighted calculation results of the confidence loss function and the bounding box loss function are used as the total training loss value , the calculation formula is as follows: ; in Total loss function for model training, confidence loss weight Set to 0.05, the bounding box loss weight Set to 0.75; The confidence loss function and the bounding box loss function guide model optimization by measuring the gap between the model prediction and the actual annotation, thereby improving the performance and accuracy of the model. The confidence loss function and the bounding box loss function are used to evaluate the model and adjust the parameters as a guide in the model optimization process, so that the model can better fit the data.

[0057] In order to evaluate the performance of the model, we used the commonly used evaluation indicators: precision, recall, F1 score and Params. Precision is expressed as the ratio of the number of retrieved relevant items (i.e. correctly classified samples) to the number of retrieved items; Recall is expressed as the ratio of the number of retrieved relevant items to all relevant items; F1 is the harmonic mean of the model, describing the stability of the model. The specific calculation formula is as follows:

[0058]

[0059]

[0060] Adopt the scheme of this embodiment to obtain the public track surface image and the sample label corresponding to the image; input the track surface defect image into the RVMNet model for track surface defect detection; first, the patch segmentation module divides the track surface image into patches, and the patches are encoded into high-dimensional embeddings through linear projection and recorded as labels. Then, the hierarchical omnidirectional continuity scanning module learns the key information such as the category, shape and position of the track surface defects, extracts the multi-level features of the labels, and the feature fusion module fuses the extracted multi-level features. The comprehensive detection module receives the fused feature map and outputs the category information and location information of the defects. Based on the visual Mamba architecture, the automatic recognition and positioning of track surface defects are realized, and the technical problems of poor defect detection performance caused by uneven brightness of track surface images and difficulty in distinguishing defects from background are solved, thereby improving the efficiency and accuracy of track surface defect detection.

[0061] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0062] Example 2 In this embodiment, a track surface defect detection device based on visual Mamba is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, can also be conceived.

[0063] Figure 6 is a structural block diagram of a track surface defect detection device based on visual Mamba according to an embodiment of the present invention, such as Figure 6 As shown, including: An acquisition module 60 is used to acquire a track surface image to be detected; A segmentation module 62, configured to segment the track surface image into a plurality of non-overlapping patch images, and create a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; A scanning module 64, configured to perform omnidirectional continuous scanning on the plurality of embedded images and extract multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; A fusion module 66, configured to fuse the plurality of embedded images and the multi-level feature information to obtain a fusion feature; The detection module 68 is used to input the fusion feature into the target visual mamba-based rail surface defect detection network RVMNet model, and output the defect position and defect category of the rail surface defect in the rail surface image.

[0064] Optionally, the segmentation module includes: a segmentation unit, used to segment the track surface image into a plurality of non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; and an encoding unit, used to encode the two-dimensional feature map in the patch map into multiple embedded images through linear projection.

[0065] Optionally, the encoding unit includes: an encoding subunit, used to encode the two-dimensional feature map in the patch image into a first embedded image of an initial resolution through linear projection; a first sampling subunit, used to downsample the first embedded image to obtain a second embedded image of a first intermediate resolution, and double the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; a second sampling subunit, used to downsample the second embedded image to obtain a third embedded image of a second intermediate resolution, and double the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; a third sampling subunit, used to downsample the third embedded image to obtain a fourth embedded image of a third intermediate resolution, and double the number of channels of the fourth embedded image, wherein the second intermediate resolution is twice the third intermediate resolution.

[0066] Optionally, the scanning module includes: a scanning unit, used to perform omnidirectional continuous scanning on each embedded image according to multiple scanning trajectories to obtain multiple scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; an extraction unit, used to extract a feature map of each of the scanning sequences; a merging unit, used to merge the feature maps of the multiple scanning sequences to obtain global feature information; and a pooling unit, used to perform fast spatial pyramid pooling on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

[0067] Optionally, the scanning unit includes: a first scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the first row and first column of the embedded image to obtain a first scanning sequence; a second scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the first row and first column of the embedded image to obtain a second scanning sequence; a third scanning subunit, used to perform omnidirectional continuous scanning row by row starting from the last row and last column of the embedded image to obtain a third scanning sequence; a fourth scanning subunit, used to perform omnidirectional continuous scanning column by column starting from the last row and last column of the embedded image to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

[0068] Optionally, the fusion module includes: a sampling unit, used to upsample the multi-level feature information to obtain hierarchical features of the same scale as the embedded image; a splicing unit, used to perform pixel-by-pixel splicing of the embedded image and the hierarchical features of the same scale for each embedded image to obtain a spliced ​​feature; and a fusion unit, used to fuse the spliced ​​features of multiple scales through a convolution operation to obtain a fused feature.

[0069] Optionally, the device further includes: an acquisition module, which is used to acquire a track surface defect image set before the detection module inputs the fusion feature into the target RVMNet model; an enhancement module, which is used to perform data enhancement on the track surface defect image set to obtain a sample set; a division module, which is used to divide the sample set into a training set, a validation set and a test set; a training module, which is used to train the initial RVMNet model using the training set and calculate the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; a first adjustment module, which is used to adjust the prediction result based on the classification loss. and regression loss to adjust the parameter values ​​of the initial RVMNet model to obtain an intermediate RVMNet model; a second adjustment module, used to cross-validate the intermediate RVMNet model with the validation set, and adjust the parameter values ​​of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; a testing module, used to test the target RVMNet model with the test set to obtain qualitative indicator values ​​and quantitative indicator values; a judgment module, used to judge whether the qualitative indicator values ​​and the quantitative indicator values ​​both meet the preset conditions; an output module, used to output the target RVMNet model if the qualitative indicator values ​​and the quantitative indicator values ​​both meet the preset conditions.

[0070] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0071] Example 3 An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0072] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing: S1, obtaining the track surface image to be detected; S2, dividing the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; S3, performing omnidirectional continuous scanning on the multiple embedded images, and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; S4, fusing the multiple embedded images and the multi-level feature information to obtain a fused feature; S5, inputting the fusion features into the target visual mamba-based rail surface defect detection network RVMNet model, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0073] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0074] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0075] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0076] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, obtaining the track surface image to be detected; S2, dividing the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; S3, performing omnidirectional continuous scanning on the multiple embedded images, and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; S4, fusing the multiple embedded images and the multi-level feature information to obtain a fused feature; S5, inputting the fusion features into the target visual mamba-based rail surface defect detection network RVMNet model, and outputting the defect position and defect category of the rail surface defect in the rail surface image.

[0077] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0078] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0079] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, controller or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., various media that can store program codes.

[0084] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A track surface defect detection method based on visual Mamba, characterized in that: include: Acquiring a track surface image to be inspected; Segmenting the track surface image into a plurality of non-overlapping patch images, and creating a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; Performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; fusing the multiple embedded images and the multi-level feature information to obtain fused features; The fused features are input into a target visual Mamba-based rail surface defect detection network RVMNet model, and the defect position and defect category of the rail surface defect in the rail surface image are output.

2. The method according to claim 1, characterized in that The track surface image is segmented into a plurality of non-overlapping patch images, and a plurality of embedded images are created based on the patch images, including: Segmenting the track surface image into a plurality of non-overlapping two-dimensional feature maps of the same scale to obtain a patch map; The two-dimensional feature map in the patch image is encoded into multiple embedded images through linear projection.

3. The method according to claim 2, characterized in that Encoding the two-dimensional feature map in the patch image into multiple embedded images by linear projection includes: Encoding the two-dimensional feature map in the patch image into a first embedded image of an initial resolution by linear projection; Downsampling the first embedded image to obtain a second embedded image of a first intermediate resolution, and doubling the number of channels of the second embedded image, wherein the initial resolution is twice the first intermediate resolution; Downsampling the second embedded image to obtain a third embedded image of a second intermediate resolution, and doubling the number of channels of the third embedded image, wherein the first intermediate resolution is twice the second intermediate resolution; The third embedded image is downsampled to obtain a fourth embedded image of a third intermediate resolution, and the number of channels of the fourth embedded image is doubled, wherein the second intermediate resolution is twice the third intermediate resolution.

4. The method according to claim 1, characterized in that: Performing omnidirectional continuous scanning on the multiple embedded images and extracting global feature information includes: For each embedded image, the embedded image is omnidirectionally and continuously scanned according to a plurality of scanning trajectories to obtain a plurality of scanning sequences, wherein each scanning sequence corresponds to a scanning trajectory; Extracting a feature map of each of the scanning sequences; Merging the feature maps of the multiple scanning sequences to obtain global feature information; Fast spatial pyramid pooling is performed on multiple global feature information corresponding to the multiple embedded images to obtain multi-level feature information.

5. The method according to claim 4, characterized in that The embedded image is scanned omnidirectionally and continuously according to a plurality of scanning trajectories to obtain a plurality of scanning sequences, including: Starting from the first row and the first column of the embedded image, performing omnidirectional continuous scanning row by row to obtain a first scanning sequence; Starting from the first row and the first column of the embedded image, omnidirectionally and continuously scanning column by column to obtain a second scanning sequence; Starting from the last row and last column of the embedded image, performing omnidirectional continuous scanning row by row to obtain a third scanning sequence; Starting from the last row and last column of the embedded image, omnidirectional continuous scanning is performed column by column to obtain a fourth scanning sequence, wherein the multiple scanning sequences include the first scanning sequence, the second scanning sequence, the third scanning sequence, and the fourth scanning sequence.

6. The method according to claim 1, characterized in that The multiple embedded images and the multi-level feature information are fused to obtain fused features, including: Upsampling the multi-level feature information to obtain hierarchical features with the same scale as the embedded image; For each embedded image, the embedded image and the hierarchical features of the same scale are concatenated pixel by pixel to obtain a concatenated feature; The concatenated features of multiple scales are fused through convolution operations to obtain fused features.

7. The method according to claim 1, characterized in that Before inputting the fused features into the target RVMNet model, the method further includes: Collect track surface defect image sets; Performing data enhancement on the track surface defect image set to obtain a sample set; Dividing the sample set into a training set, a validation set and a test set; Using the training set to train the initial RVMNet model, and calculating the classification loss and regression loss during the training process, wherein the classification loss is used to characterize the difference between the predicted category and the true category, and the regression loss is used to characterize the difference between the predicted bounding box and the true bounding box; Adjusting the parameter values ​​of the initial RVMNet model based on the classification loss and the regression loss to obtain an intermediate RVMNet model; Cross-validating the intermediate RVMNet model using the validation set, and adjusting parameter values ​​of the intermediate RVMNet model based on the validation result to obtain a target RVMNet model; The target RVMNet model is tested using the test set to obtain qualitative index values ​​and quantitative index values; Determine whether the qualitative index value and the quantitative index value both meet the preset conditions; If both the qualitative index value and the quantitative index value meet the preset conditions, the target RVMNet model is output.

8. A track surface defect detection device based on visual Mamba, characterized in that: include: An acquisition module, used for acquiring a track surface image to be detected; A segmentation module, configured to segment the track surface image into a plurality of non-overlapping patch images, and create a plurality of embedded images based on the patch images, wherein each embedded image corresponds to a resolution; A scanning module, used for performing omnidirectional continuous scanning on the multiple embedded images and extracting multi-level feature information, wherein the multi-level feature information includes feature information of embedded images of all resolutions; A fusion module, used for fusing the multiple embedded images and the multi-level feature information to obtain a fusion feature; The detection module is used to input the fusion feature into the target visual mamba-based rail surface defect detection network RVMNet model, and output the defect position and defect category of the rail surface defect in the rail surface image.

9. A storage medium, characterized in that: A computer program is stored in the storage medium, wherein the computer program is configured to execute the steps of the method for detecting track surface defects in any one of claims 1 to 7 when the computer program is run.

10. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface, and the memory communicate with each other via a communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the steps of the method for detecting track surface defects according to any one of claims 1 to 7 by running a program stored in a memory.

Images