Surface defect detection method, device and readable storage medium for rail transit

Abstract

The application discloses a surface defect detection method and device for rail transit and a readable storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a video stream, and performing image preprocessing on the video stream to obtain an image sequence; wherein the to-be-detected objects in the video stream comprise a train box body and / or a rail; based on the image sequence, an improved RT-DETR model is called to generate a defect detection result of the to-be-detected objects in each image; wherein the improvement comprises a residual block, and the residual block comprises a first channel segmentation layer, n Fourier modulation attention (FMA) modules, a cross-stage local connection (CSP) structure and a first feature splicing layer. The application improves the accuracy of surface defect detection.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, specifically relating to a surface defect detection method, equipment, and readable storage medium for rail transit. Background Technology

[0002] With the rapid development of rail transit systems, high-speed railways, urban rail transit, and heavy-haul freight railways have become important components of the modern transportation system. However, during long-term operation, external components such as train carriages and rails are susceptible to environmental corrosion, mechanical fatigue, vibration, and impact, leading to structural defects such as rust and cracks, which puts the safety, stability, and reliability of train operation to the test.

[0003] In traditional technologies, surface defect detection of the aforementioned external components is achieved through computer vision and deep learning techniques. For example, surface defect detection can be performed using a Real-Time Detection Transformer (RT-DETR).

[0004] However, the RT-DETR model still faces several challenges when applied to surface defect detection tasks, such as the RT-DETR (R18) model when used for joint surface defect detection of train car bodies and rails. Specifically, the environment in which train car bodies and rails are located experiences drastic changes in lighting and complex backgrounds, and defects such as rust and cracks are characterized by large scale variations, blurred boundaries, and fine textures. The RT-DETR (R18) model uses a ResNet18 feature extraction network, whose serial residual connections and fixed downsampling structure easily lead to the loss of shallow detail features of the target during transmission. Therefore, this model has weak feature capture capabilities for small-scale defects and fine-textured defects, meaning it lacks sensitivity to small targets and weak texture features, resulting in poor detection performance. Summary of the Invention

[0005] The purpose of this application is to provide a surface defect detection method, device, and readable storage medium for rail transit, which can improve the accuracy of surface defect detection.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a surface defect detection method for rail transit, the method comprising: A video stream is acquired, and image preprocessing is performed on the video stream to obtain an image sequence; wherein, the objects to be detected in the video stream include train carriages and / or railway tracks; Based on the image sequence, the improved RT-DETR model is invoked to generate defect detection results for the objects to be detected in each image; The improved RT-DETR model's backbone network includes at least multiple combinations of convolutional blocks and residual blocks, with each combination cascaded. Each combination includes one convolutional block and one residual block. The residual block includes at least a first channel segmentation layer, n Fourier Modulated Attention (FMA) modules, a cross-stage local connection (CSP) structure, and a first feature concatenation layer, where n ≥ 2. The first channel segmentation layer performs channel-averaged segmentation on the input features to obtain two segmented features, which are then input into the first feature concatenation layer. One of the segmented features is also input into the FMA module for modulation to obtain modulated features, which are then input into the first feature concatenation layer based on the CSP structure. The first feature concatenation layer performs feature concatenation processing on all inputs to this layer.

[0007] In a second aspect, embodiments of this application provide a computer device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Thirdly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] In this embodiment, by combining the global frequency domain modeling capability of the FMA module with the efficient gradient flow design of the CSP structure, the backbone network's ability to capture high-frequency details and local texture information is enhanced without significantly increasing computational overhead. This allows the backbone network to maintain a lightweight design while possessing stronger texture representation capabilities and retaining more small pixel features, making it better suited for extracting crack and corrosion features. Specifically, the model effectively preserves image detail information during multiple downsampling processes, enabling it to possess both a macroscopic view of defect distribution within the global scene and the ability to finely distinguish multi-scale corrosion and crack details, thereby improving the accuracy of surface defect detection. Attached Figure Description

[0011] Figure 1 This is a schematic flowchart of a surface defect detection method for rail transit provided in some embodiments of this application; Figure 2a These are some embodiments of the present application providing images of the test results for train carriages; Figure 2bThese are images of inspection results for railway tracks provided in some embodiments of this application; Figure 3 This is a structural block diagram of the CSP-FMA module provided in some embodiments of this application; Figure 4 These are structural block diagrams of the HLFS-AIFI module provided in some embodiments of this application; Figure 5 This is a structural block diagram of the HLFS-SA module provided in some embodiments of this application; Figure 6 These are structural block diagrams of EPGO related to some embodiments of this application; Figure 7 This is a structural block diagram of an improved RT-DETR model provided in some embodiments of this application; Figure 8 These are internal structural diagrams of a computer device provided in some embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] In one exemplary embodiment, this application proposes a surface defect detection method for rail transit. The surface defect detection method for rail transit provided by this application will be described in detail below with reference to the accompanying drawings and specific embodiments and application scenarios.

[0015] like Figure 1 As shown, the method includes steps 102-104. Wherein: Step 102: Acquire a video stream and perform image preprocessing on the video stream to obtain an image sequence; wherein, the objects to be detected in the video stream include train carriages and / or railway tracks.

[0016] In some embodiments, the video stream is acquired by an image acquisition device deployed on the side of the track, and can be a real-time video data stream.

[0017] In some embodiments, the step of performing image preprocessing on the video stream to obtain an image sequence includes: Keyframes are extracted from the video stream.

[0018] The extracted keyframes are then denoised.

[0019] Adjust the denoised keyframes to the preset size and perform normalization to obtain the image sequence.

[0020] This image preprocessing method can transform raw, diverse input video streams into a set of high-quality, standardized static images, thereby providing unified and clean input data for subsequent feature extraction and detection networks.

[0021] In some embodiments, to achieve a dynamic balance between real-time performance and the integrity of critical information, a time-interval-based uniform sampling strategy can be used for keyframe extraction.

[0022] Specifically, firstly, a fixed time interval is defined. Combined with the frame rate of the input video Through formula Get the number of sampling interval frames ,in, The time interval is determined by the speed of the train being monitored, for example, the train's speed entering the station, with a time interval of 0.3 seconds; then, starting from the beginning frame of the video stream, according to... Periodically perform frame skipping operations, that is, every interval One frame is selected as a keyframe, and the final image sequence used for subsequent object detection tasks is constructed, denoted as... .in, The total number of frames extracted is determined by the total video duration and the sampling interval. Joint decision; For the first One keyframe.

[0023] In some embodiments, since the images extracted directly from frames have problems such as dynamic visual blurring and large contrast changes, it is necessary to perform denoising processing on the above image sequences to ensure the quality of the input data for the deep learning model.

[0024] Specifically, Gaussian blur can be used for noise reduction, and the Gaussian blur formula is shown in formula (1).

[0025] in, Represents points in the original image The pixel value after Gaussian blurring, i.e., the value of that point. The new value after smoothing; Point conduct The pixel value after the neighborhood offset is usually selected from a square or circular neighborhood; A Gaussian kernel is used to determine the degree of influence of each pixel in the neighborhood on the center pixel. The neighborhood radius determines the size of the filter; For relative The offset, For relative The offset.

[0026] In some embodiments, to accommodate the input requirements of deep learning models, it is necessary to ensure that all images are of consistent size. Specifically, the steps for adjusting the denoised keyframes to a preset size and performing normalization are as follows: First, the images of all frames are adjusted to a preset size, such as 640*640 pixels. This embodiment does not specifically limit the preset size.

[0027] Second, the values ​​in each channel of the resized keyframe are converted into standardized values ​​using a normalization formula to fit the numerical range during model training. The keyframe can be an RGB image. The normalization formula is shown in formula (2).

[0028] in, These are the pixel values ​​of the image after normalization. It is the average of all channels. It represents the standard deviation of each channel.

[0029] It should be noted that normalization operations help accelerate model training and convergence.

[0030] Step 104: Based on the image sequence, call the improved RT-DETR model to generate defect detection results for the objects to be detected in each image.

[0031] In some embodiments, such as Figure 2a and Figure 2b As shown, the defect detection results include a preselected box containing the target category confidence score and bounding box coordinates.

[0032] The improved RT-DETR model's backbone network (i.e., feature extraction network) includes at least multiple combinations of convolutional blocks and residual blocks, with each combination cascaded. Each combination includes one convolutional block and one residual block. The residual block includes at least a first channel segmentation layer, n Fourier Modulation Attention (FMA) modules, a Cross Stage Partial (CSP) structure, and a first feature concatenation layer, where n ≥ 2. The first channel segmentation layer is used to perform channel-average segmentation on the input features of the first channel segmentation layer, obtaining two segmented features, which are then input into the first feature concatenation layer. One of the segments is also input into the FMA module for modulation, obtaining modulated features, and based on the CSP structure, each modulated feature is input into the first feature concatenation layer. The first feature concatenation layer is used to perform feature concatenation processing on all inputs of this layer.

[0033] In this embodiment, the improvement to the feature extraction network of the RT-DETR model refers to replacing the ResNet18 structured feature extraction network with an improved CSPDarknet. The improved CSPDarknet includes at least multiple combinations of convolutional blocks and residual blocks, where the residual blocks refer to the CSP-FMA (Cross Stage Partial-Fourier Modulation Attention) module. It can be understood that this CSPDarknet employs a hierarchical, progressive forward propagation architecture to gradually extract multi-scale features from the input image.

[0034] This embodiment combines the global frequency domain modeling capability of the FMA module with the efficient gradient flow design of the CSP structure, thereby enhancing the backbone network's ability to capture high-frequency details and local texture information without significantly increasing computational overhead. This allows the backbone network to maintain a lightweight design while possessing stronger texture representation capabilities and retaining more small pixel features, making it better suited for extracting crack and corrosion features. Specifically, it enables the model to effectively preserve image detail information during multiple downsampling processes, giving it both a macroscopic view of defect distribution across the global scene and the ability to finely distinguish multi-scale corrosion and crack details, thus improving the accuracy of surface defect detection.

[0035] In some embodiments, the residual block further includes two convolutional layers; one convolutional layer is used to perform convolution processing on the input features of the residual block and serve as the input features of the first channel segmentation layer; the other convolutional layer is used to perform convolution processing on the output features of the first feature splicing layer to obtain the output features of the residual block.

[0036] like Figure 3 As shown, Figure 3 This is a schematic diagram of the CSP-FMA module provided in this embodiment. Where h*w*c is the feature map size, Conv is the convolutional layer, Split is the channel segmentation layer, and Concat is the feature concatenation layer.

[0037] In this embodiment, after the image enters the backbone network, it first undergoes rapid downsampling and initial transformation through a convolutional layer to capture low-level texture features such as edges, corners, and points, forming a high-resolution, highly detailed but semantically shallow feature representation.

[0038] Subsequently, the backbone network enters a stage centered around the CSP-FMA module. This stage first performs progressive downsampling and feature transformation through convolutional layers, thereby changing the scale and dimension of the feature map to reduce computational complexity. Then, the CSP-FMA module constructs and outputs feature maps with both global features and local details, preventing the loss of crack and corrosion features during multiple downsampling processes. This process is repeated multiple times during feature extraction, each time halving the spatial size of the feature map, increasing the channel dimension, and gradually expanding the receptive field and enhancing the semantic hierarchy of the features, thus obtaining multi-level features of the image.

[0039] Ultimately, the backbone network outputs a set of feature maps with decreasing spatial resolution, forming a feature pyramid. In this pyramid, shallow features retain rich spatial details, suitable for locating small-scale targets; deep features carry highly abstract semantic information, beneficial for recognizing large-scale objects. This allows for better handling of defects with large scale variations.

[0040] Here is an example: like Figure 3 As shown, after the input enters the CSP-FMA module, the channel dimension of the overall feature map is first reduced through convolutional layers to change the computational complexity. Then, the channels are evenly divided into two paths to achieve cross-stage gradient propagation. The main path consists of n stacked FMA modules. The FMA modules effectively compensate for the receptive field limitation of convolution through cross-domain feature fusion and the dynamic weight generation mechanism of depthwise separable dynamic convolution (DConv), and achieve efficient mixing and recalibration between feature channels.

[0041] The CSP-FMA module employs a densely concatenated strategy, concatenating the basic path features with the output features of each FMA module through channels. Finally, a 1*1 convolutional layer reduces the dimensionality of the concatenated high-dimensional feature map and enables information exchange between channels, enhancing feature representation capabilities. Through this design, the CSP-FMA module not only retains the computational burden and gradient flow optimization advantages of the CSP architecture, improving the model's multi-level feature representation capabilities, but also significantly enhances the model's ability to model complex textures and global semantics through FMA and DConv.

[0042] In some embodiments, the FMA module includes a Cross-Domain Feature Fusion (CDFF) module, a second channel segmentation layer, a 1*1 Conv, two depthwise separable dynamic convolutions DConv of different sizes, a residual connection structure, a second feature concatenation layer, and a third feature concatenation layer.

[0043] In this embodiment, the FMA module first obtains a feature map with global semantics and local details through the CDFF module. Then, it expands the number of channels of the CDFF output to 2c using 1*1Conv and then divides it into averages, where c is the number of channels of the input features. Next, it performs convolution operations on the segmented input using two depthwise separable dynamic convolutions of different sizes, and performs feature concatenation operations on the convolution results through a second feature concatenation layer.

[0044] Subsequently, a 1*1 Conv is used to achieve information interaction between the two branches, obtaining feature representations with multi-scale local information, thereby enhancing the model's adaptability to targets of different scales. In addition, in order to preserve the original features of the input, the FMA module performs residual connections, specifically through the third feature concatenation layer, to prevent the loss of features of low-pixel targets such as cracks during depthwise convolution.

[0045] The CDFF module's function is to obtain feature maps with global and local dependencies through cross-domain feature fusion. This CDFF module performs dual-path processing on the normalized input features. One branch uses 2D Discrete Real Fast Fourier Transform, convolution, and Inverse 2D Discrete Real Fast Fourier Transform operations to quickly obtain the global context information of the input from the frequency domain. Specifically: First, the normalized spatial features are projected onto the frequency domain using the 2D discrete real fast Fourier transform to learn the frequency domain feature information. This step can be achieved using the following formulas (3) and (4).

[0046] in, This represents the input to the CDFF module. This indicates that the input will undergo layer normalization. This is the normalized feature map.

[0047] in, and Representing the spatial domain and frequency domain respectively. This indicates that a 2D discrete real fast Fourier transform is performed on the normalized feature map. yes Feature map projected into the frequency domain; This represents the projection from the spatial domain to the frequency domain. and Indicates spatial dimension.

[0048] It should be noted that the 2D discrete real fast Fourier transform is used for... The dimensional transformation utilizes the conjugate symmetry property of the spectrum, retaining only half of it. Therefore, the spatial domain Transformation into the frequency domain / 2.

[0049] Then, the frequency domain features are extracted using 1*1Conv. This step can be achieved using the following formula (5).

[0050] in, This indicates a convolution pair with a 1x1 kernel. Perform convolution operations; Represents frequency domain characteristics, This indicates that feature extraction did not cross domains and the spatial dimension remained unchanged.

[0051] Next, the learned frequency domain features need to be reprojected into the spatial domain through a 2D discrete real fast Fourier inverse transform operation. This step can be achieved by the following formula (6).

[0052] in, Indicates will Perform 2D discrete real inverse fast Fourier transform; The spatial domain representation of the finally learned frequency domain features has global contextual information of the original features; This indicates a projection from the frequency domain to the spatial domain.

[0053] The other branch obtains shallow features of the input through a 1*1 convolution, which can be achieved by the following formula (7).

[0054] in, It represents the learned spatial features, which contain local detail information of the original feature map.

[0055] The features extracted from the two branches are then merged into channels through the fourth feature concatenation layer. Large Kernel Attention (LKA) is used to capture the dependencies between global and local detailed features to achieve global dependency modeling. Finally, a 1*1 convolution is used to achieve cross-channel information interaction and adjust the scale of the feature map to ensure that the input and output feature map sizes remain unchanged.

[0056] In some embodiments, the neck network of the improved RT-DETR model includes a High and Low Frequency Sparse Attention-based Intra-scale Feature Interaction (HLFS-AIFI) module; the HLFS-AIFI module includes at least a High and Low Frequency Sparse Self-Attention (HLFS-SA) module, a first merged channel layer, and a feedforward neural network.

[0057] The HLFS-SA module is used to decouple the low-resolution feature map output from the last stage of the cascaded convolutional block and residual block to obtain high-frequency and low-frequency features. The low-resolution feature map output from the last stage of the cascaded convolutional block and residual block refers to the low-resolution feature map in the aforementioned feature pyramid.

[0058] The first merging channel layer is used to merge the low-resolution feature map, the high-frequency feature and the low-frequency feature to obtain the first merged feature.

[0059] The feedforward neural network is used to perform a fusion operation based on the first merged features to obtain a first fused feature map that has the original feature global information and high and low frequency information associated with it, thereby achieving the purpose of enhancing the spatial information interaction of the input feature map.

[0060] In some embodiments, the feedforward neural network can be a multilayer perceptron (MLP).

[0061] In some embodiments, such as Figure 4 As shown, the HLFS-AIFI module also includes a first normalized Layer Norm layer, a second Layer Norm layer, and a second merged channel layer.

[0062] The first Layer Norm layer is used to perform layer normalization on the first merged features and input the output to the feedforward neural network. After the feedforward neural network propagates forward, it inputs the output to the second merged channel layer. The second merged channel layer performs channel merging on the output of the feedforward neural network and the output of the first Layer Norm layer and inputs the output to the second Layer Norm layer for layer normalization to obtain the final output of the HLFS-AIFI module.

[0063] In some embodiments, such as Figure 5 As shown, the HLFS-SA module includes at least a window structure, a first efficient prompt guide operator (EPGO), a second efficient prompt guide operator, a high-frequency local self-attention (HiFA) branch, and a low-frequency global self-attention (LoFA) branch.

[0064] The windowing structure is used to divide the low-resolution feature map into non-overlapping and equal parts to obtain multiple windowed features.

[0065] The first and second efficient prompting operators are used to generate prompt information based on the windowed features, and to generate a k-value based on the prompt information. The k-value is used to perform sparsity filtering on the first value matrix in the high-frequency local self-attention branch and the value matrix in the low-frequency global self-attention branch based on the Top-k mechanism. Specifically, the Top operator implementing the Top-k mechanism in the high-frequency local self-attention branch is TopMi, and the Top operator implementing the Top-k mechanism in the low-frequency global self-attention branch is TopN.

[0066] It should be noted that when implementing this Top-k mechanism, the top k elements with the highest scores in the value matrix are retained, while the rest are set to zero. This reduces noise interference in the attention operation and retains the most important attention information, thereby reducing redundant information in the attention score matrix (i.e., the value matrix) and improving the computational efficiency of the attention mechanism.

[0067] It should be noted that EPGO can dynamically generate prompts based on input features, and dynamically adjust the value of k. Specifically, for example... Figure 6As shown, the input feature cue information is first obtained through a linear layer and the ReLU activation function; then, a unified expression of the cue information is obtained through a linear layer and Softmax, while restricting the feature values ​​to [0,1]; next, the cue information is flattened into a low-dimensional vector, and the flattened vector is averaged to obtain a global average value, resulting in a scaling factor between 0 and 1. This is compared with the total number of elements in the attention score matrix. Multiplying them together gives the value of k, where, , and The width and height are the dimensions of the attention input. Through the above operations, a mapping relationship between the input and the k value is established, thereby achieving the goal of adaptively adjusting the k value and optimizing the attention matrix selection process.

[0068] The high-frequency local self-attention branch is used to output the high-frequency feature based on the k value, and the low-frequency global self-attention branch is used to output the low-frequency feature based on the k value.

[0069] In some embodiments, refer to Figure 5 The prompt information includes a first prompt information generated by the first efficient prompt guidance operator and a second prompt information generated by the second efficient prompt guidance operator; the k value includes a first k value generated by the first efficient prompt guidance operator and a second k value generated by the second efficient prompt guidance operator; the HLFS-SA module further includes an average pooling layer and a counting layer.

[0070] The counting layer is used to determine the total number of the first elements of the first fusion matrix. The first fusion matrix is ​​formed by the high-frequency local self-attention branch on the first query matrix. and the first key matrix It is obtained by fusion.

[0071] The counting layer is also used to determine the total number of the second elements of the second fusion matrix. The second fusion matrix is ​​formed by the low-frequency global self-attention branch on the second query matrix. Second bond matrix It is obtained by fusion.

[0072] For each window, the first efficient prompt guidance operator flattens the first prompt information corresponding to each window and performs an average calculation operation to obtain the first scaling factor. and calculate each of the first scaling factors The total number of the first element The product of these values ​​yields the first k value for each window.

[0073] The average pooling layer is used to perform average pooling on all windowed features to obtain pooled features; the second prompt information is generated based on the pooled features.

[0074] The second efficient prompting operator flattens the second prompt information and performs a mean calculation to obtain the second scaling factor. and calculate each of the second scaling factors The total number of the second element The product of these yields the second value of k.

[0075] In some embodiments, the high-frequency features are calculated using the following formula (8).

[0076] in, This represents the high-frequency feature; This indicates a splicing operation. Indicates the total number of windows; This indicates a normalization operation; Represents the Top-k operator. Indicates the first The first k value of each window; Indicates the first The first query matrix of each window, Indicates the first Transpose of the first key matrix of each window; This represents the dimension of the row vector of each first query matrix or the dimension of the row vector of the first key matrix in the high-frequency local self-attention branch. Indicates the first The first value matrix of each window.

[0077] The low-frequency characteristics are calculated using the following formula (9).

[0078] in, This indicates the low-frequency characteristic; Represents the Top-k operator. This represents the second value of k; This represents the second query matrix. This represents the transpose of the second bond matrix; This represents the dimension of the row vector of each second query matrix or the dimension of the row vector of the second key matrix in the low-frequency global self-attention branch. This represents the second-valued matrix.

[0079] In some embodiments, refer to Figure 5After obtaining the high-frequency features and low-frequency features respectively, a feature map with both high-frequency and low-frequency feature information is obtained by channel merging. This step can be calculated by the following formula (10).

[0080] in, This indicates the output of the HLFS-SA module; This indicates a channel merging operation.

[0081] It should be noted that, Figure 5 The linear transformation layer in the model is used to map the obtained features back to the same dimension as the input features, so that the dimensions of the high-frequency and low-frequency paths are aligned.

[0082] This embodiment is based on the design concept of high- and low-frequency attention. It introduces an efficient prompting and guidance operator to design a high- and low-frequency sparse self-attention module, thereby improving the Attention-based Intra-scale Feature Interaction (AIFI) module. HLFS-SA decouples the original features into high-frequency and low-frequency branches. The high-frequency branch uses high-frequency local attention to preserve detailed structure and emphasize local gradients and texture changes; the low-frequency branch uses low-frequency global self-attention for global modeling, enhancing the semantic perception of small-pixel targets such as cracks and rust in the deep network. Furthermore, by introducing an efficient prompting and guidance operator, the attention mechanism can be guided to adaptively suppress background noise, thereby reducing the computational cost of attention. In other words, by introducing an efficient prompting and guidance operator, sparsity of high-frequency local self-attention and low-frequency global self-attention is achieved, optimizing the computational cost of the AIFI module in the improved RT-DETR. Simultaneously, through the decoupling and reconstruction of frequency domain features, the AIFI module's ability to express detailed features is strengthened, enabling the model to obtain richer feature representations in low-contrast and multi-scale mixed train body surfaces and rail defects.

[0083] In some embodiments, after obtaining the multi-scale features of the image, the multi-scale feature maps are fused using the Cross-scale Feature Fusion Module (CCFM) of the improved RT-DETR model.

[0084] The CCFM module is the core of the efficient hybrid encoder in the RT-DETR model. It employs lightweight convolution and bidirectional feature aggregation paths to perform channel alignment and uplink / downlink fusion on the multi-scale features output by the backbone network. Through the built-in RepC3 fusion block, it enhances the interaction and integration of features at adjacent scales, efficiently coupling low-level fine-grained location information with high-level semantic information. While avoiding redundant multi-scale attention computations, it provides high-quality feature sequences with uniform resolution and both semantic and detail-oriented features for the subsequent Transformer decoder.

[0085] The decoder of the improved RT-DETR model takes the global feature sequence output by the efficient hybrid encoder as module input, interacts with the multi-layer decoder through a learnable target query, and achieves semantic matching between the query and features by combining a cross-attention mechanism. Finally, it directly outputs a preselected box containing the target category confidence and bounding box coordinates.

[0086] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0087] For ease of understanding, a specific embodiment will be used as an example: like Figure 7 As shown, an image with a size of 640*640 is input into the backbone network of the improved RT-DETR model. The feature pyramid output by the backbone network is input into the CCFM module of the improved RT-DETR model. The last layer in the feature pyramid is processed by the HLFS-AIFI module. After processing by the CCFM module, the final defect detection result is output by the RTDETR decoder.

[0088] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a surface defect detection method for rail transit. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0089] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0090] In one embodiment, a computer-readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps in the above-described method embodiments.

[0091] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0093] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0094] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting surface defects in rail transit, characterized in that, The surface defect detection method for rail transit includes: A video stream is acquired, and image preprocessing is performed on the video stream to obtain an image sequence; wherein, the objects to be detected in the video stream include train carriages and / or railway tracks; Based on the image sequence, the improved RT-DETR model is invoked to generate defect detection results for the objects to be detected in each image; The improved RT-DETR model's backbone network includes at least multiple combinations of convolutional blocks and residual blocks, with each combination cascaded. Each combination includes one convolutional block and one residual block. The residual block includes at least a first channel segmentation layer, n Fourier Modulated Attention (FMA) modules, a cross-stage local connectivity (CSP) structure, and a first feature concatenation layer, where n ≥ 2. The first channel segmentation layer performs channel-averaged segmentation on the input features, obtaining two segmented features, which are then input into the first feature concatenation layer. One of these features is also input to the FMA modules for modulation, resulting in modulated features. Based on the CSP structure, each modulated feature is input into the first feature concatenation layer. The first feature concatenation layer performs feature concatenation processing on all inputs to this layer. The neck network of the improved RT-DETR model includes a high-low frequency sparse-intra-scale feature interaction (HLFS-AIFI) module; the HLFS-AIFI module includes at least a high-low frequency sparse self-attention (HLFS-SA) module, a first merged channel layer, and a feedforward neural network; The HLFS-SA module is used to decouple the low-resolution feature map of the last stage output in the cascaded convolutional block and residual block to obtain high-frequency features and low-frequency features. The first merging channel layer is used to merge the low-resolution feature map, the high-frequency feature, and the low-frequency feature to obtain the first merged feature; The feedforward neural network is used to perform a fusion operation based on the first merged features to obtain a first fused feature map that has the original feature global information and high and low frequency information associated with it. The HLFS-SA module includes at least a window structure, a first efficient prompting and guidance operator, a second efficient prompting and guidance operator, a high-frequency local self-attention branch, and a low-frequency global self-attention branch; The windowing structure is used to divide the low-resolution feature map into non-overlapping and equal parts to obtain multiple windowed features. The first efficient prompting and guidance operator and the second efficient prompting and guidance operator are respectively used to generate prompt information based on the windowed features, and to generate a k value based on the prompt information; the k value is used to perform sparsification filtering on the first value matrix in the high-frequency local self-attention branch and the value matrix in the low-frequency global self-attention branch based on the Top-k mechanism; The high-frequency local self-attention branch is used to output the high-frequency feature based on the k value, and the low-frequency global self-attention branch is used to output the low-frequency feature based on the k value.

2. The surface defect detection method for rail transit according to claim 1, characterized in that, The step of performing image preprocessing on the video stream to obtain an image sequence includes: Keyframes are extracted from the video stream; The extracted keyframes are then denoised. Adjust the denoised keyframes to the preset size and perform normalization to obtain the image sequence.

3. The surface defect detection method for rail transit according to claim 1, characterized in that, The residual block further includes two convolutional layers; one convolutional layer is used to perform convolution processing on the input features of the residual block and serve as the input features of the first channel segmentation layer; the other convolutional layer is used to perform convolution processing on the output features of the first feature splicing layer to obtain the output features of the residual block.

4. The surface defect detection method for rail transit according to claim 1, characterized in that, The FMA module includes a cross-domain feature fusion CDFF module, a second channel segmentation layer, a 1*1 Conv, two depthwise separable dynamic convolutions DConv of different sizes, a residual connection structure, a second feature splicing layer, and a third feature splicing layer.

5. The surface defect detection method for rail transit according to claim 1, characterized in that, The prompt information includes a first prompt information generated by the first efficient prompt guidance operator and a second prompt information generated by the second efficient prompt guidance operator; the k value includes a first k value generated by the first efficient prompt guidance operator and a second k value generated by the second efficient prompt guidance operator; the HLFS-SA module further includes an average pooling layer and a counting layer; The counting layer is used to determine the total number of the first elements of the first fusion matrix; the first fusion matrix is ​​obtained by fusing the first query matrix and the first key matrix by the high-frequency local self-attention branch; The counting layer is also used to determine the total number of the second elements of the second fusion matrix; the second fusion matrix is ​​obtained by fusing the second query matrix and the second key matrix by the low-frequency global self-attention branch; For each window, the first efficient prompting and guiding operator flattens the first prompt information corresponding to each window, performs an average calculation operation to obtain a first scaling factor, and calculates the product of each first scaling factor and the total number of the first elements to obtain the first k value of each window. The average pooling layer is used to perform average pooling on all windowed features to obtain pooled features. The second prompt message is generated based on the pooled features; The second efficient prompting operator flattens the second prompt information and performs a mean calculation to obtain a second scaling factor, and calculates the product of each of the second scaling factors and the total number of the second elements to obtain a second k value.

6. The surface defect detection method for rail transit according to claim 5, characterized in that, The high-frequency features are calculated using the following formula: in, This represents the high-frequency feature; This indicates a splicing operation. Indicates the total number of windows; This indicates a normalization operation; Represents the Top-k operator. Indicates the first The first k value of each window; Indicates the first The first query matrix of each window, Indicates the first Transpose of the first key matrix of each window; This represents the dimension of the row vector of each first query matrix or the dimension of the row vector of the first key matrix in the high-frequency local self-attention branch. Indicates the first The first value matrix of each window; The low-frequency characteristics are calculated using the following formula: in, This indicates the low-frequency characteristic; Represents the Top-k operator. This represents the second value of k; This represents the second query matrix. This represents the transpose of the second bond matrix; This represents the dimension of the row vector of each second query matrix or the dimension of the row vector of the second key matrix in the low-frequency global self-attention branch. This represents the second-valued matrix.

7. A computer device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the surface defect detection method for rail transit as described in any one of claims 1-6.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the surface defect detection method for rail transit as described in any one of claims 1-6.

Images