Segmentation method for rail defect detection based on improved YOLOv10 and SETR

Abstract

The invention discloses a segmentation method for rail defect detection based on improved YOLOv10 and SETR, and the method comprises the steps: S1, collecting image data, carrying out the enhancement processing of the image data, carrying out the segmentation processing, and making a data set; s2, an improved YOLOV10 algorithm model is built and trained; s3, using the improved YOLOV10 model weight to detect image data to be detected, judging whether defects exist or not, if the defects are detected, storing detection frame information of the defects, and outputting a target detection frame and confidence; and S4, transmitting the detection frame information as prompt information into the SETR model, segmenting the target by adopting the SETR model to obtain a segmentation result, and outputting a mask pattern. The improved YOLOv10 algorithm and the SETR algorithm are adopted to detect the rail defects, the calculation complexity and the efficient detection speed are kept, the detection precision is improved, and the robustness of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the field of novel artificial intelligence technology, and in particular relates to a segmentation method for rail defect detection based on improved YOLOv10 and SETR. Background Art

[0002] As the infrastructure of the railway transportation system, the safety of rails is directly related to the safe operation of trains and the safety of people's lives and property. However, rails are prone to cracks, fractures, corrosion and other defects during long-term use. If not discovered and repaired in time, it may lead to serious accidents such as train derailment and overturning, causing significant casualties and property losses. Therefore, the development of an efficient and accurate rail defect detection system has important research significance and practical value for ensuring railway transportation safety and improving railway transportation efficiency.

[0003] At present, the existing technologies are mainly based on deep learning to implement detection algorithms. Object detection is mostly based on mainstream object detection algorithms, and improvements are made in feature extraction, feature fusion, attention mechanism and detection head to enhance the ability to detect small targets. The recursive neural network (CRF-RCNN) proposed by some researchers combines bilateral convolutional networks and conditional random fields with a two-stage extractor, which helps to obtain fine-grained inspection results with smooth constraints. Some researchers have developed a method for locating track surface defects by combining the YOLOv3 network and the MobileNet network. Some researchers have proposed a new lightweight fusion model that combines the advantages of two lightweight network models, SqueezeNet and MobileNetV2, which can significantly reduce the detection time, but its detection accuracy will also be reduced. Some researchers use MobileNetv3 as the backbone network of YOLOv4 to extract image features and apply deep separable convolution at the same time, thereby realizing a lightweight detection network and real-time detection. Some researchers have built a BiFPN-YOLOv8 network model for rail surface defect detection. The rail surface image data is enhanced by Gauss noise transformation and horizontal transformation. In the network construction, the BiFPN-YOLOv8 network model is constructed by introducing the weighted bidirectional pyramid structure BiFPN into the original YOLOv8 network model to improve the performance of rail surface defect detection. Some researchers have built an improved YOLOv5s-CBE network model for rail surface defect detection. This paper introduces a coordinated attention module in the backbone module of the constructed network model to enhance the feature extraction capabilities of channels and spatial positions; in the neck module, a weighted bidirectional feature pyramid structure is used to combine multi-scale feature maps to improve the applicability of the network model; in the head module, the original CIoU is adjusted to an EIoU-type loss function to improve prediction accuracy.

[0004] The above target detection methods have some shortcomings: First, the small target detection algorithm based on traditional computer vision methods has high requirements for the environment, and requires a clear camera field of view and good lighting conditions in order to accurately capture the image information. Therefore, clear images play an important role, and existing lightweight networks often face the problem of reduced accuracy when processing such images; some researchers have proposed combining SqueezeNet and MobileNetV2, which can greatly reduce the detection time, but its detection accuracy will also be reduced. Some researchers have proposed an attention neural network based on the center point estimation (CCEANN) guided by the joint intersection consistency (IoU), which requires the introduction of a large number of parameters in the calculation process, resulting in excessive network weights and reduced model portability. In contrast, the target detection method based on deep learning in computer vision has better flexibility and generalization ability.

[0005] Chinese patent document CN 118711000A discloses a bearing surface defect detection method and system based on improved YOLOv10. The method acquires bearing defect images, enhances and preprocesses image data, constructs a bearing defect data set, designs and improves the YOLOv10 model, trains the improved YOLOv10 model, tests and evaluates model performance, and comprehensively compares multiple models. In this technical solution, the improved YOLOv10 model introduces the EMAttention attention module and the SPPELAN module of YOLOv9, and uses FasterNeT to replace Backbone to construct an improved bearing defect detection model, and the improved bearing defect detection model is an improved YOLOv10 model.

[0006] Chinese patent document CN 117409003 A discloses a detection method for pads of rail damping fasteners, comprising the following steps: collecting image data of rail damping fastener pads, preprocessing images using histogram equalization technology, generating pad image data sets; and extracting features using convolutional neural networks based on the pad image data sets. This technical solution uses advanced convolutional neural networks and support vector machines, i.e., YOLO or SSD object detection algorithms, to locate defects in rail fasteners.

[0007] Chinese patent document CN 112419289A discloses an intelligent detection method for defects of urban subway track fasteners, including building a TensorFlow Slim fine-tuning model to identify fasteners in images, and extracting spatial features, texture features and contour features of fasteners in images respectively; diagnosing fastener defects based on the SVM model, and outputting the diagnosis results at the same time, marking the position of the defective fasteners in the original image according to the output diagnosis structure; calculating the average distance between two adjacent fasteners, and locating the position information of the defective fasteners in combination with the positioning point information when the train takes the image. The technical solution uses the TensorFlow Slim fine-tuning model for target detection.

[0008] Therefore, it is necessary to provide a segmentation method for rail defect detection based on improved YOLOv10 and SETR, which maintains the computational complexity and efficient detection speed while improving the detection accuracy. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a segmentation method for rail defect detection based on improved YOLOv10 and SETR, which maintains the computational complexity and efficient detection speed while also improving the detection accuracy.

[0010] In order to solve the above technical problems, the technical solution adopted by the present invention is: the segmentation method for rail defect detection based on improved YOLOv10 and SETR specifically includes the following steps:

[0011] S1: Collect image data, enhance and segment the image data, and create a data set;

[0012] S2: Build and train an improved YOLOv10 algorithm model to obtain the weight of the improved YOLOv10 algorithm model;

[0013] S3: Use the improved YOLOv10 model weights to detect the image data to be detected and determine whether there are defects. If a defect is detected, save the detection box information of the defect and output the target detection box and confidence level.

[0014] S4: The detection box information is passed into the SETR model as prompt information, the SETR model is used to segment the target, the segmentation result is obtained, and the mask map is output.

[0015] By adopting the above technical solution, improvements are made to YOLOv10, which not only maintains the computational complexity and efficient detection speed, but also improves the detection accuracy, that is, improves the confidence and accuracy.

[0016] Preferably, the specific steps of step S1 are:

[0017] S11: acquiring image data by inputting an image or a video;

[0018] S12: Use image segmentation to segment the image data into small blocks, then use the Copy Paste data enhancement strategy to enhance the segmented image data, and then divide the data set into training set, validation set and test set.

[0019] In terms of data preprocessing, the image segmentation combined with the Copy Paste target detection training strategy is used for data enhancement. First, the image is segmented into small blocks, and some targets are randomly pasted onto the image, input into the network for training, and then input into the large image for detection. This can significantly improve the detection ability of small targets, that is, effectively improve the detection accuracy of the target and reduce the missed detection rate. Because through segmentation, even small targets can be fully reflected in the local area, reducing the risk of being ignored in the large image due to the small size of the target. Because when the image input to the network model is 640*640, if a large image is used to detect a target of about 100 pixels, it is very easy to miss the target. Among them, Copy Paste: is a data enhancement technology, which generates new training samples by copying and pasting a part of the object instances in the training set to the reasonable position of other images. This method can increase the model's ability to recognize the diversity of objects in different scenes, and is particularly helpful to improve the accuracy of small sample categories and bounding box positioning. These technologies are widely used in target detection algorithms to increase data diversity and improve the adaptability and accuracy of models. Their use can make the model better adaptable to various scenarios and objectives and achieve better performance in practical applications.

[0020] Preferably, in the Neck network of the backbone network Backbone network, the full-dimensional dynamic convolution module ODConv is used to replace the traditional convolution module Conv for downsampling, the lightweight downsampling module ADown is referenced, and the rectangular self-calibration module RCM is introduced; ODConv adopts a multi-dimensional attention mechanism; ADown is a lightweight downsampling method; the SPPF module is upgraded to a dual-channel spatial pyramid pooling SPPFCSPC module, and the pyramid slice attention PSA module is updated to the C2PSA structure of YOLOv11, and the dual-level routing attention Biformer is integrated to optimize the C2PSA structure to form a C2PSA_Biformer module;

[0021] The dynamic upsampling method DySample is referenced in the Head output layer. The full-dimensional dynamic convolution module ODConv replaces the traditional convolution module Conv for downsampling, and the lightweight module ADown replaces the SCDown module.

[0022] ODConv adopts a multi-dimensional attention mechanism. This mechanism uses a parallel strategy to learn the complementary attention of the convolution kernel in all four dimensions of the kernel space (i.e., the spatial size, the number of input channels, and the number of output channels of each convolution layer). Compared with Conv, this convolution module reduces the amount of calculation and improves the detection accuracy.

[0023] ADown is a lightweight downsampling method that can effectively reduce the computational cost and the number of parameters while maintaining the accuracy of the model. It provides support for small targets in target detection in terms of edge and small-scale feature detection.

[0024] The rectangular self-calibration module (RCM) captures global context information through the rectangular self-calibration attention mechanism and shape self-calibration, and combines local detail fusion to improve the model's modeling ability and boundary recognition accuracy for foreground objects. In order to solve the problems of multi-scale object detection that are difficult to handle in complex scenes, RCM is applied to the feature extraction stage, and the rectangular self-calibration mechanism is used to improve the network's attention to foreground objects, making detection more accurate.

[0025] SPPFCSPC combines the advantages of SPPF (Spatial Pyramid Pooling Faster) and CSPNet, and uses grouped convolution operations to process feature maps. It divides the input feature maps into several groups and performs independent convolution operations in each group. SPPFCSPC reduces the computational load and the number of parameters while maintaining high performance, with higher parameter efficiency and lower computational load.

[0026] The target detection network often inserts some layers between the backbone and the head output layer. YOLOV10 adds pyramid slice attention (PSA) after SPPF. At this time, the image resolution is low, which is suitable for capturing and processing global information. By introducing the PSA module, YOLOv10 can capture global information more effectively and improve the learning ability and accuracy of the model. The C2PSA module proposed in YOLOV11 combines the CSP (Cross Stage Partial) structure and the PSA (Pyramid SqueezeAttention) attention mechanism, thereby improving the multi-scale feature extraction capability. PSA is updated to the C2PSA structure of YOLOv11 and integrated into the Biformer attention mechanism to form the C2PSA_Biformer module; the core idea of ​​the Biformer attention mechanism is to introduce a two-layer routing attention mechanism. In BiFormer, each image block is associated with a position router. These position routers assign image blocks to upper and lower routers according to specific rules. The upper router is responsible for capturing global context information, while the lower router is responsible for capturing details of local areas. Suitable for detecting small-scale targets and dense targets.

[0027] Head output layer: The output layer mainly adds a small target detection layer; the dynamic upsampling method DySample is referenced, ODConv replaces the traditional Conv for downsampling, and the lightweight module ADown replaces the SCDown module.

[0028] In addition, for some defects, the image resolution will be reduced when inputting the model. YOLO generally uses 3 detection layers of different scales, and adds a small target detection layer to detect smaller targets; the PSA structure in the original YOLOv10 network is updated to the C2PSA structure of YOLOv11, and the Biformer attention mechanism is integrated to form the C2PSA_Biformer module. This improvement not only optimizes the network structure, but also improves the model's ability to extract target features and robustness. The neck structure SPPF of YOLOv10 is replaced by the SPPFCSCP structure. This change not only reduces the amount of model calculation, but also improves the accuracy of target detection; the rectangular self-calibration module RCM is used to improve the model's modeling ability and boundary recognition accuracy for foreground objects; replacing upsampling with DySample requires less computing resources and can achieve image resolution improvement without adding additional burden. ODConv is used to replace the traditional Conv for downsampling. ODConv has a smaller amount of calculation and higher accuracy, further improving the detection speed and performance of the model.

[0029] Preferably, the specific steps of training the YOLOV10 algorithm model are:

[0030] S21: First, mark the data in the training set, input the built model, set the number of training iterations, perform training, and obtain the weight file of the YOLOV10 algorithm model;

[0031] S22: The unlabeled test set is then used to evaluate the trained YOLOV10 algorithm model. The evaluation indicators include precision, mean average precision (mAP), recall rate and F1 score. The calculation formula is:

[0032]

[0033] Among them, TP represents the number of positive samples judged as positive samples, FP represents the number of negative samples judged as negative samples judged as positive samples, FN represents the number of positive samples judged as negative samples, precision represents accuracy, Recall represents recall rate, n represents the number of categories, with recall rate Recall as the horizontal coordinate and precision rate Precision as the vertical coordinate. The curve formed under different thresholds is called PR curve. Since recall rate and precision rate are a set of contradictory indicators, it is necessary to find an indicator to take both into account. Therefore, the area enclosed under the PR curve is taken as the average precision AP as a new indicator. For multiple categories, the AP value of each single category is obtained respectively, and then divided by the total number of categories to obtain the average precision mAP. F1 is the harmonic mean of precision and recall rate. It can balance the influence of the two and is an indicator that comprehensively considers precision and recall rate.

[0034] Preferably, the specific steps of step S3 are:

[0035] S31: Use the trained improved YOLOv10 model to detect each frame of image data in the image data to be detected;

[0036] S32: Determine whether there is a defect in the image; if a defect is detected, save the detection frame information of the defect; if no defect is detected, return to detect the next frame of image data.

[0037] Preferably, the specific steps of segmenting the target using the SETR model in step S4 are:

[0038] S41: First, SETR is used to train the optimization model and set the image data set to be segmented. The input image to be segmented is cut into image blocks of a uniform fixed size, i.e., a patch grid (patch). Each image block is converted into a feature vector through a linear transformation, and then the position information, i.e., position encoding, is added to form a series of patch embedding vectors, i.e., a patch embedded image block sequence is obtained.

[0039] S42: input the patch-embedded image block sequence processed in step S41 into the encoder of the SETR model, perform feature representation learning through multi-layer multi-head self-attention (MSA) and multi-layer perceptron (MLP) blocks, extract features from the input image and perform feature transformation;

[0040] S43: The decoder part of the SETR model maps the encoded features back to the original image space, that is, the 2D encoder output vector is first converted into a 3D feature map, and then the features are fused through a multi-stream network, and finally the image resolution of the 3D feature map is restored by upsampling, and the feature map is output;

[0041] S44: Predict the segmentation mask for each pixel on the feature map output by the decoder; calculate the loss between the segmentation result and the true label, and use the back propagation algorithm to update the model weights to optimize the segmentation performance of the model until the training round is completed;

[0042] S45: Then apply the segmentation mask to the image and draw a bounding box on the image, that is, the model generates a mask map and superimposes it on the original image and visualizes the mask. The encoder of the SETR model is used to extract features from the input image; the decoder is responsible for gradually decoding the output of the encoder into pixel-level predictions; the predictor generates the final semantic segmentation result based on the output of the decoder.

[0043] Preferably, the specific steps of step S41 are:

[0044] S411: cutting the image into a plurality of grid blocks;

[0045] S412: performing vector conversion on the plurality of grid blocks to convert them into a plurality of one-dimensional vectors;

[0046] S413: performing dimension transformation on the one-dimensional vector through a linear projection function, that is, converting it into a feature vector;

[0047] S414: embedding position information, i.e., position encoding, into each grid block to retain the spatial position relationship of the image block, and then combining features and position information to form a series of patch embedding vectors, i.e., obtaining a D-embedded image block sequence.

[0048] Preferably, the specific steps of step S42 are:

[0049] The image block sequence is input into the encoder Transformer Encoder. Transformer Encoder is a model with many layers. Each layer has two parts, including the Multi-Head Self-Attention (MSA) mechanism and the Multi-Layer Perceptron (MLP) block. The Multi-Head Self-Attention (MSA) mechanism is used to capture the long-range dependencies and global context information in the image. The multi-layer structure of Transformer Encoder gradually extracts and transforms features. Each layer may enhance the sensitivity to information of different scales, thereby achieving a deep understanding of the image content. For each layer l, the model calculates the vectors: query, key, value; the calculation formula is:

[0050] query=Z l-1 W Q , key = Z l-1 W K ,value=Z l-1 W V ;

[0051] The query vector is used to match the key vector to calculate the attention weight; the key vector and the query vector perform dot product or similarity calculation to determine the contribution of each input element to the output; the value vector is the weighted sum of the calculated attention weights to generate the output of the attention mechanism; Z l-1 is the output of the previous layer (or initial input), W Q , W K , W V is the parameter matrix learned by the three linear projection layers;

[0052] The self-attention operation calculates the relationship between each part or block to determine which parts are more important and which parts need more attention; self-attention (SA) is expressed as:

[0053]

[0054] Among them, softmax converts the attention score into a probability form to ensure that the sum of all scores is 1, and d is the dimension of (query, key, value);

[0055] The multi-head self-attention mechanism MSA is obtained by splicing m SA operations and then projecting them, that is, calculating multiple attentions. Each head will look at different parts or angles of the image. Combining this information can capture more details and complex relationships in the image. The calculation formula is:

[0056] MSA(Z l-1 )=[SA1 (Z l-1 );SA 2 (Z l-1 );…;SA m (Z l-1 )]W O ;

[0057] Among them, W O is another weight matrix used to combine the results of all heads into a new vector;

[0058] The multi-layer perceptron (MLP) block is a simple neural network structure that further processes the output of multi-head self-attention; the calculation formula is:

[0059]

[0060] It means that the output of each layer is obtained through two parts: the result of multi-head self-attention MSA and the result of multi-head self-attention MSA processed by MLP multi-layer perceptron through residual connection (similar to "residual connection", used to prevent information loss and gradient disappearance); where L represents the sequence length of the image converted to Patch block, C represents the dimension of the input image, and Z l-1 is the output of the previous layer, MSA(Z l-1 ) is the result of the previous layer after multi-head attention, MLP(MSA(Z l-1 ))The multi-layer perceptron processes the results of multi-head self-attention MSA to obtain the output of each layer.

[0061] Preferably, the decoder in step S43 adopts three decoder upsampling methods, the first upsampling method is SETR-Naiv e Image pixel restoration is achieved through simple 1x1 convolution plus bilinear interpolation; the second upsampling method is to adopt a progressive upsampling strategy, alternating between convolutional layers and upsampling operations to gradually increase the resolution of the feature map to the original image size; the third method uses Multi-Level Featur e The multi-level feature aggregation strategy of Aggregation (MLA) takes the feature representations from different Transformer layers as input, performs feature fusion through a multi-stream network, and finally restores the image resolution through upsampling.

[0062] Preferably, the improved YOLOv10 model adds a small target detection layer of 160*160, that is, the scale detection layers of the improved YOLOv10 model are 160×160, 80×80, 40×40, and 20×20 respectively; in step S32, the feature vector is preprocessed first, and the preprocessing includes feature normalization and dimensionality reduction. These preprocessing steps help optimize model performance.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) High accuracy: The combination of image segmentation and Copy-Paste training strategy is introduced. By dividing the image into small blocks and randomly pasting the target part, the model's recognition ability for small targets and complex backgrounds is significantly enhanced, the detection accuracy of small targets is significantly improved, and the missed detection rate is reduced. For some small defect targets, a small target detection head is added to optimize the model's sensitivity to tiny defects, further improving the overall detection accuracy. The PSA structure in the original YOLOv10 network is updated to the C2PSA structure of YOLOv11, and the Biformer attention mechanism is integrated to form the C2PSA_Biformer module. This innovation not only optimizes the network structure, but also improves the model's ability to extract target features and robustness. The SPPF structure of YOLOv10 is replaced by the SPPFCSPC structure. This change not only reduces the amount of model calculation, but also improves the accuracy of target detection. ODConv is used to replace the traditional Conv for downsampling. ODConv has a smaller amount of calculation and higher accuracy, further improving the detection speed and performance of the model. The rectangular self-calibration module RCM is used to improve the modeling ability and boundary recognition accuracy of the foreground object; replacing upsampling with DySample requires less computing resources and can improve image resolution without adding extra burden, and the lightweight downsampling ADown is used to replace the SCDown module;

[0065] (2) SETR image segmentation: Combined with target segmentation, the target detection box information output by the model is saved and input into the SETR model for segmentation; this more accurately outlines the boundary of the target, distinguishes subtle features in the complex background, and can provide more accurate positioning information;

[0066] (3) Multi-task learning: Integrate YOLOv10 to form a multi-task learning framework, where each module collaborates with each other and shares features, improving the performance of the entire system.

[0067] (4) Universality: The object detection method based on YOLOv10 can be applied to various types of cameras and scenes, and has strong versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of the overall algorithm flow in the segmentation method for rail defect detection based on improved YOLOv10 and SETR of the present invention;

[0069] Figure 2A network structure diagram of an improved YOLOv10 algorithm in a segmentation method for rail defect detection based on improved YOLOv10 and SETR of the present invention;

[0070] Figure 3 It is a flow chart of the SETR segmentation algorithm in the segmentation method for rail defect detection based on improved YOLOv10 and SETR of the present invention;

[0071] Figure 4 It is a network structure diagram of the SETR model in the segmentation method of rail defect detection based on improved YOLOv10 and SETR of the present invention;

[0072] Figure 5 This is a defect detection segmentation effect diagram in the segmentation method for rail defect detection based on improved YOLOv10 and SETR of the present invention. DETAILED DESCRIPTION

[0073] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0074] The known technical contents underlying the present invention include the YOLOv10 target detection algorithm and the SETR semantic segmentation algorithm.

[0075] (1) YOLOv10 target detection algorithm:

[0076] YOLOv10 technical features: YOLOv10 is a single-stage target detection algorithm. The structure of the algorithm is based on the previous YOLO model and adds some new improvement ideas. In the past, the optimization of YOLO was mainly focused on architecture design, optimization goals, and data enhancement strategies. However, no one tried to remove the post-processing step NMS, which is not conducive to the end-to-end deployment of the YOLO series and will increase the inference time. Compared with the previous YOLOv9 algorithm, the YOLOv10 target detection algorithm has the following improvements:

[0077] Dual assignment strategy: Previous YOLO usually utilized a one-to-many label strategy to assign multiple positive samples to each instance. The one-to-many assignment generates rich supervisory signals, promotes optimization and achieves excellent performance. However, this makes YOLO rely on NMS post-processing to select the best detection results, resulting in unsatisfactory inference efficiency when deployed. Although previous studies explored one-to-one matching to suppress redundant predictions, it led to weak supervision problems during training, and the optimization accuracy and inference time could not reach the optimal level. YOLOv10 introduces a dual assignment strategy that not only utilizes one-to-many rich supervisory signals to train and optimize model parameters to ensure model accuracy, but also adopts one-to-one inference to avoid NMS post-processing and thus increase inference speed. The need for NMS is eliminated, resulting in faster and more efficient target detection YOLOv10.

[0078] Pyramid Slice Attention (PSA): An efficient Partial Self-Attention (PSA) module is proposed; first, the features are divided into two parts in the channel after 1x1 convolution. Only one part is input into the NPSA block consisting of a multi-head self-attention module MHSA and a feed-forward network (FFN). The two parts are then connected and fused through 1x1 convolution. PSA is placed only after the 4th stage with the lowest resolution, avoiding the excessive overhead caused by the squared computational complexity of self-attention. In this way, the global representation learning ability can be introduced into the YOLO model at a low computational cost, which significantly enhances the model's capabilities and improves performance.

[0079] (2) Main features of the SETR semantic segmentation algorithm

[0080] 1) Encoder: The SETR model adopts the Encoder-Decoder architecture. Different from the CNN encoder in the traditional semantic segmentation model, SETR uses a pure Transformer as the encoder.

[0081] 2) Sequence-to-sequence prediction task: SETR regards semantic segmentation as a sequence-to-sequence prediction task. The encoder first decomposes the input image into a fixed-size patch grid, and each patch is converted into a feature vector through a linear embedding layer to form a series of patch embedding vectors. These vectors are input into the Transformer and feature representation learning is performed through multi-layer multi-head self-attention (MSA) and multi-layer perceptron (MLP) blocks.

[0082] 3) Decoder design: The SETR model designs a variety of decoder upsampling methods, including Naive Upsampling, Progressive Upsampling, and Multi-Level Feature Aggregation. These decoder designs enable SETR to flexibly restore image resolution and make full use of the feature information of each layer of Transformer to improve segmentation performance.

[0083] 4) Stronger ability to capture contextual information: Due to the self-attention mechanism in the Transformer structure, the SETR model can better capture the contextual information in the image, making it more robust when processing complex scenes.

[0084] Example: Figure 1 As shown, the segmentation method for rail defect detection based on improved YOLOv10 and SETR specifically includes the following steps:

[0085] S1: Collect image data, enhance and segment the image data, and create a data set;

[0086] The specific steps of step S1 are:

[0087] S11: acquiring image data by inputting an image or a video;

[0088] S12: Use image segmentation to segment the image data into small blocks, then use the Copy Paste data enhancement strategy to enhance the segmented image data, and then divide the data set into training set, validation set and test set; the data set includes the rail defects we need to detect; first segment the image into small blocks, and then train it. This can significantly improve the detection ability of small targets, because through segmentation, even small targets can be fully reflected in the local area, reducing the risk of being ignored in the large image due to the small size of the target. Because when the network model inputs a 640*640 image, if a large image is used to detect an object of about 100 pixels, it is very easy to miss the object. Specifically, first, a large number of images in real scenes such as railway tracks are collected, and image segmentation and Copy Paste data enhancement are used. Compared with the original image, small-block images contain less background and redundant information, which can focus more on feature learning of the target itself, alleviate overfitting, and generate a small target data set. At the same time, common data enhancement methods such as Mosaic data enhancement and adaptive anchor box calculation are used to enhance the data set, realize the generalization ability of the enhanced model, and improve the accuracy of model detection.

[0089] S2: Build and train the improved YOLOV10 algorithm model to obtain the weight of the improved YOLOV10 algorithm model;

[0090] like Figure 2 As shown, when building the improved YOLOv10 algorithm model in the step S2, the full-dimensional dynamic convolution module ODConv is used in the Neck network in the backbone network Backbone network to replace the traditional convolution module Conv for downsampling, the lightweight downsampling module ADown is quoted, and the rectangular self-calibration module RCM is introduced; wherein the full-dimensional dynamic convolution module ODConv adopts a multi-dimensional attention mechanism; ADown is a lightweight downsampling method; the SPPF module is upgraded to a dual-channel spatial pyramid pooling SPPFCSPC module, and the pyramid slice attention PSA module is updated to the C2PSA structure of YOLOv11, and the dual-level routing attention Biformer is integrated to optimize the C2PSA structure to form a C2PSA_Biformer module;

[0091] The dynamic upsampling method DySample is used in the Head output layer. The full-dimensional dynamic convolution module ODConv replaces the traditional convolution module Conv for downsampling, and the lightweight module ADown replaces the SCDown module.

[0092] The full-dimensional dynamic convolution module ODConv adopts a multi-dimensional attention mechanism. This mechanism uses a parallel strategy to learn the complementary attention of the convolution kernel in all four dimensions of the kernel space (i.e., the spatial size, the number of input channels, and the number of output channels of each convolution layer). Compared with the traditional convolution module Conv, this convolution module reduces the amount of calculation while improving the detection accuracy;

[0093] The lightweight downsampling module ADown is a lightweight downsampling method that can effectively reduce the computational cost and the number of parameters while maintaining the accuracy of the model. This provides support for small targets in target detection in terms of edge and small-scale feature detection.

[0094] The rectangular self-calibration module (RCM) captures global context information through the rectangular self-calibration attention mechanism and shape self-calibration, and combines local detail fusion to improve the model's modeling ability and boundary recognition accuracy for foreground objects. In order to solve the problems of multi-scale object detection that are difficult to handle in complex scenes, the rectangular self-calibration module (RCM) is applied to the feature extraction stage. Through the rectangular self-calibration mechanism, the network's attention to foreground objects is improved, making detection more accurate.

[0095] SPPFCSPC combines the advantages of SPPF module (Spatial Pyramid Pooling Faster) and CSPNet, and uses grouped convolution operations to process feature maps. It divides the input feature maps into several groups and performs independent convolution operations in each group. SPPFCSPC reduces the computational load and the number of parameters while maintaining high performance, and has higher parameter efficiency and lower computational load.

[0096] The target detection network often inserts some layers between the backbone and the head output layer. YOLOV10 adds pyramid slice attention (PSA) after SPPF. At this time, the image resolution is low, which is suitable for capturing and processing global information. By introducing the PSA module, YOLOv10 can capture global information more effectively and improve the learning ability and accuracy of the model. The C2PSA module proposed in YOLOV11 combines the CSP (Cross Stage Partial) structure and the PSA (Pyramid SqueezeAttention) attention mechanism, thereby improving the multi-scale feature extraction capability. The PSA module is updated to the C2PSA structure of YOLOv11 and integrated into the Biformer attention mechanism to form the C2PSA_Biformer module; the core idea of ​​the Biformer attention mechanism is to introduce a two-layer routing attention mechanism. In BiFormer, each image block is associated with a position router. These position routers assign image blocks to upper and lower routers according to specific rules. The upper router is responsible for capturing global context information, while the lower router is responsible for capturing details of local areas. Applicable to the detection of small-scale targets and dense targets;

[0097] Head output layer: The output layer mainly adds a small target detection layer; the dynamic upsampling method DySample is used, ODConv replaces the traditional Conv for downsampling, and the lightweight module ADown replaces the SCDown module;

[0098] During feature conversion, the C2f module performs feature conversion on the input data through two convolutional layers (cv1 and cv2); the cv1 convolutional layer transforms the number of channels of the input data from c1 to 2*self.c, and the cv2 convolutional layer transforms the number of channels of the feature map after a series of operations from (2+n)*self.c to c2; these convolution operations help extract features of different levels and abstraction levels in the input data;

[0099] In addition, for some defects, the image resolution will be reduced when inputting the model. YOLO generally has three detection heads, and one more head is added, making it 4 heads. The extra head is used to detect smaller targets. The PSA structure in the original YOLOv10 network is updated to the C2PSA structure of YOLOv11, and the Biformer attention mechanism is integrated to form the C2PSA_Biformer module. This improvement not only optimizes the network structure, but also improves the model's ability to extract target features and robustness. The neck structure SPPF of YOLOv10 is replaced with the SPPFCSCP structure. This change not only reduces the amount of model calculation, but also improves the accuracy of target detection. The rectangular self-calibration module RCM is used to improve the model's modeling ability and boundary recognition accuracy for foreground objects. Replacing upsampling with DySample requires less computing resources and can achieve image resolution improvement without adding additional burden. ODConv is used to replace the traditional Conv for downsampling. ODConv has a smaller amount of calculation and higher accuracy, further improving the detection speed and performance of the model.

[0100] DySample uses a point sampling method to split a point into multiple points to achieve clearer edges. Compared with the traditional kernel-based dynamic upsampler, DySample has fewer parameters, floating-point operations, GPU memory and latency; it is suitable for a variety of intensive prediction tasks;

[0101] In the YOLO algorithm, the Concat module usually appears between different layers of the network to achieve the fusion of feature maps. Specifically, the Concat module concatenates the low-resolution but semantically rich feature map with the high-resolution but less semantically informative feature map to form a larger output feature map;

[0102] The improved YOLOv10 model adds a small target detection layer of 160*160, that is, the scale detection layers of the improved YOLOv10 model are 160×160, 80×80, 40×40, and 20×20 respectively; smaller feature maps have larger receptive fields and richer semantic information, but their local detail features are not obvious, and they are more suitable for large targets. A large amount of downsampling of small targets will lead to the loss of shallow position information, and coupled with the fact that they have fewer pixels, they are prone to insufficient local features, which in turn causes missed detections and false detections. After the improvement, a small target detection layer is added to more effectively capture and identify small targets in images, thereby improving the overall target detection accuracy of the model;

[0103] The specific steps for training the YOLOV10 algorithm model are:

[0104] S21: First, mark the data in the training set, input the built model, set the number of training iterations, perform training, and obtain the weight file of the YOLOV10 algorithm model;

[0105] S22: The unlabeled test set is then used to evaluate the trained YOLOV10 algorithm model. The evaluation indicators include accuracy, average precision (AP), recall rate, F1 score and detection speed. These indicators comprehensively measure the performance of the detection model. The calculation formula is:

[0106]

[0107] Among them, TP represents the number of positive samples judged as positive samples, FP represents the number of negative samples judged as negative samples judged as positive samples, FN represents the number of positive samples judged as negative samples, precision represents accuracy, Recall represents recall rate, n represents the number of categories, with recall rate Recall as the horizontal axis and precision rate Precision as the vertical axis, the curve formed under different thresholds is called PR curve; because recall rate and precision rate are a set of contradictory indicators, it is necessary to find an indicator to take both into account, so the area enclosed under the PR curve is taken as the average precision AP as a new indicator. For multiple categories, the AP value of each single category is obtained respectively, and then divided by the total number of categories to obtain the average precision mAP; F1 is the harmonic mean of precision and recall rate; it can balance the influence of the two and is an indicator that comprehensively considers precision and recall rate;

[0108] S3: Use the improved YOLOV10 model weights to detect the image data to be detected and determine whether there are defects. If a defect is detected, save the detection box information of the defect and output the target detection box and confidence level.

[0109] like Figure 3 As shown, the specific steps of step S3 are:

[0110] S31: Use the trained improved YOLOv10 model to detect each frame of image data in the image data to be detected, that is, to extract features;

[0111] S32: Determine whether there is a defect in the image; if a defect is detected, save the detection frame information of the defect; if no defect is detected, return to detect the next frame of image data; in step S32, the feature vector is first preprocessed, and the preprocessing includes feature normalization and dimensionality reduction processing; these preprocessing steps help optimize model performance;

[0112] S4: The detection box information is passed into the SETR model as prompt information, and the SETR model is used to segment the target to obtain the segmentation result and output the mask map; the network structure diagram of the SETR model is as follows Figure 4 As shown,

[0113] The specific steps of using the SETR model to segment the target in step S4 are:

[0114] S41: First, SETR is used to train and optimize the model, and an image dataset to be segmented is set; the input image to be segmented is cut into image blocks of a uniform fixed size, namely, a patch grid (patch), and each image block is converted into a Patch Embedding through linear projection (Linear Projection), and then the position information, namely, position encoding (PositionEmbedding), is added to form a series of embedding vectors to obtain an embedded image block sequence;

[0115] The specific steps of step S41 are:

[0116] S411: cutting the image into a plurality of grid blocks;

[0117] S412: performing vector conversion on the plurality of grid blocks to convert them into a plurality of one-dimensional vectors;

[0118] S413: performing dimension transformation on the one-dimensional vector through a linear projection function, that is, converting it into a feature vector;

[0119] S414: embedding position information, i.e., position coding, into each grid block to retain the spatial position relationship of the image block, and then combining the features and the position information to form a series of embedding vectors, i.e., obtaining an embedded image block sequence;

[0120] S42: input the patch-embedded image block sequence processed in step S41 into the encoder of the SETR model, perform feature representation learning through multi-layer multi-head self-attention (MSA) and multi-layer perceptron (MLP) blocks, extract features from the input image and perform feature transformation;

[0121] The specific steps of step S42 are:

[0122] The image block sequence is input into the encoder Transformer Encoder. Transformer Encoder is a model with many layers. Each layer has two parts, including the Multi-Head Self-Attention (MSA) mechanism and the Multi-Layer Perceptron (MLP) block. The Multi-Head Self-Attention (MSA) mechanism is used to capture the long-range dependencies and global context information in the image. The multi-layer structure of Transformer Encoder gradually extracts and transforms features. Each layer may enhance the sensitivity to information of different scales, thereby achieving a deep understanding of the image content. For each layer l, the model calculates the vectors: query, key, value; the calculation formula is:

[0123] query=Z l-1 W Q , key = Z l-1 W K ,value=Z l-1 W V ;

[0124] The query vector is used to match the key vector to calculate the attention weight; the key vector and the query vector perform dot product or similarity calculation to determine the contribution of each input element to the output; the value vector is the weighted sum of the calculated attention weights to generate the output of the attention mechanism; Z l-1 is the output of the previous layer (or initial input), W Q , W K , W V is the parameter matrix learned by the three linear projection layers;

[0125] The self-attention operation calculates the relationship between each part or block to determine which parts are more important and which parts need more attention; self-attention (SA) is expressed as:

[0126]

[0127] Among them, softmax converts the attention score into a probability form to ensure that the sum of all scores is 1, and d is the dimension of (query, key, value);

[0128] The multi-head self-attention mechanism MSA is obtained by splicing m SA operations and then projecting them, that is, calculating multiple attentions. Each head will look at different parts or angles of the image. Combining this information can capture more details and complex relationships in the image. The calculation formula is:

[0129] MSA(Z l-1 )=[SA1 (Z l-1 );SA 2 (Z l-1 );...;SA m (Z l-1 )]W O ;

[0130] Among them, W O is another weight matrix used to combine the results of all heads into a new vector;

[0131] The multi-layer perceptron (MLP) block is a simple neural network structure that further processes the output of multi-head self-attention; the calculation formula is:

[0132]

[0133] It means that the output of each layer is obtained through two parts: the result of multi-head self-attention MSA and the result of multi-head self-attention MSA processed by MLP multi-layer perceptron through residual connection (similar to "residual connection", used to prevent information loss and gradient disappearance); where L represents the sequence length of the image converted to Patch block, C represents the dimension of the input image, and Z l-1 is the output of the previous layer, MSA(Z l-1 ) is the result of the previous layer after multi-head attention, MLP(MSA(Z l-1 ))The multi-layer perceptron processes the results of the multi-head self-attention MSA to obtain the output of each layer;

[0134] S43: The decoder part of the SETR model maps the encoded features back to the original image space, that is, the 2D encoder output vector is first converted into a 3D feature map, and then the features are fused through a multi-stream network, and finally the image resolution of the 3D feature map is restored by upsampling, and the feature map is output;

[0135] The decoder in step S43 uses three decoder upsampling methods. The first upsampling method is SETR-Naive, which uses a simple 1x1 convolution plus bilinear interpolation to restore image pixels. The second upsampling method is to use a progressive upsampling strategy, alternately use convolution layers and upsampling operations, and gradually increase the resolution of the feature map to the original image size. The third method uses the Multi-Level Feature Aggregation (MLA) multi-level feature aggregation strategy, takes feature representations from different Transformer layers as input, performs feature fusion through a multi-stream network, and finally restores the image resolution through upsampling. This method can make full use of the feature information of each Transformer layer to further improve the segmentation performance. Finally, the segmentation mask is applied to the image, and a bounding box is drawn on the image;

[0136] S44: Predict the segmentation mask for each pixel on the feature map output by the decoder; calculate the loss between the segmentation result and the true label, and use the back propagation algorithm to update the model weights to optimize the segmentation performance of the model until the training round is completed;

[0137] S45: Then apply the segmentation mask to the image and draw a bounding box on the image, that is, the model generates a mask map and superimposes it on the original image and visualizes the mask. The encoder of the SETR model is used to extract features from the input image; the decoder is responsible for gradually decoding the output of the encoder into pixel-level predictions; the predictor generates the final semantic segmentation result based on the output of the decoder.

[0138] By adopting the above technical solution, the target detection model of the present invention uses an improved YOLO v10 network model in the target detection part to obtain target detection frame information, confidence information and classification information. Special optimization processing is also performed on the data set to ensure that small targets can be effectively captured even in complex backgrounds or low-resolution images, which can reduce the missed detection rate.

[0139] The YOLO v10 model is combined with the SETR segmentation model to iteratively process the stored detection box information. First, each detection box is screened, and low-confidence boxes are removed according to the confidence information to reduce the amount of calculation for subsequent processing and improve the accuracy of segmentation. The SETR model extracts features from the input image and integrates context information through its unique encoder-decoder architecture. The encoder part uses the Transformer structure, which can efficiently capture the global features in the image; the decoder part is responsible for mapping the encoded features back to the original image space to generate an accurate segmentation mask. When making accurate predictions for each bounding box, the segmentation mask generated by the SETR model is used to carefully classify the pixels in each detection box. Since the SETR model has excellent ability to capture image details, it can generate high-quality segmentation results even when facing complex backgrounds or small targets in low-resolution images. To segment the target, the details of the image are extracted through SETR, and accurate predictions are made for each bounding box to obtain the corresponding segmentation mask. The effect comparison chart is as follows Figure 5 As shown, Figure 5 (a) in the figure is the original image collected. Figure 5 (b) in the figure is the detected defect. Figure 5 (c) in the figure is the result after segmentation using the SETR model.

[0140] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A segmentation method for rail defect detection based on improved YOLOv10 and SETR, characterized in that: The specific steps include: S1: Collect image data, enhance and segment the image data, and create a data set; S2: Build and train the improved YOLOV10 algorithm model to obtain the weight of the improved YOLOV10 algorithm model; S3: Use the improved YOLOV10 model weights to detect the image data to be detected and determine whether there are defects. If a defect is detected, save the detection box information of the defect and output the target detection box and confidence level. S4: The detection box information is passed into the SETR model as prompt information, the SETR model is used to segment the target, the segmentation result is obtained, and the mask map is output.

2. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 1, characterized in that: The specific steps of step S1 are: S11: acquiring image data by inputting an image or a video; S12: Use image segmentation to segment the image data, then use the Copy Paste data enhancement strategy to enhance the segmented image data, and then divide the data set into training set, validation set and test set.

3. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 2, characterized in that: When building the improved YOLOv10 algorithm model in the step S2, the full-dimensional dynamic convolution module ODConv is used in the Neck network in the backbone network Backbone network to replace the traditional convolution module Conv for downsampling, the lightweight downsampling module ADown is referenced, and the rectangular self-calibration module RCM is introduced; wherein ODConv adopts a multi-dimensional attention mechanism; ADown is a lightweight downsampling method; the SPPF module is upgraded to a dual-channel spatial pyramid pooling SPPFCSPC module, and the pyramid slice attention PSA module is updated to the C2PSA structure of YOLOv11, and the dual-level routing attention Biformer is integrated to optimize the C2PSA structure to form a C2PSA_Biformer module; The dynamic upsampling method DySample is referenced in the Head output layer. The full-dimensional dynamic convolution module ODConv replaces the traditional convolution module Conv for downsampling, and the lightweight module ADown replaces the SCDown module.

4. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 3, characterized in that: The specific steps for training the YOLOV10 algorithm model are: S21: First, mark the data in the training set, input the built model, set the number of training iterations, perform training, and obtain the weight file of the YOLOV10 algorithm model; S22: The unlabeled test set is then used to evaluate the trained YOLOV10 algorithm model. The evaluation indicators include accuracy, average precision, recall rate and F1 score. The calculation formula is: Among them, TP represents the number of positive samples judged as positive samples, FP represents the number of negative samples judged as negative samples judged as positive samples, FN represents the number of positive samples judged as negative samples, precision represents accuracy, Recall represents recall rate, n represents the number of categories, with recall rate Recall as the horizontal coordinate and precision rate Precision as the vertical coordinate. The curve formed under different thresholds is called PR curve; the area enclosed under the PR curve is taken as the average precision AP as a new indicator. For multiple categories, the AP value of each single category is obtained respectively, and then divided by the total number of categories to obtain the average precision mAP; F1 is the harmonic mean of precision and recall rate.

5. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 3, characterized in that: The specific steps of step S3 are: S31: Use the trained improved YOLOv10 model to detect each frame of image data in the image data to be detected; S32: Determine whether there is a defect in the image; if a defect is detected, save the detection frame information of the defect; if no defect is detected, return to detect the next frame of image data.

6. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 3, characterized in that: The specific steps of using the SETR model to segment the target in step S4 are: S41: First, SETR is used to train the optimization model and set the image data set to be segmented. During the training process, the input image to be segmented is cut into image blocks of a uniform fixed size, i.e., a patch grid. Each image block is converted into a feature vector through a linear transformation, and then the position information, i.e., position encoding, is added to form a series of embedding vectors, i.e., a feature embedding vector sequence is obtained. S42: Input the feature embedding vector sequence processed in step S41 into the encoder of the SETR model, perform feature representation learning through multi-layer multi-head self-attention and multi-layer perceptron blocks, extract features from the input image and perform feature transformation; S43: The encoded features are then mapped back to the original image space through the decoder of the SETR model, that is, the 2D encoder output vector is first converted into a 3D feature map, and then the features are fused through a multi-stream network, and finally the image resolution of the 3D feature map is restored by upsampling, and the feature map is output; S44: predicting a segmentation mask for each pixel on the feature map output by the decoder; By calculating the loss between the segmentation result and the true label, and using the back-propagation algorithm to update the model weights, the segmentation performance of the model is optimized until the training round is completed; S45: Use the trained and optimized model to segment the content in the detection box coordinates, draw the bounding box on the image, and visualize the mask.

7. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 6, characterized in that: The specific steps of step S41 are: S411: cutting the image into a plurality of grid blocks; S412: performing vector conversion on the plurality of grid blocks to convert them into a plurality of one-dimensional vectors; S413: Then, the one-dimensional vector is transformed into a feature embedding vector through a linear projection function; S414: embedding position information, i.e., position encoding, into each grid block to retain the spatial position relationship of the image block, and then combining the features and the position information to form a series of embedding vectors, i.e., obtaining a feature embedding vector sequence.

8. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 7, characterized in that: The specific steps of step S42 are: The feature embedding vector sequence is input into the encoder Transformer Encoder. Transformer Encoder is a model with many layers. Each layer has two parts, including the multi-head self-attention MSA mechanism and the multi-layer perceptron MLP block. The multi-head self-attention mechanism captures the long-range dependency and global context information in the image. The multi-layer structure of Transformer Encoder gradually extracts and transforms features. For each layer l, the model calculates the vectors: query, key, value; the calculation formula is: query=Z l-1 W Q ,key=Z l-1 W K ,value=Z l-1 W V ; The query vector is used to match the key vector to calculate the attention weight; the key vector and the query vector perform dot product or similarity calculation to determine the contribution of each input element to the output; the value vector is the weighted sum of the calculated attention weights to generate the output of the attention mechanism; Z l-1 is the output of the previous layer, W Q , W K , W V is the parameter matrix learned by the three linear projection layers; Self-attention SA is expressed as: Among them, softmax converts the attention score into a probability form to ensure that the sum of all scores is 1, and d is the dimension of (query, key, value); The multi-head self-attention mechanism MSA is obtained by splicing m SA operations and then projecting them, that is, calculating multiple attentions. The calculation formula is: MSA(Z l-1 )=[SA1(Z l-1 );SA2(Z l-1 );...;ARE m (WITH l-1 )]IN O ; Among them, W O is another weight matrix used to combine the results of all heads into a new vector; The multi-layer perceptron MLP block is a neural network structure that further processes the output of multi-head self-attention; the calculation formula is: It means that the output of each layer is obtained through residual connection through two parts: the result of multi-head self-attention MSA and the result of multi-head self-attention MSA processed by MLP multi-layer perceptron; where L represents the sequence length of the image converted to Patch block, C represents the dimension of the input image, and Z l-1 is the output of the previous layer, MSA(Z l-1 ) is the result of the previous layer after multi-head attention, MLP(MSA(Z l-1 ))The multi-layer perceptron processes the results of multi-head self-attention MSA to obtain the output of each layer.

9. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 8, characterized in that: Three decoder upsampling methods are used in the decoder in step S43. The first upsampling method is SETR-Naive, which realizes image pixel restoration through 1x1 convolution plus bilinear interpolation; the second upsampling method is to adopt a progressive upsampling strategy, alternately use convolution layers and upsampling operations, and gradually increase the resolution of the feature map to the original image size; the third upsampling method adopts the Multi-Level Feature Aggregation (MLA) multi-level feature aggregation strategy, takes the feature representations from different Transformer layers as input, performs feature fusion through a multi-stream network, and finally restores the image resolution through upsampling.

10. The segmentation method for rail defect detection based on improved YOLOv10 and SETR according to claim 5, characterized in that: The improved YOLOv10 model adds a small target detection layer of 160*160, that is, the scale detection layers of the improved YOLOv10 model are 160×160, 80×80, 40×40, and 20×20 respectively; in the step S32, the feature vector is first preprocessed, and the preprocessing includes feature normalization and dimensionality reduction processing.

Images