A smart track pavement damage segmentation method based on image instance segmentation

Abstract

A method for intelligent rail pavement damage segmentation based on image instance segmentation relates to the technical field of intelligent rail pavement damage assessment, and includes the following steps: Step 1: Dataset creation; Step 2: Constructing a YOLOv8s-D network suitable for intelligent rail pavement damage segmentation based on an improved YOLOv8s network and configuring hyperparameters; Step 3: GUI design. This invention proposes a YOLOv8s-D network suitable for intelligent rail pavement damage segmentation. Compared with other networks, it can identify magnetic nail damage, marking line damage, cracks, and potholes, effectively improving recognition accuracy under different lighting environments and being applicable to pavement defects of different sizes. It has been verified that this method achieves an optimal segmentation accuracy of 92.7%, enabling automatic intelligent rail pavement damage segmentation and providing technical support for subsequent damage quantification.

Description

Technical Field

[0001] The present invention relates to the technical field of smart track pavement damage assessment, and in particular to a smart track pavement damage segmentation method based on image instance segmentation. Background Art

[0002] Common asphalt pavement damage includes cracks, potholes, and blurred markings. Smart rail pavement damage encompasses not only common asphalt pavement damage but also its unique characteristic—magnetic nail damage. Existing deep learning-based image technology for detecting common pavement damage has achieved a series of successes and results. However, current deep learning-based smart rail pavement damage detection methods still have the following shortcomings: First, they are unable to detect magnetic nail damage (missing nails, protrusions, and depressions) and can only detect the characteristics of common asphalt pavement damage, such as cracks, potholes, and missing or blurred markings. Second, they are significantly affected by the environment; for example, the accuracy and precision of detection vary under different lighting conditions. Finally, they have poor adaptability to pavement defects of varying scales and cannot effectively adapt to pavement defects of varying sizes and shapes. Summary of the Invention

[0003] The present invention provides a method for intelligent rail pavement damage segmentation based on image instance segmentation and proposes a YOLOV8s-D network suitable for intelligent rail pavement damage segmentation. Compared with other networks, it can identify magnetic nail damage, marking line damage, cracks and potholes, effectively improve the recognition accuracy under different lighting environments, and can be applied to pavement defects of different sizes. It has been verified that the present invention has achieved an optimal segmentation accuracy of 92.7%, realizing automatic damage segmentation of intelligent rail pavement and providing technical support for subsequent damage quantification.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] An intelligent track pavement damage segmentation method based on image instance segmentation includes the following steps:

[0006] Step 1: Dataset Creation: Identify the types and characteristics of four types of damage: magnetic nail damage, marking line damage, cracks, and potholes on the smart rail pavement, and construct corresponding datasets for network training and verification.

[0007] Step 2: Based on the improvement of the YOLOv8s network, a YOLOv8s-D network suitable for intelligent rail pavement damage segmentation is constructed, and hyperparameters are configured;

[0008] Step 3: GUI design.

[0009] Preferably, step 1 includes the following specific steps: obtaining photos of smart rail damage through automatic cruising of a drone, all pictures are square, expanding the data set using image augmentation technology, dividing the data set into a training set and a validation set after expansion, labeling the smart rail damage through labelme software, generating a json file, drawing the boundaries of the two types of damage with closed curves, generating a mask file, and exporting a txt file. During the training process, the input image is resized to generate image samples with different aspect ratios.

[0010] Preferably, the step 2 includes the following specific steps: based on the YOLOv8s network architecture, designing a DDCNv4 (Dilated Dense Convolutional Network v4) module to replace the C2F module in the backbone module, adding a BRA (BiLevel Routing Attention) attention mechanism, and adding a DSConv (Depthwise Separable Convolution) convolution module to the Neck layer to replace the ordinary convolution of the Neck layer.

[0011] Preferably, in the step 2, DDCNv4 refers to a convolutional neural network structure that combines dilated convolution and dense connection. The deformable convolution of DDCNv4 is used to dynamically adjust the position of the convolution kernel to better adapt to smart rail pavement defects of different sizes and shapes; the dilated convolution is used to extract the contour information of large-size smart rail pavement defects to ensure the complete detection of the entire picture of smart rail pavement defects, thereby improving the adaptability of the model to detect smart rail pavement defects of different sizes.

[0012] Preferably, in the step 2, in order to solve the problem that the original YOLOv8s model has low recognition accuracy when detecting large differences in lighting conditions and when defects in the smart rail pavement are partially obscured, a BiLevel Routing Attention mechanism is introduced. The BiLevel Routing Attention mechanism simultaneously focuses on low-level and high-level features of the image; the BiLevel Routing Attention mechanism is a dynamic, query-aware sparse attention mechanism, which is used to filter out most irrelevant key-value pairs at the coarse area level so that only a small number of routing areas are retained; secondly, fine-grained token-to-token attention is applied to the union of these routing areas.

[0013] Preferably, in step 2, the BiLevel Routing Attention mechanism includes the following three parts:

[0014] First, the feature map is divided into S×S non-overlapping regions and linearly mapped:

[0015] ;

[0016] Where Q: query, K: key, V: value, X r Represents the regional feature map after division, W q 、W k 、W v are the linear mapping weight matrices for query, key, and value respectively;

[0017] Then, the attention weight is calculated on the coarse-grained Token, and only the Topk regions are taken as relevant regions to participate in the fine-grained calculation;

[0018] ; ;

[0019] Where A r : The coarse-grained attention weight matrix, which represents the correlation between each coarse-grained region (or token); Q r : Coarse-grained query matrix, which is obtained from the original region feature map through linear mapping and is used to represent the information of other regions that the current region needs to pay attention to; K r : Coarse-grained key matrix, which is also obtained through linear mapping and is used to represent the information that other regions can pay attention to; (K r ) T :K r The transposed matrix of Q r Perform matrix multiplication to calculate attention weights; I r : The index set of Top-k regions, which contains the coarse-grained attention weight matrix A r In , for each query region, the indexes of the top k most relevant regions; : A function that selects the region indices corresponding to the top k largest weights from the attention weight matrix;

[0020] Finally, the most relevant Topk coarse-grained regions of each token are used as keys and values ​​for the final operation. To enhance locality, a deep convolution is used on the keys and values:

[0021] ; ; ;

[0022] Where K g : The key matrix for fine-grained operations, which is obtained by aggregating the top-k region index I from the original key matrix Kr The corresponding key vector is obtained; V g : The value matrix of fine-grained operation, which is obtained by aggregating the top-k region index I from the original value matrix V r The corresponding value vector is obtained; : A function that aggregates the corresponding vectors from the original matrix according to the index set; O: The final output feature map or output vector, which is obtained by fine-grained attention operation and local convolution enhancement (LCE); : Attention operation function, used to calculate the attention weight between the query vector and the aggregated key and value vectors, and to obtain the output by weighted summation; Q: The original query matrix or the query vector after some transformation, used to calculate the attention weight with the aggregated key vector; LCE(V): The output of Local Convolutional Enhancement, which is obtained by applying deep convolution to the original value matrix V or the aggregated value matrix Vg, and is used to enhance the locality of the output.

[0023] Preferably, in the step 2, when performing feature fusion of different dimensions, the DSConv convolution decomposes the traditional convolution kernel into two components: a variable quantization kernel (VQK) and a distribution offset, and applies kernel-based and channel-based distribution offsets to maintain the same output as the original convolution, and achieves lower memory usage by storing only integer values, thereby achieving the purpose of improving model speed and reducing the number of parameters.

[0024] Preferably, in step 2, during the quantization process of DSConv, the quantization function takes the number of bits to be quantized by the network as input and uses 2's complement to save the integer value. For a number with a bit length of b, the following relationship exists:

[0025] (1);

[0026] In formula (1), Represents the value of each parameter in the tensor; b: quantization bit, indicating the number of bits used to represent the weight value; Z: integer set; N: natural number set; : The range of weight values ​​after quantization, using 2's complement to represent integer values;

[0027] First, the weights of each convolutional layer are scaled so that the original weights The maximum absolute value of matches the maximum value of the above quantization constraint; the new weight Stored in memory as integer values ​​for subsequent use in training and inference; by replacing ordinary convolution with DSConv, the memory saved for each tensor weight is:

[0028] (2);

[0029] Where p is the memory ratio saved for each tensor weight; b is the number of quantization bits; 32 is the number of bits usually used to represent floating point numbers (such as float32); C i : number of input channels; B: block size or group size, which may be used to process weights in blocks during quantization or grouped convolution; : Indicates the number of blocks after the input channels are divided (rounded up or may vary depending on the actual implementation);

[0030] By moving the VQK value through KDS and CDS, the weights of each block of the pre-trained network will be stretched or rounded to fit the interval in formula (1) and stored in VQK, where the optimal value of KDS is:

[0031] (3);

[0032] Where, ξ: optimal scaling factor (or quantization scale factor); : candidate value of the scaling factor; w qi : quantized weight value; w i : original weight value; B: block size or number of samples, which here indicates the number of weights used to calculate the scaling factor;

[0033] Its closed form is:

[0034] , (4).

[0035] Preferably, the step 3 includes: the GUI interface includes four windows: image file selection, image display, recognition results and file saving.

[0036] Beneficial effects of the intelligent track pavement damage segmentation method based on image instance segmentation of the present invention:

[0037] Based on the image instance segmentation method in deep learning, the present invention constructs an intelligent rail pavement damage image dataset for network training and verification; proposes a YOLOv8s-D network suitable for automatic segmentation of intelligent rail pavement damage, which realizes pixel-level segmentation of cracks, potholes, marking line damage, and magnetic nail damage in a single image; based on the YOLOv8s network architecture, the method replaces the C2F module in the backbone module with the DDCNv4 module, adds the BiLevel Routing Attention mechanism and adds the DSConv convolution module to replace the ordinary convolution of the Neck layer, thereby improving the network segmentation effect and reducing the parameter calculation amount; in actual use, images can be collected through devices such as mobile phones, high-definition cameras and drones, and input into the GUI interface developed by the present invention for automatic damage segmentation, and the results are saved to provide data support for subsequent inspection personnel to quantify the damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a data map taken and collected by the drone of the present invention.

[0039] Figure 2 This is the data processing diagram of the present invention (in the figure: (a) labelme annotation diagram; (b) original image; (c) mask image; (d) label image).

[0040] Figure 3 This is the YOLOv8s-D diagram of the intelligent rail pavement damage segmentation network architecture of the present invention (in the figure, Conv: convolution; DDCNv4: a convolutional neural network structure that combines dilated convolution and dense connection; BR-Attention: a two-layer routing attention mechanism, a neural network attention mechanism; DSConv: distributed shift convolution; Concat: connection; Upsample: upsampling).

[0041] Figure 4 This is the framework diagram of the DDCNv4 module in the backbone network of the present invention (in the figure, conv: convolution; dilation: expansion).

[0042] Figure 5 This is an improved diagram of the attention mechanism of the present invention (in the figure, gather: collection).

[0043] Figure 6 This is the DSConv network architecture of the present invention (in the figure, conv: convolution).

[0044] Figure 7 It is a training result diagram of the present invention.

[0045] Figure 8This is a comparison chart of MIoU of different networks.

[0046] Figure 9 This is a comparison chart of different network segmentation effects.

[0047] Figure 10 This is the result of the ablation test.

[0048] Figure 11 This is the segmentation effect diagram of the crack-bphdr dataset.

[0049] Figure 12 This is the segmentation effect diagram of the CrackForest dataset.

[0050] Figure 13 This is the GUI interface diagram of the Zhigui pavement damage detection system. DETAILED DESCRIPTION

[0051] The following describes in detail the implementation methods of the present invention in a step-by-step manner. This description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0052] In the description of the present invention, it should be noted that the terms "up", "down", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings. They are only for describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, and a specific orientation structure and operation. Therefore, they cannot be understood as limiting the present invention.

[0053] Example 1:

[0054] The present invention provides an intelligent track pavement damage segmentation method based on image instance segmentation, such as Figure 1-13 As shown, the following steps are included:

[0055] Step 1: Dataset Creation: Identify the types and characteristics of four types of damage: magnetic nail damage, marking line damage, cracks, and potholes on the smart rail pavement, and construct corresponding datasets for network training and verification.

[0056] Step 2: Based on the improvement of the YOLOv8s network, a YOLOv8s-D network suitable for intelligent rail pavement damage segmentation is constructed, and hyperparameters are configured;

[0057] Step 3: GUI design.

[0058] Example 2:

[0059] Based on Example 1, this embodiment discloses: Step 1 includes the following specific steps: the data sets are all obtained by automatic cruise shooting of drones on the spot, such as Figure 1 As shown in the figure, a DJI Air2 drone was used to automatically capture 2,000 photos of smart rail damage. All images were taken in square format. To avoid overfitting during model training, the dataset was expanded using image augmentation technology. After expansion, the number of images was 8 times that of the original dataset, and it was divided into a training set and a validation set in a ratio of 7:3. The smart rail damage was then annotated using the labelme software to generate a json file, as shown in the figure. Figure 2 As shown in (a), the boundaries of the two types of damage are drawn with closed curves, mask files are generated and exported as txt files, and the original image, labeled image, generated mask image and visualized image are compared. Figure 2 (b) to Figure 2 As shown in (d); during the training process, in order to enhance the diversity of data and improve the generalization ability of the model, the input image will be resized to generate image samples with different aspect ratios, and the ratios are set to (0.5, 0.75, 1.0, 1.25, 1.5 and 1.75).

[0060] Example 3:

[0061] Based on Examples 1 and 2, this embodiment discloses: Figure 3 As shown in the figure, step 2 includes the following specific steps: based on the YOLOv8s network architecture, a DDCNv4 (Dilated Dense Convolutional Network v4) module is designed to replace the C2F module in the backbone module, a BRA (BiLevel Routing Attention) attention mechanism is added, and a DSConv (Depthwise Separable Convolution) convolution module is added to the Neck layer to replace the ordinary convolution of the Neck layer.

[0062] Example 4:

[0063] Based on Example 3, this embodiment discloses: Figure 4As shown, in the step 2, DDCNv4 refers to a convolutional neural network structure that combines dilated convolution and dense connection. In order to solve the problem that the original YOLOv8 semantic segmentation model has poor adaptability to smart rail pavement defects of different scales, the DDCNv4 module is designed to replace the C2F module in the backbone module; the deformable convolution of DDCNv4 can dynamically adjust the position of the convolution kernel to better adapt to smart rail pavement defects of different sizes and shapes; because the receptive field of ordinary convolution is limited, in order to obtain the global information of large-scale smart rail pavement defects, dilated convolution is introduced; through the above-mentioned dilated convolution, the contour information of large-scale smart rail pavement defects can be better extracted, ensuring the complete detection of the entire picture of smart rail pavement defects, thereby improving the adaptability of the model to detect smart rail pavement defects of different sizes.

[0064] Example 5:

[0065] Based on Example 4, this embodiment discloses: in the step 2, the attention mechanism is a neural network resource allocation scheme used to allocate computing resources to more important tasks; the existing attention module is usually inherited into each block to improve the output from the previous layer; in order to address the problem that the original YOLOv8s model has low recognition accuracy due to large differences in detection lighting conditions and partial occlusion of smart rail pavement defects, the BiLevel Routing Attention mechanism is introduced; the BiLevel Routing Attention mechanism can simultaneously focus on low-level and high-level features of the image; low-level features (such as edges, textures, etc.) help to identify detailed information of asphalt surface defects, while high-level features (such as shape, structure, etc.) help to understand the overall outline and type of smart rail pavement defects.

[0066] Example 6:

[0067] Based on Example 5, this embodiment discloses: Figure 5 As shown, in step 2, the BiLevelRouting Attention mechanism includes the following three parts:

[0068] First, the feature map is divided into S×S non-overlapping regions and linearly mapped:

[0069] ;

[0070] Then, the attention weight is calculated on the coarse-grained Token, and only the Topk regions are taken as relevant regions to participate in the fine-grained calculation;

[0071] ; ;

[0072] Finally, the most relevant Topk coarse-grained regions of each token are used as keys and values ​​for the final operation. To enhance locality, a deep convolution is used on the keys and values:

[0073] ; ; .

[0074] Example 7:

[0075] Based on Example 6, this embodiment discloses: in the step 2, the designed DDCNv4 module can better solve the problem of difficulty in identifying different scales, and the introduction of the BiLevel Routing Attention mechanism can solve the problem of high difficulty in identifying defects in the smart rail pavement and complex backgrounds; but the DDCNv4 module introduces the learning and calculation of the offset, which will increase the complexity and computational overhead of the model reasoning, which increases the computational complexity and inference time of the model, and is not suitable for real-time monitoring of multiple defects in the smart rail pavement; secondly, the calculation and storage of the feature matrix offset requires additional memory, which will increase the demand for hardware resources; in order to reduce the number of model parameters without affecting the model accuracy as much as possible, the DSConv convolution module is used in the Neck layer to replace the ordinary convolution of the Neck layer; when performing feature fusion of different dimensions, the DSConv convolution decomposes the traditional convolution kernel into two components: variable quantization kernel (VQK) and distribution offset, and applies kernel-based and channel-based distribution offset to maintain the same output as the original convolution; by storing only integer values ​​to achieve lower memory usage, the purpose of improving model speed and reducing the number of parameters is achieved. The network structure of DSConv is as follows Figure 6 shown.

[0076] Example 8:

[0077] Based on Example 7, this embodiment discloses:

[0078] In step 2, during the quantization process of DSConv, the quantization function takes the number of bits to be quantized by the network as input and uses 2's complement to save the integer value. For a number with a bit length of b, the following relationship exists:

[0079] (1);

[0080] In formula (1), Represents the value of each parameter in the tensor;

[0081] First, the weights of each convolutional layer are scaled so that the original weights The maximum absolute value of matches the maximum value of the above quantization constraint; the new weight Stored in memory as integer values ​​for subsequent use in training and inference; by replacing ordinary convolution with DSConv, the memory saved for each tensor weight is:

[0082] (2);

[0083] By moving the VQK value through KDS and CDS, the weights of each block of the pre-trained network will be stretched or rounded to fit the interval in formula (1) and stored in VQK, where the optimal value of KDS is:

[0084] (3);

[0085] Its closed form is:

[0086] , (4).

[0087] As mentioned above, by choosing DSConv convolution to replace the ordinary convolution of the Neck layer in YOLOv8, the purpose of speeding up model training and reducing model storage space is achieved.

[0088] Example 9:

[0089] Based on Example 8, this embodiment discloses: Step 3 includes: establishing a GUI interface for intelligent rail pavement damage segmentation to achieve automatic segmentation of damaged targets on the intelligent rail pavement. The GUI interface includes four windows: image file selection, image display, recognition results, and file saving. The GUI main interface is as follows: Figure 13 As shown: Click "Upload Image" to select the image to be segmented and present it in the upper left part of the system. The system then automatically segments the damaged area in the image and obtains the network segmented image in the upper right part of the system. Finally, click "Save Image" to save the file in a customized path. The system helps inspectors make quick judgments on severely damaged areas of the smart rail pavement, take corresponding measures, and provide necessary technical support for subsequent damage quantification.

[0090] Example 10:

[0091] Based on Example 9, this embodiment discloses the implementation of network training and performance evaluation, which is as follows:

[0092] 10.1 Test environment:

[0093] The experimental environment for damage segmentation based on YOLOv8s-D is shown in Table 1:

[0094] Table 1 Test environment configuration table:

[0095] ;

[0096] 10.2 Evaluation indicators:

[0097] In order to quantify the performance of the model, pixel accuracy (PA) and mean intersection over union (MIoU) are set. The former is the ratio of the correct prediction of positive samples to the total correct predictions, and the latter is the ratio of the intersection and union of the true value and the predicted value. The PA calculation method is shown in formula (5). The IoU expression is shown in formulas (6) and (7). The average value of the intersection and union ratio of all categories is MIoU, as shown in formula (8):

[0098] (5);

[0099] (6);

[0100] (7);

[0101] (8);

[0102] In formula (5), the numerator is the sum of all predicted correct pixels, and the denominator is the total number of pixels in the image. FN (False Negative), FP (False Positive), TN (True Negative), and TP (True Positive) are the values ​​in the confusion matrix. In network training, the loss function is used as an indicator to measure the deviation between the predicted value and the true value. This embodiment selects CIou (Complete IoU) as the loss function. On the basis of DIoU (Distance-IoU), it can also consider the aspect ratio of the two rectangles, that is, the similarity of the shapes. The original intention of CIou is to introduce the aspect ratio influencing factor to make up for the deficiency of DIoU in expressing the distance between borders, so that the model can more accurately distinguish between cracks and bursts. By adopting CIou, the model can better learn the characteristics of cracks and bursts, thereby improving the accuracy of segmentation.

[0103] 10.3 Training Results:

[0104] The model's backbone network layer parameters were initialized using weights pre-trained on the ImageNet dataset, which is crucial for improving model performance and accelerating convergence. A checkpoint was saved every 50 iterations to preserve the model weights, and log information was recorded and output at each iteration to track the model's training status in real time. For optimization, the Adam optimizer was used to adjust model parameters to minimize the loss function, and dynamic loss scaling was used to address gradient overflow. The momentum parameters of the Adam optimizer were set to (β1=0.937, β2=0.999), and the learning rate was initialized to 0.01. To prevent model overfitting and reduce its complexity, a weight decay strategy was adopted with a decay coefficient set to 0.0005. The learning rate was adjusted dynamically based on the number of iterations starting from the beginning of training and stopping after 500 iterations. The initial learning rate was set to 0.01. A total of 500 iterations were performed throughout the training process, with verification every 20 iterations.

[0105] Figure 7 (ac) Comprehensive display of training and validation results, including a series of indicators such as box loss, classifyloss-CLS loss, segmentation loss, DFL loss, precision, recall, and meanaverage precision (mAP). "B" and "M" represent box and mask, respectively. A careful inspection of these results reveals the convergence trend of all loss indicators, precision and mAP with the increase of iteration number. From the data results, the loss value gradually decreases and tends to be stable. Especially in the later stage of training process, the loss value hardly changes. The convergence of training process shows that the model effectively learns to minimize the error and improves the prediction accuracy over time. mAP50 represents the average precision calculated at the IoU threshold of 0.5. Figure 7 As can be seen from the table, metrics / mAP50 (Box) is close to 0.9; under the stricter 50-95% IoU (mAP50-95), it also shows performance above 0.8, further verifying that the model has strong segmentation performance and robustness under strict evaluation criteria. In summary, the final loss value is low and stable, achieving a good balance between precision and recall. In addition, the model performs well in segmenting damaged defect targets at different IoU thresholds, effectively learning defect features without overfitting, and achieving good training results.

[0106] Example 11:

[0107] Based on Example 10, this example discloses the implementation part of the comparative test, which is as follows:

[0108] In order to further demonstrate the excellent performance of the YOLOv8s-D model, multiple state-of-the-art models including YOLOv8s, UNet, DeepLabV3, and LR-ASPP were used to train and test the same dataset. By comparing the MIoU of each model, the performance of each network was evaluated in a targeted manner. The training results are shown in the figure below. Figure 8 As shown;

[0109] In order to more intuitively demonstrate the segmentation effect of the model and verify the accuracy of the model in segmenting the damage of the smart rail road surface, Figure 9 The segmentation comparison results of the five networks after training;

[0110] Depend on Figure 8 and Figure 9 It can be clearly seen that:

[0111] (1) The five networks have good detection and segmentation effects on marking lines and pits, but there are large differences in the segmentation of magnetic nail damage and cracks. The YOLOv8s-D network has the best effect, and the MIoU can finally reach 93.5%. It can clearly and accurately detect and segment each crack and each magnetic nail damage. The LR-ASPP network has the worst effect, and the MIoU can only reach 40.2% and then no longer increase. The segmentation integrity of the target object is insufficient, and the detection error rate is high. When multiple damages such as missing nails, pits, and cracks coexist, the YOLOv8s-D network can accurately classify and segment the damage, but the other networks cannot perform effective classification and segmentation. This shows the advantage of instance segmentation network over semantic segmentation network.

[0112] (2) Compared with the unimproved network YOLOv8s, YOLOv8s-D improves MIoU by 3.7%, detects more cracks and magnetic nail damage more accurately, and achieves higher segmentation accuracy;

[0113] In summary, the YOLOv8s-D network proposed in this paper outperforms other networks in terms of overall image performance, edge delineation, and detail capture. By improving the original network structure and compensating for its defects, the YOLOv8s-D network successfully improves the segmentation accuracy of target information and demonstrates strong robustness and stability.

[0114] Example 12:

[0115] Based on Example 11, this embodiment discloses the implementation of the ablation test, which is as follows:

[0116] The purpose of ablation testing is to conduct comparative ablation tests to verify the effectiveness of optimization techniques, ensure that there is no competition or conflict between various methods, optimize resource utilization, and avoid inconsistencies and confusion in model training and decision-making. We use ablation tests on DDCNv4, BRA, and DSConv modules to confirm the effectiveness of the enhancements. To evaluate how these changes affect performance, we use YOLOv8s as a baseline for ablation.

[0117] Table 2 Ablation test data table:

[0118] ;

[0119] From Table 2 and Figure 10 As shown in the figure, the YOLOv8s algorithm adds DDCNv4, BRA and DSConv modules, which have improved the three indicators of parameter quantity, mAP0.5 and FPS. Compared with the YOLOv8s algorithm, the YOLOv8s-D network increased mAP0.5 by 5.9%. After adding DSConv, the FPS (frames Per Second) increased significantly by 19.72. After adding the DDCNv4 module and the BRA attention mechanism, although the FPS decreased, the mAP0.5 increased significantly by 4.3%. The experimental data show that each module of the proposed method has an improvement effect on the model, and the effect of combined use is better than that of using it alone, indicating that the new model is superior to the original YOLOv8s model in the target segmentation task.

[0120] Example 13:

[0121] Based on Example 12, this example discloses the implementation of the data set experiment, which is as follows:

[0122] In order to verify the generalization ability of the network provided by the present invention, a comprehensive test of the network proposed by the present invention was conducted using multiple public crack datasets, including the crack-bphdr dataset and CrackForest. The crack-bphdr dataset includes about 4,000 images of concrete cracks under different backgrounds. These images were taken under different lighting conditions, angles, and crack types to ensure the diversity and generalization ability of the dataset. The segmentation results are shown in Figure 2. Figure 11 As shown in the figure; CrackForest includes 712 asphalt pavement crack images taken under various lighting and photography conditions, including single cracks and cross cracks. The segmentation results of this dataset are shown in the figure. Figure 12 As shown;

[0123] (1) From Figure 11It can be clearly seen that the segmentation performance of the YOLOv8s-D network is excellent, and the predicted MIoU value is between 0.87 and 0.95, which is an excellent effect. This is because the characteristics and background information of the cracks in the crack-bphdr dataset are relatively obvious, making segmentation easier;

[0124] (2) By Figure 12 It can be seen that the MIoU value of using the YOLOv8s-D network to segment a single crack is generally between 0.85 and 0.95, showing good segmentation effect; the MIoU value of segmenting cross cracks and multiple cracks is generally between 0.70 and 0.95, which is lower than the accuracy of single crack segmentation. This is because the crack morphology in this dataset is more complex, which increases the difficulty of the model to segment the cracks from the background, resulting in reduced segmentation efficiency.

[0125] Comprehensive test results on two public crack datasets show that the YOLOv8s-D network has excellent performance and strong generalization ability in crack segmentation tasks. The network can accurately identify and segment cracks of various shapes, scales, and complexities. It not only performs well on specific datasets, but also demonstrates stable and efficient performance in cross-dataset tests.

Claims

1. A smart track pavement damage segmentation method based on image instance segmentation, characterized by: The steps include: Step 1: Dataset Creation: Identify the types and characteristics of four types of damage: magnetic nail damage, marking line damage, cracks, and potholes on the smart rail pavement, and construct corresponding datasets for network training and verification. Step 2: Based on the improvement of the YOLOv8s network, a YOLOv8s-D network suitable for intelligent rail pavement damage segmentation is constructed, and hyperparameters are configured; Step 3: GUI design; Step 4: Use the YOLOV8s-D network to segment the track pavement damage; The step 2 includes the following specific steps: based on the YOLOv8s network architecture, designing a DDCNv4 module to replace the C2F module in the backbone module, adding an attention mechanism, and adding a DSConv convolution module to the Neck layer to replace the ordinary convolution of the Neck layer; In step 2, DDCNv4 refers to a convolutional neural network structure that combines dilated convolution and dense connections. The deformable convolution of DDCNv4 is used to dynamically adjust the position of the convolution kernel; the dilated convolution is used to extract the contour information of large-scale smart rail pavement defects to ensure complete detection of the entire picture of smart rail pavement defects; In step 2, an attention mechanism is introduced to simultaneously focus on low-level and high-level features of the image. The attention mechanism is a dynamic, query-aware sparse attention mechanism that filters out most irrelevant key-value pairs at the coarse region level so that only a small number of routing regions are retained. Secondly, fine-grained token-to-token attention is applied to the union of these routing regions. In step 2, the attention mechanism includes the following three parts: First, the feature map is divided into S×S non-overlapping regions and linearly mapped: ； Then, the attention weight is calculated on the coarse-grained Token, and only the Topk regions are taken as relevant regions to participate in the fine-grained calculation; ； ； Finally, the most relevant Topk coarse-grained regions of each token are used as keys and values ​​for the final operation. To enhance locality, a deep convolution is used on the keys and values: ； ； 。 2. The intelligent track pavement damage segmentation method based on image instance segmentation according to claim 1 is characterized by: The step 1 includes the following specific steps: obtaining photos of smart rail damage through automatic cruising of a drone, all pictures are square, expanding the data set using image augmentation technology, and dividing the data set into a training set and a validation set after expansion, labeling the smart rail damage through labelme software, generating a json file, drawing the boundaries of the two types of damage with closed curves, generating a mask file, and exporting a txt file. During the training process, the input image is resized to generate image samples with different aspect ratios.

3. The intelligent track pavement damage segmentation method based on image instance segmentation according to claim 2 is characterized by: In step 2, when fusing features of different dimensions, the DSConv convolution decomposes the traditional convolution kernel into two components: a variable quantization kernel and a distribution offset. Kernel-based and channel-based distribution offsets are applied to maintain the same output as the original convolution. Lower memory usage is achieved by storing only integer values, thereby improving model speed and reducing the number of parameters.

4. The intelligent track pavement damage segmentation method based on image instance segmentation according to claim 3 is characterized by: In step 2, during the quantization process of DSConv, the quantization function takes the number of bits to be quantized by the network as input and uses 2's complement to save the integer value. For a number with a bit length of b, the following relationship exists: (1)； In formula (1), Represents the value of each parameter in the tensor; First, the weights of each convolutional layer are scaled so that the original weights The maximum absolute value of matches the maximum value of the above quantization function; the new weight Stored in memory as integer values ​​for subsequent use in training and inference; by replacing ordinary convolution with DSConv, the memory saved for each tensor weight is: (2)； By moving the VQK value through KDS and CDS, the weights of each block of the pre-trained network will be stretched or rounded to fit the interval in formula (1) and stored in VQK, where the optimal value of KDS is: (3)； Its closed form is: , (1,B,1,1)slices (4)。 5. The intelligent track pavement damage segmentation method based on image instance segmentation according to claim 4 is characterized by: The step 3 includes: the GUI interface includes four windows: image file selection, image display, recognition results and file saving.

Images