Rail track instance segmentation model establishing method and device, equipment and storage medium

Abstract

The application discloses a railway track instance segmentation model establishing method and device, equipment and a storage medium, relates to the technical field of computer vision and intelligent image processing, and comprises the following steps: acquiring railway track image data, and pre-processing and data enhancement are performed on the image to obtain enhanced track training samples; an edge enhancement algorithm based on a Scharr operator is introduced in the data enhancement stage to strengthen the track region boundary features; an improved YOLO11n-seg instance segmentation network structure is constructed, the network comprises a double-path multi-branch feature pyramid DMBFPN module, a multi-scale edge feature fusion module MSEFF and a decoupled efficient edge perception detection segmentation head DEED-Seg; the enhanced samples are input into the network for training to obtain a railway track instance segmentation model; and the to-be-detected image is input into the model to output an instance segmentation result. Through the fusion of the edge enhancement and the multi-scale feature fusion strategy, high-precision railway track instance segmentation is realized under low calculation amount, and the method has the advantages of high precision, strong real-time performance and easy deployment.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent image processing technology, specifically to a method, apparatus, electronic device, and storage medium for establishing a railway track instance segmentation model. Background Technology

[0002] With the continuous expansion of railway networks and the constant increase in train speeds, railway scene perception technology plays an increasingly important role in intelligent railway systems. Accurate identification of track areas is fundamental to multiple tasks such as track condition analysis, line monitoring, and operational safety assurance; its segmentation accuracy directly affects the reliability of subsequent systems. Therefore, how to achieve high-precision visual segmentation of track areas in complex and ever-changing railway environments has become an important research direction in the field of intelligent visual perception.

[0003] In recent years, deep learning has brought significant breakthroughs to image segmentation tasks, with models such as the YOLO series, Mask R-CNN, and Segment Anything demonstrating excellent performance in various scenarios. YOLOv11-seg, as a next-generation real-time instance segmentation algorithm, achieves a balance between speed and accuracy through structural optimization and a more efficient feature fusion strategy. However, most of these general-purpose models are designed for natural scenes. For slender, regular structures like railway tracks with relatively fixed boundaries, they are prone to problems such as unclear boundaries, missing details, and sensitivity to complex background interference during segmentation.

[0004] On the other hand, railway scenarios exhibit significant environmental complexity, such as strong light reflection, shadow occlusion, sleeper texture interference, and blurring and noise caused by rain and snow. These factors weaken track edge features, making it difficult for models to learn and maintain stable track segmentation capabilities. Furthermore, track targets often exhibit slender structures and high aspect ratios, and traditional convolutional methods have limitations in handling these boundary details. Some high-precision models also have high computational demands, making them unsuitable for deployment in railway front-end equipment. Therefore, existing methods still struggle to balance edge detail extraction, lightweight design, and real-time performance in track scenarios, necessitating a more suitable railway track instance segmentation method, device, electronic equipment, and computer-readable storage medium tailored to the characteristics of track structures. Summary of the Invention

[0005] The purpose of this invention is to provide a railway track instance segmentation method, device, electronic device, and computer-readable storage medium. By constructing an image enhancement module that integrates edge priors, improving the YOLO11-seg backbone structure, and designing a multi-scale edge feature fusion module, high-precision, lightweight, and real-time segmentation of the track area is achieved. By constructing a DMBFPN feature fusion network, designing an MSEFF_C3K2 structure, and introducing an adaptive upsampling operator DySample, an improved DEED-Seg segmentation detection head is developed to enhance boundary recognition capabilities and the positioning accuracy of small-scale track elements.

[0006] The first aspect of this invention provides a method for establishing a railway track instance segmentation model, comprising:

[0007] S1. Obtain the railway track image dataset and perform data preprocessing and enhancement. Employ edge enhancement operations based on the Scharr edge operator to highlight the edge features of the track area.

[0008] S2, based on YOLO11n-seg, introduces a multi-scale edge feature fusion module MSEFF in the backbone network, adopts a dual-path multi-branch feature pyramid DMBFPN in the neck network, and constructs a decoupled and efficient edge-aware detection and segmentation head DEED-Seg to achieve edge-guided multi-scale feature aggregation and high-precision instance segmentation in the orbital region, thus constructing an improved SMDE-YOLO network model.

[0009] S3, The training set and validation set from the edge-enhanced railway track image dataset are input into the improved SMDE-YOLO model for training to obtain the railway track instance segmentation model.

[0010] Preferably, S1 includes:

[0011] S11, edge intensity maps are obtained by calculating the gradient response of the image in the horizontal and vertical directions, and gradient magnitude fusion is used to highlight the details of the track edge;

[0012] S12, then morphological dilation is performed on the edge map to enhance edge continuity, thereby obtaining an enhanced image with higher edge contrast, providing clearer boundary information for subsequent feature extraction.

[0013] Preferably, S11 includes:

[0014] S111, uses the Scharr edge operator to perform edge detection on the input image; where the horizontal convolution kernel... and vertical convolution kernel These are used to extract gradient responses in the horizontal and vertical directions, respectively.

[0015] S112, Based on the convolution results, calculate the gradients in the horizontal and vertical directions respectively, and approximate the edge intensity by weighting to generate a preliminary edge detection map, as shown in equation (1):

[0016] (1)

[0017] in, Represents pixels Edge response strength at the location, and These are the gradient magnitudes of the Scharr operator in the horizontal and vertical directions, respectively.

[0018] S113, Perform a morphological dilation operation on the edge detection map to enhance the connectivity and saliency of the edge structure.

[0019] Preferably, S12 includes:

[0020] S121, Perform bitwise operations on the dilated edge detection map and the corresponding mask map, retaining only the edge response within the track area, to generate the processed edge map, as shown in equation (2):

[0021] (2)

[0022] in, For the mask image, This represents a pixel-by-pixel bitwise AND operation. The edge response results are preserved within the masked area;

[0023] S122, The processed edge map and the original image are pixel-weightedly fused at a ratio of 6:4 to obtain an enhanced image containing significant edge information, as shown in equation (3):

[0024] (3)

[0025] in, Represents the original input image. This is the edge enhancement image after fusion.

[0026] Preferably, S2 includes:

[0027] S21, The MSEFF module is integrated into the backbone network and deeply fused with the C3K2 module in the original YOLO11n-seg structure to extract edge information from feature maps of different scales, and combined with the dual-domain selection mechanism to achieve adaptive filtering of edge features, thereby enhancing the expression of fine-grained boundary features.

[0028] S22, The DMBFPN network structure is adopted in the neck network. Through the DySample dynamic upsampling and weighted fusion mechanism, efficient semantic alignment of multi-scale features and cross-layer information transmission are achieved, thereby improving the network's global feature aggregation capability.

[0029] S23, The detection and segmentation head constructs the DEED-Seg Head, which decouples the detection branch from the segmentation branch to handle the category prediction and mask generation tasks respectively, so as to reduce interference between tasks and improve the recognition accuracy of railway track boundary areas.

[0030] Preferably, S21 includes:

[0031] S211, Edge Magnification Module (EAB), enhances local edge responses in the spatial dimension; firstly, it processes the input feature map... conduct Average pooling is used to smooth local regions, and edge response maps are calculated by difference, as shown in Equation (4):

[0032] (4)

[0033] in, Indicates the input feature map, and These represent the height and width of the feature map, respectively. Number of channels; Indicates with filling Average pooling operation, For edge response feature maps;

[0034] pass Convolution yields a spatial soft mask to adaptively control the enhancement region, as shown in Equation (5):

[0035] (5)

[0036] in, express Convolution operation, for Activation function A spatial weight mask representing the edge region;

[0037] After passing through Convolution and Activate and enhance edge features, as shown in Equation (6):

[0038] (6)

[0039] in, express Convolution operation, This indicates element-wise multiplication. This is the enhanced edge feature map;

[0040] Finally, the learnable scaling factor is used. The enhanced features are superimposed on the input to obtain the enhanced output, as shown in equation (7):

[0041] (7)

[0042] in, These are learnable parameters that control the edge enhancement intensity. Output feature map;

[0043] S212, utilizing a multi-scale edge feature extraction unit, each branch first passes the input features through an adaptive average pooling layer. Compression is performed with pooling sizes of 3, 6, 9, and 12 to obtain contextual features at different scales; multi-scale edge information is then extracted using depthwise separable convolution DSConv; features at each scale are upsampled to their original size via bilinear interpolation and then input into the corresponding lightweight edge enhancer for fine-grained feature enhancement.

[0044] S213 utilizes a local feature extraction unit to extract local texture information of the input features through depthwise separable convolution, and then concatenates it with multi-scale enhanced features to form a fused input.

[0045] S214 utilizes the dual-domain feature selection mechanism (DSM), whose input is the concatenated multi-scale features. The system sequentially passes through the spatial selection module (SSM) and the frequency selection module (FSM) to achieve dynamic fusion and adaptive filtering of spatial and frequency domain features, as expressed in equation (8):

[0046] (8)

[0047] in This represents the output feature map after dual-domain filtering;

[0048] S215 utilizes a feature fusion unit to integrate the features processed by DSM through depthwise separable convolution, outputting a fused edge enhancement feature map to replace the Bottleneck module in the original C3K2 structure, thereby achieving adaptive fusion of multi-scale edge features and enhanced representation of key boundary regions.

[0049] Preferably, S22 includes:

[0050] In the neck network of YOLOv11, the original PAN-FPN structure is replaced with a DMBFPN network structure to achieve efficient semantic fusion of deep and shallow features and cross-layer information alignment, thereby enhancing the ability to identify railway track boundaries.

[0051] The DMBFPN network includes a bidirectional fusion path from top to bottom and from bottom to top, as well as a dynamic upsampling and weighted feature concatenation mechanism. The specific steps are as follows:

[0052] Top-down path fusion stage: fusing the highest-level semantic features conduct Convolutional downsampling yields low-resolution features. The feature is then concatenated with the C2PSA mid-layer feature from the backbone network and input into the C3K2 module for initial fusion, resulting in the fused feature, as shown in equation (9):

[0053] (9)

[0054] in, Indicates a channel splicing operation;

[0055] Dynamic upsampling stage: from high-level features During the upsampling process, the DySample dynamic upsampler is introduced, which uses a point resampling mechanism based on a learnable sampling weight matrix. Adaptive interpolation of key points:

[0056] (10)

[0057] in, For the location on the input feature map pixel values, Indicates the sampling neighborhood range. For learnable interpolation weights, To output the upsampling result;

[0058] DySample learns the optimal sampling distribution through end-to-end training, thereby preserving key texture details at the track edges during upsampling.

[0059] Feature splicing and fusion stage: For the feature layer Upsampled features from DySample Original features after convolution downsampling With shallow edge enhancement features The data is concatenated along the channel dimension and then fed into the C3K2 module for fusion. The expression is:

[0060] (11)

[0061] Bottom-up path fusion stage: During the bottom-up feature reconstruction process, the following steps are executed sequentially. The convolutional downsampling is then concatenated with the fused features of the corresponding layer and refined by the C3K2 module, thereby achieving multi-round bidirectional interaction between deep and shallow features; this process can be formally represented as:

[0062] (12)

[0063] in, This indicates the current level index;

[0064] Output fusion and feature alignment stage: Multi-scale feature layer of the final output By dynamically fusing multiple branches, bidirectional transmission of high-level semantic information and shallow-level detailed information can be achieved.

[0065] in, Indicates the first Layer feature map; This represents an improved residual fusion structure; Edge enhancement feature maps from the MSEFF module; This represents the feature output after fusion;

[0066] Through this DMBFPN structure, the model achieves efficient interaction and detail alignment of multi-scale information while maintaining the lightweight nature of PAN-FPN.

[0067] Preferably, S23 includes:

[0068] S231, Construct the detection branch:

[0069] A lightweight feature enhancement structure is formed by concatenating two sets of DSConv and the high-efficiency channel attention module on the input feature map at each scale.

[0070] On the enhanced feature map, respectively through Convolution yields the regression prediction matrix With classification prediction matrix ;

[0071] The regression output is: The classification output is as follows: ;Will Decoding is performed using focus regression to obtain the bounding box:

[0072] (13)

[0073] The final detection output is:

[0074] (14)

[0075] in, This is the bounding box regression distribution prediction matrix; It is the number of discrete regression channels for each bounding box in DFL; For classification prediction vectors; Number of categories; This represents the distributed focus regression decoding function; This represents the coordinates of the decoded bounding box; This represents the set of detection results.

[0076] S232, Construct split branches:

[0077] The prototype mask is obtained through the Proto prototype generation network. The mask coefficient matrix is ​​generated through a three-layer convolutional structure. The final segmentation mask is obtained by linearly combining the two:

[0078] (15)

[0079] in, As a prototype mask feature, The number of prototype masks. The mask coefficient matrix corresponding to each instance; The spatial length after flattening all feature points; Use a segmentation mask for the final generated instance;

[0080] S233, achieving decoupled collaboration between detection and segmentation tasks:

[0081] By performing independent forward inference for the detection branch and the segmentation branch, the detection regression-classification prediction and mask generation processes are made independent of each other; the detection branch depends on... The bounding box space decoding is performed using strides, as shown in the following formula:

[0082] (16)

[0083] The branch segmentation relies solely on formula (15) to avoid gradient conflicts and improve the segmentation accuracy of the orbital edge region;

[0084] in, Anchor point position matrix; The step size for each feature layer, This refers to the bounding box decoding functions in the YOLO series.

[0085] Preferably, S3 includes:

[0086] S31, the railway track image dataset after edge enhancement is divided into training set and validation set according to a preset ratio, and the images, masks and their enhancement results are loaded synchronously, normalized and batch packaged;

[0087] S32, The training set is input into the SMDE-YOLO network model, the prediction result is obtained through forward inference, and the detection branch and the segmentation branch are jointly optimized based on the preset loss function;

[0088] S33. The model weights are iteratively updated based on the performance metrics of the validation set. The network parameters are adjusted through backpropagation until the training process converges or reaches the preset training rounds, thereby obtaining the final trained model weights.

[0089] A second aspect of the present invention provides a railway track instance segmentation method, comprising: S4, inputting test set images into the railway track instance segmentation model to obtain instance segmentation results of the track region.

[0090] According to the above railway track instance segmentation method, step S4 specifically includes:

[0091] S41, Input the railway track images and their corresponding augmented images from the test set into the trained SMDE-YOLO network model;

[0092] S42, The detection branch and segmentation branch of the model are used to perform forward inference on the input image to generate target category prediction and orbital region mask prediction respectively;

[0093] S43, The mask prediction and detection prediction are fused to generate the final instance segmentation result of the railway track area, and the result is visualized or saved as needed for subsequent analysis and application.

[0094] Thirdly, the present invention also provides a railway track instance segmentation model building device, comprising:

[0095] The data acquisition and processing module is used to acquire railway track image datasets and perform data preprocessing and enhancement. It adopts edge enhancement operation based on the Scharr edge operator method to highlight the edge features of the track area.

[0096] The model building module is used to build an improved SMDE-YOLO network model based on YOLO11n-seg by introducing a multi-scale edge feature fusion module MSEFF in the backbone network, adopting a dual-path multi-branch feature pyramid DMBFPN in the neck network, and constructing a decoupled and efficient edge-aware detection and segmentation head DEED-Seg to achieve edge-guided multi-scale feature aggregation and high-precision instance segmentation in the orbital region.

[0097] The model training and generation module is used to input the training set and validation set from the edge-enhanced railway track image dataset into the improved SMDE-YOLO model for training, thereby obtaining a railway track instance segmentation model.

[0098] Fourthly, the present invention also provides an electronic device, comprising: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions, which, when executed by the processor, implement the railway track instance segmentation model establishment method.

[0099] Fifthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for establishing a railway track instance segmentation model is implemented.

[0100] In a sixth aspect, the present invention also provides a railway track instance segmentation device, comprising: an input module for inputting test set images into the above-mentioned railway track instance segmentation model to obtain instance segmentation results of the track region.

[0101] Compared with existing technologies, the beneficial effects of this invention are as follows: by edge enhancement and multi-scale feature fusion, the accuracy of railway track boundary recognition is significantly improved; by using a detection segmentation head, task interference is reduced and instance segmentation performance is improved; the improved network structure enhances robustness and adaptability in complex environments, providing a high-precision and reliable technical means for intelligent railway monitoring. Attached Figure Description

[0102] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0103] Figure 1 This is a flowchart of a method for establishing a railway track instance segmentation model according to an embodiment of the present invention;

[0104] Figure 2 This is a flowchart of the edge enhancement algorithm provided in an embodiment of the present invention;

[0105] Figure 3 This is a network structure diagram of the improved SMDE-YOLO model provided in this embodiment of the invention;

[0106] Figure 4 This is a network structure diagram of the MSEFF model provided in an embodiment of the present invention;

[0107] Figure 5 This is a structural diagram of DEED-Seg provided in an embodiment of the present invention;

[0108] Figure 6This is a table of ablation experiment results provided in an embodiment of the present invention;

[0109] Figure 7 This is a table of performance experiment results provided in the embodiments of the present invention;

[0110] Figure 8 This is a schematic diagram of a railway track instance segmentation model establishment device provided in an embodiment of the present invention. Detailed Implementation

[0111] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0112] In the description of this invention, unless otherwise expressly stated, the terms "first," "second," "third," etc., are used only to distinguish different steps, modules, or objects and do not represent any order restriction or priority relationship; the terms "comprising" and "including" should be understood as non-exclusive inclusion; the terms "connection," "coupling," and "combination," etc., should be interpreted broadly and can refer to direct or indirect, physical or logical relationships. Descriptions of algorithms, modules, and network structures can be replaced or implemented equivalently without changing the core idea.

[0113] With the continuous expansion of railway networks and the constant increase in train speeds, railway scene perception technology plays an increasingly important role in intelligent railway systems. Accurate identification of track areas is fundamental to multiple tasks such as track condition analysis, line monitoring, and operational safety assurance; its segmentation accuracy directly affects the reliability of subsequent systems. Therefore, how to achieve high-precision visual segmentation of track areas in complex and ever-changing railway environments has become an important research direction in the field of intelligent visual perception.

[0114] In recent years, deep learning has brought significant breakthroughs to image segmentation tasks, with models such as the YOLO series, Mask R-CNN, and Segment Anything demonstrating excellent performance in various scenarios. YOLOv11-seg, as a next-generation real-time instance segmentation algorithm, achieves a balance between speed and accuracy through structural optimization and a more efficient feature fusion strategy. However, most of these general-purpose models are designed for natural scenes. For slender, regular structures like railway tracks with relatively fixed boundaries, they are prone to problems such as unclear boundaries, missing details, and sensitivity to complex background interference during segmentation.

[0115] On the other hand, railway scenarios exhibit significant environmental complexity, including strong light reflection, shadow occlusion, sleeper texture interference, and blurring and noise caused by rain and snow. These factors weaken track edge features, making it difficult for models to learn and maintain stable track segmentation capabilities. Furthermore, track targets often exhibit slender structures and high aspect ratios, limiting the effectiveness of traditional convolutional methods in handling such boundary details. Some high-precision models also have high computational demands, making them unsuitable for deployment in railway front-end equipment. Therefore, existing methods still struggle to balance edge detail extraction, lightweight design, and real-time performance in track scenarios, necessitating a more suitable method, device, electronic equipment, and storage medium for establishing railway track instance segmentation models based on track structural characteristics.

[0116] like Figure 1 As shown, this embodiment provides a method for establishing a railway track instance segmentation model, including the following steps:

[0117] S1, Obtain the railway track image dataset, and preprocess and enhance the data, dividing the dataset into training, validation, and test sets, such as... Figure 2 As shown, an edge enhancement operation based on the Scharr operator is applied to the orbit image to highlight the edge features of the orbit region;

[0118] In a preferred embodiment, S1 includes:

[0119] S11: The orbit edge intensity map is obtained by calculating the gradient response of the image in the horizontal and vertical directions, and the gradient magnitude fusion method is used to highlight the orbit edge details.

[0120] In this embodiment, step S11 includes: S111, performing edge detection on the input image using the Scharr edge operator; wherein, the horizontal convolution kernel and vertical convolution kernel These are used to extract gradient responses in the horizontal and vertical directions, respectively.

[0121] S112, Based on the convolution results, calculate the gradients in the horizontal and vertical directions respectively, and approximate the edge intensity by weighting to generate a preliminary edge detection map, as shown in equation (1):

[0122] (1)

[0123] In equation (1), Represents pixels Edge response strength at the location, and These are the gradient magnitudes of the Scharr operator in the horizontal and vertical directions, respectively.

[0124] S113, Perform a morphological dilation operation on the edge detection map to enhance the connectivity and saliency of the edge structure.

[0125] The technical effect of step S11: By fusing the horizontal and vertical gradient responses to obtain the edge intensity map, the detailed information of the track boundary can be significantly highlighted, and the feature expression quality of the edge region can be improved.

[0126] S12, the orbital edge intensity map is obtained by calculating the gradient response of the image in the horizontal and vertical directions, and the orbital edge details are highlighted by gradient magnitude fusion. The steps include:

[0127] S121, In this embodiment, the dilated edge detection map and the corresponding mask map are performed bitwise, and only the edge response within the track area is retained to generate the processed edge map, as shown in equation (2):

[0128] (2)

[0129] in, For the mask image, This represents a pixel-by-pixel bitwise AND operation. The edge response results are preserved within the masked area;

[0130] S122 performs pixel-level weighted fusion of the processed edge map and the original image at a ratio of 6:4 to obtain an enhanced image containing significant edge information, as shown in Equation (3):

[0131] (3)

[0132] in, Represents the original input image. This is the edge enhancement image after fusion.

[0133] The technical effect of step S12: Dilation of the edge map can enhance the continuity and connectivity of the edges, thereby further improving the model's ability to recognize the complete track contour.

[0134] S2, Construct the improved SMDE-YOLO network model, such as Figure 3 As shown, this model is based on YOLO11n-seg. It achieves edge-guided multi-scale feature aggregation and high-precision instance segmentation in the orbital region by introducing a multi-scale edge feature fusion module (MSEFF) in the backbone network, adopting a dual-path multi-branch feature pyramid (DMBFPN) in the neck network, and constructing a decoupled and efficient edge-aware detection and segmentation head (DEED-Seg Head).

[0135] In a preferred embodiment, S2 includes:

[0136] S21, as Figure 4 As shown, the MSEFF module is integrated into the backbone network and deeply fused with the original YOLO11n-seg C3K2 module to extract edge information from feature maps of different scales. Furthermore, the dual-domain selection mechanism (DSM) is used to adaptively filter edge features, thereby enhancing the fine-grained boundary feature representation, including:

[0137] In the YOLO11 backbone network, the MSEFF module is deeply integrated with the original C3K2 module to construct an improved MSEFF_C3K2 structure, thereby enhancing the feature representation capability and lightweight performance of railway track edge regions.

[0138] S211 utilizes the Edge Boosting Module (EAB) to enhance local edge responses in the spatial dimension. First, the input feature map... conduct Average pooling is used to smooth local regions, and edge response maps are calculated by difference, as shown in Equation (4):

[0139] (4)

[0140] in, Indicates the input feature map, and These represent the height and width of the feature map, respectively. Number of channels; Indicates with filling Average pooling operation, For edge response feature maps;

[0141] pass Convolution yields a spatial soft mask to adaptively control the enhancement region, as shown in Equation (5):

[0142] (5)

[0143] in, express Convolution operation, for Activation function A spatial weight mask representing the edge region;

[0144] After passing through Convolution and Activate and enhance edge features, as shown in Equation (6):

[0145] (6)

[0146] in, express Convolution operation, This indicates element-wise multiplication. This is the enhanced edge feature map;

[0147] Finally, the learnable scaling factor is used. The enhanced features are superimposed on the input to obtain the enhanced output, as shown in equation (7):

[0148] (7)

[0149] in, These are learnable parameters that control the edge enhancement intensity. Output feature map;

[0150] S212 utilizes a multi-scale edge feature extraction unit; each branch first processes the input features through an adaptive average pooling layer. Compression is performed with pooling sizes of 3, 6, 9, and 12 to obtain contextual features at different scales; then, multi-scale edge information is extracted through depthwise separable convolution DSConv; after each scale feature is upsampled to its original size by bilinear interpolation, it is input into the corresponding lightweight edge enhancer for fine-grained feature enhancement; the multi-scale edge feature extraction unit includes several parallel branches.

[0151] S213 utilizes a local feature extraction unit to extract local texture information of the input features through depthwise separable convolution, and then concatenates it with multi-scale enhanced features to form a fused input.

[0152] S214 utilizes the dual-domain feature selection mechanism (DSM), whose input is the concatenated multi-scale features. The system sequentially passes through the spatial selection module (SSM) and the frequency selection module (FSM) to achieve dynamic fusion and adaptive filtering of spatial and frequency domain features, as expressed in equation (8):

[0153] (8)

[0154] in This represents the output feature map after dual-domain filtering;

[0155] By utilizing the feature fusion unit, the features processed by DSM are uniformly integrated through depthwise separable convolution, and the fused edge enhancement feature map is output to replace the Bottleneck module in the original C3K2 structure, thereby realizing the adaptive fusion of multi-scale edge features and the enhanced representation of key boundary regions.

[0156] S22, a DMBFPN structure was designed in the neck network. Through DySample dynamic upsampling and weighted fusion mechanisms, efficient semantic alignment and cross-layer information transfer of multi-scale features were achieved, thereby improving the network's global feature aggregation capability, including:

[0157] S221, Top-down path fusion stage:

[0158] For the highest level semantic features conduct Convolutional downsampling yields low-resolution features. The feature is then concatenated with the C2PSA mid-layer feature from the backbone network and input into the C3K2 module for initial fusion, resulting in the fused feature, as shown in equation (9):

[0159] (9)

[0160] in, Indicates a channel splicing operation;

[0161] S222, Dynamic Upsampling Stage:

[0162] From the perspective of high-level characteristics During the upsampling process, the DySample dynamic upsampler is introduced, which uses a point resampling mechanism based on a learnable sampling weight matrix. Adaptive interpolation of key points:

[0163] (10)

[0164] in, For the location on the input feature map pixel values, Indicates the sampling neighborhood range. For learnable interpolation weights, To output the upsampling result.

[0165] DySample learns the optimal sampling distribution through end-to-end training, thereby preserving key texture details such as track edges during upsampling.

[0166] S223, Feature splicing and fusion stage:

[0167] For feature layer Upsampled features from DySample Original features after convolution downsampling With shallow edge enhancement features The data is concatenated along the channel dimension and then fed into the C3K2 module for fusion. The expression is:

[0168] (11)

[0169] This fusion method integrates global semantics, local details, and significant boundary information, enabling mid-level features to have stronger context awareness and boundary sensitivity.

[0170] S225, Bottom-up path fusion stage:

[0171] In the bottom-up feature reconstruction process, the following steps are executed sequentially. The convolutional downsampling is then concatenated with the fused features of the corresponding layer, and further refined by the C3K2 module, thereby achieving multi-round bidirectional interaction between deep and shallow layer features. This process can be formally represented as:

[0172] (12)

[0173] in, , indicates the current level index.

[0174] S226, Output fusion and feature alignment:

[0175] The final output multi-scale feature layer By dynamically fusing multiple branches, bidirectional transmission of high-level semantic information and shallow-level detailed information is achieved, thereby improving the segmentation accuracy and boundary integrity of orbital targets without significantly increasing computational load.

[0176] in, Indicates the first Layer feature map; This represents an improved residual fusion structure; Edge enhancement feature maps from the MSEFF module; This represents the feature output after fusion.

[0177] By using this DMBFPN structure, the model achieves efficient interaction and detail alignment of multi-scale information while maintaining the lightweight nature of PAN-FPN, significantly improving the segmentation accuracy and edge consistency of the orbital region.

[0178] S23, the detection segmentation head adopts a DEED-Seg structure, such as Figure 5 As shown, the detection branch and segmentation branch are decoupled to handle the category prediction and mask generation tasks respectively, reducing interference between tasks and improving the accuracy of track boundary region recognition, including:

[0179] S231, Construct the detection branch:

[0180] A lightweight feature enhancement structure is formed by concatenating two sets of DSConv and the high-efficiency channel attention module (ECA) on the input feature map at each scale;

[0181] On the enhanced feature map, respectively through Convolution yields the regression prediction matrix With classification prediction matrix ;

[0182] The regression output is: The classification output is as follows: ;Will The bounding box is obtained by decoding using Distributed Focal Regression (DFL):

[0183] (13)

[0184] The final detection output is:

[0185] (14)

[0186] in, This is the bounding box regression distribution prediction matrix; It is the number of discrete regression channels for each bounding box in DFL; For classification prediction vectors; Number of categories; This represents the distributed focus regression decoding function; This represents the coordinates of the decoded bounding box; This represents the set of detection results (bounding boxes + class probabilities).

[0187] S232, Construct split branches:

[0188] The prototype mask is obtained through the Proto prototype generation network. The mask coefficient matrix is ​​generated through a three-layer convolutional structure. The final segmentation mask is obtained by linearly combining the two:

[0189] (15)

[0190] in, As a prototype mask feature, The number of prototype masks. The mask coefficient matrix corresponding to each instance; The spatial length after flattening all feature points; Use a segmentation mask for the final generated instance.

[0191] S233, achieving decoupled collaboration between detection and segmentation tasks.

[0192] By performing independent forward inference for the detection branch and the segmentation branch, the detection regression-classification prediction and mask generation processes do not interfere with each other.

[0193] Detect branch dependencies Decoding bounding box space with strides:

[0194] (16)

[0195] The branching relies solely on formula (15), thus avoiding gradient conflicts and improving the segmentation accuracy of the orbital edge region. Anchor point position matrix; The step size for each feature layer, This refers to the bounding box decoding functions in the YOLO series.

[0196] S3. Input the training set and validation set into the improved SMDE-YOLO model for training, obtain the prediction results through forward inference, and jointly optimize the detection branch and segmentation branch based on the loss function. At the same time, evaluate the model performance through the validation set and iteratively update the network parameters until the training converges.

[0197] As a preferred implementation, S3, the training process includes:

[0198] S31, the edge-enhanced image dataset is divided into training set and validation set according to the ratio, and the images, masks and enhanced images are loaded, normalized and batch processed synchronously;

[0199] S32, the training set is input into the improved SMDE-YOLO model, the prediction results are obtained through forward inference, and the detection branch and segmentation branch are jointly optimized;

[0200] S33, iteratively update the network parameters based on the performance indicators of the validation set until the training converges or reaches the preset number of training rounds, and obtain the final trained model weights.

[0201] like Figure 6 As shown, the ablation experiment results indicate that the Scharr edge enhancement algorithm, MSEFF module, DMBFPN structure, and DEED-Seg detector head can all improve segmentation accuracy. Introducing these modules reduces the number of model parameters by 8.6% and the computational cost by 5.9%. Furthermore, the mAP&0.5 and mAP&0.5~0.95 of the bounding box are improved by 3.5% and 7.1%, respectively; while the mAP&0.5 and mAP&0.5~0.95 of the mask are improved by 5.8% and 7.8%, respectively.

[0202] After achieving overall fusion, the segmentation performance can be significantly improved while maintaining the lightweight nature of the model, which verifies the effectiveness and practicality of the proposed method.

[0203] S4. Input the test set images into the trained and improved SMDE-YOLO model, use the detection branch and segmentation branch to generate target category prediction and mask prediction for the track area, and fuse the two to obtain the final railway track area instance segmentation result, which can be visualized or saved as output.

[0204] As a preferred implementation, the S4 test process includes:

[0205] S41, Input the test image and its augmented image into the trained improved SMDE-YOLO model;

[0206] S42, using model detection branch and segmentation branch to generate target category prediction and orbital region mask prediction;

[0207] S43 integrates the prediction results to generate the final orbital region instance segmentation result, which can be visualized or saved for subsequent analysis and application.

[0208] like Figure 7 As shown, the improved SMDE-YOLO method of this invention is compared with several mainstream instance segmentation models. Experimental results show that, under balanced model parameters, the method achieves the best accuracy. The Box mAP&0.5 is 92.8%, mAP&0.5~0.95 is 83.7%, and Mask mAP&0.5 is 86.5%, mAP&0.5~0.95 is 72.1%, outperforming other comparative methods in terms of both lightweight design and accuracy. The method of this invention achieves significant advantages in both Box and Mask metrics, exhibiting higher orbital instance segmentation accuracy and boundary fit.

[0209] This specification also provides a railway track instance segmentation method, including: S4, inputting test set images into the above-mentioned railway track instance segmentation model to obtain the instance segmentation result of the track region.

[0210] Step S4 specifically includes:

[0211] S41, Input the railway track images and their corresponding augmented images from the test set into the trained and improved SMDE-YOLO network model;

[0212] S42, The detection branch and segmentation branch of the model are used to perform forward inference on the input image to generate target category prediction and orbital region mask prediction, respectively;

[0213] S43, the mask prediction and detection prediction are fused to generate the final instance segmentation result of the railway track area, and the output can be visualized or saved as needed for subsequent analysis and application.

[0214] like Figure 8 As shown, based on the same idea, this specification also provides a railway track instance segmentation model building device, including:

[0215] The data acquisition and processing module 101 is used to acquire railway track image datasets and perform data preprocessing and enhancement. It adopts edge enhancement operation based on the Scharr edge operator method to highlight the edge features of the track area.

[0216] The model building module 102 is used to build an improved SMDE-YOLO network model based on YOLO11n-seg by introducing a multi-scale edge feature fusion module MSEFF in the backbone network, adopting a dual-path multi-branch feature pyramid DMBFPN in the neck network, and constructing a decoupled and efficient edge-aware detection and segmentation head DEED-Seg to achieve edge-guided multi-scale feature aggregation and high-precision instance segmentation in the orbital region.

[0217] The model training and generation module 103 is used to input the training set and validation set from the edge-enhanced railway track image dataset into the improved SMDE-YOLO model for training, thereby obtaining a railway track instance segmentation model.

[0218] Based on the same idea, this specification also provides an electronic device, including: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions, and when the instructions are executed by the processor, the above-described segmentation model establishment method is implemented.

[0219] Based on the same idea, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0220] Based on the same idea, the present invention also provides a railway track instance segmentation device, including: an input module, used to input test set images into the above-mentioned railway track instance segmentation model to obtain instance segmentation results of track regions.

[0221] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0222] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0223] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of the present invention.

[0224] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0225] It should be understood that the qualifying terms "first", "second", "third", "fourth", "fifth" and "sixth" used in the description of the embodiments of the present invention are only used to more clearly illustrate the technical solutions and are not intended to limit the scope of protection of the present invention.

[0226] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for establishing a railway track instance segmentation model, characterized in that, include: S1. Obtain the railway track image dataset and perform data preprocessing and enhancement. Employ edge enhancement operations based on the Scharr edge operator to highlight the edge features of the track area. S2, based on YOLO11n-seg, introduces a multi-scale edge feature fusion module MSEFF in the backbone network, adopts a dual-path multi-branch feature pyramid DMBFPN in the neck network, and constructs a decoupled and efficient edge-aware detection and segmentation head DEED-Seg to achieve edge-guided multi-scale feature aggregation and high-precision instance segmentation in the orbital region, thus constructing an improved SMDE-YOLO network model. S3, The training set and validation set from the edge-enhanced railway track image dataset are input into the improved SMDE-YOLO model for training to obtain the railway track instance segmentation model.

2. The method for establishing a railway track instance segmentation model according to claim 1, characterized in that, S1 includes: S11, edge intensity maps are obtained by calculating the gradient response of the image in the horizontal and vertical directions, and gradient magnitude fusion is used to highlight the details of the track edge; S12, then morphological dilation is performed on the edge map to enhance edge continuity, thereby obtaining an enhanced image with higher edge contrast, providing clearer boundary information for subsequent feature extraction.

3. The method for establishing a railway track instance segmentation model according to claim 2, characterized in that, S11 includes: S111, uses the Scharr edge operator to perform edge detection on the input image; where the horizontal convolution kernel... and vertical convolution kernel These are used to extract gradient responses in the horizontal and vertical directions, respectively. S112, Based on the convolution results, calculate the gradients in the horizontal and vertical directions respectively, and approximate the edge intensity by weighting to generate a preliminary edge detection map, as shown in equation (1): (1) in, Represents pixels Edge response strength at the location, and These are the gradient magnitudes of the Scharr operator in the horizontal and vertical directions, respectively. S113, Perform a morphological dilation operation on the edge detection map to enhance the connectivity and saliency of the edge structure.

4. The method for establishing a railway track instance segmentation model according to claim 2, characterized in that, S12 includes: S121, Perform bitwise operations on the dilated edge detection map and the corresponding mask map, retaining only the edge response within the track area, to generate the processed edge map, as shown in equation (2): (2) in, For the mask image, This represents a pixel-by-pixel bitwise AND operation. The edge response results are preserved within the masked area; S122, The processed edge map and the original image are pixel-weightedly fused at a ratio of 6:4 to obtain an enhanced image containing significant edge information, as shown in equation (3): (3) in, Represents the original input image. This is the edge enhancement image after fusion.

5. The method for establishing a railway track instance segmentation model according to claim 1, characterized in that, S2 includes: S21, The MSEFF module is integrated into the backbone network and deeply fused with the C3K2 module in the original YOLO11n-seg structure to extract edge information from feature maps of different scales, and combined with the dual-domain selection mechanism to achieve adaptive filtering of edge features, thereby enhancing the expression of fine-grained boundary features. S22, The DMBFPN network structure is adopted in the neck network. Through the DySample dynamic upsampling and weighted fusion mechanism, efficient semantic alignment of multi-scale features and cross-layer information transmission are achieved, thereby improving the network's global feature aggregation capability. S23, The detection and segmentation head constructs the DEED-Seg Head, which decouples the detection branch from the segmentation branch to handle the category prediction and mask generation tasks respectively, so as to reduce interference between tasks and improve the recognition accuracy of railway track boundary areas.

6. The method for establishing a railway track instance segmentation model according to claim 5, characterized in that, S21 includes: S211, Edge Magnification Module (EAB), enhances local edge responses in the spatial dimension; firstly, it processes the input feature map... conduct Average pooling is used to smooth local regions, and edge response maps are calculated by difference, as shown in Equation (4): (4) in, Indicates the input feature map, and These represent the height and width of the feature map, respectively. Number of channels; Indicates with filling Average pooling operation, For edge response feature maps; pass Convolution yields a spatial soft mask to adaptively control the enhancement region, as shown in Equation (5): (5) in, express Convolution operation, for Activation function A spatial weight mask representing the edge region; After passing through Convolution and Activate and enhance edge features, as shown in Equation (6): (6) in, express Convolution operation, This indicates element-wise multiplication. This is the enhanced edge feature map; Finally, the learnable scaling factor is used. The enhanced features are superimposed on the input to obtain the enhanced output, as shown in equation (7): (7) in, These are learnable parameters that control the edge enhancement intensity. Output feature map; S212, utilizing a multi-scale edge feature extraction unit, each branch first passes the input features through an adaptive average pooling layer. Compression is performed with pooling sizes of 3, 6, 9, and 12 to obtain contextual features at different scales; multi-scale edge information is then extracted using depthwise separable convolution DSConv; features at each scale are upsampled to their original size via bilinear interpolation and then input into the corresponding lightweight edge enhancer for fine-grained feature enhancement. S213 utilizes a local feature extraction unit to extract local texture information of the input features through depthwise separable convolution, and then concatenates it with multi-scale enhanced features to form a fused input. S214 utilizes the dual-domain feature selection mechanism (DSM), whose input is the concatenated multi-scale features. The system sequentially passes through the spatial selection module (SSM) and the frequency selection module (FSM) to achieve dynamic fusion and adaptive filtering of spatial and frequency domain features, as expressed in equation (8): (8) in This represents the output feature map after dual-domain filtering; S215 utilizes a feature fusion unit to integrate the features processed by DSM through depthwise separable convolution, outputting a fused edge enhancement feature map to replace the Bottleneck module in the original C3K2 structure, thereby achieving adaptive fusion of multi-scale edge features and enhanced representation of key boundary regions.

7. The method for establishing a railway track instance segmentation model according to claim 5, characterized in that, S22 includes: In the neck network of YOLOv11, the original PAN-FPN structure is replaced with a DMBFPN network structure to achieve efficient semantic fusion of deep and shallow features and cross-layer information alignment, thereby enhancing the ability to identify railway track boundaries. The specific steps are as follows, utilizing the bidirectional fusion path of the DMBFPN network (both top-down and bottom-up) and the dynamic upsampling and weighted feature concatenation mechanism: Top-down path fusion stage: fusing the highest-level semantic features conduct Convolutional downsampling yields low-resolution features. The feature is then concatenated with the C2PSA mid-layer feature from the backbone network and input into the C3K2 module for initial fusion, resulting in the fused feature, as shown in equation (9): (9) in, This indicates a channel splicing operation; Dynamic upsampling stage: from high-level features During the upsampling process, the DySample dynamic upsampler is introduced, which uses a point resampling mechanism based on a learnable sampling weight matrix. Adaptive interpolation of key points: (10) in, For the location on the input feature map pixel values, Indicates the sampling neighborhood range. For learnable interpolation weights, To output the upsampling result; DySample learns the optimal sampling distribution through end-to-end training, thereby preserving key texture details at the track edges during upsampling. Feature splicing and fusion stage: For the feature layer Upsampled features from DySample Original features after convolution downsampling With shallow edge enhancement features The data is concatenated along the channel dimension and then fed into the C3K2 module for fusion. The expression is: (11) Bottom-up path fusion stage: During the bottom-up feature reconstruction process, the following steps are executed sequentially. The convolutional downsampling is then concatenated with the fused features of the corresponding layer and refined by the C3K2 module, thereby achieving multi-round bidirectional interaction between deep and shallow features; this process can be formally represented as: (12) in, This indicates the current level index; Output fusion and feature alignment stage: Multi-scale feature layer of the final output By dynamically fusing multiple branches, bidirectional transmission of high-level semantic information and shallow-level detailed information can be achieved. in, Indicates the first Layer feature map; This represents an improved residual fusion structure; Edge enhancement feature maps from the MSEFF module; This represents the feature output after fusion.

8. The method for establishing a railway track instance segmentation model according to claim 5, characterized in that, S23 includes: S231, Construct the detection branch: A lightweight feature enhancement structure is formed by concatenating two sets of DSConv and the high-efficiency channel attention module on the input feature map at each scale. On the enhanced feature map, respectively through Convolution yields the regression prediction matrix With classification prediction matrix ; The regression output is: The classification output is as follows: ;Will Decoding is performed using focus regression to obtain the bounding box: (13) The final detection output is: (14) in, This is the bounding box regression distribution prediction matrix; It is the number of discrete regression channels for each bounding box in DFL; For classification prediction vectors; Number of categories; This represents the distributed focus regression decoding function; This represents the coordinates of the decoded bounding box; This represents the set of detection results. S232, Construct split branches: The prototype mask is obtained through the Proto prototype generation network. The mask coefficient matrix is ​​generated through a three-layer convolutional structure. The final segmentation mask is obtained by linearly combining the two: (15) in, As a prototype mask feature, The number of prototype masks. The mask coefficient matrix corresponding to each instance; The spatial length after flattening all feature points; Use a segmentation mask for the final generated instance; S233, achieving decoupled collaboration between detection and segmentation tasks: By performing independent forward inference for the detection branch and the segmentation branch, the detection regression-classification prediction and mask generation processes are made independent of each other; the detection branch depends on... The bounding box space decoding is performed using strides, as shown in the following formula: (16) The branching relies solely on formula (15), thereby avoiding gradient conflicts and improving the segmentation accuracy of the orbital edge region; in, Anchor point position matrix; The step size for each feature layer, This refers to the bounding box decoding functions in the YOLO series.

9. The method for establishing a railway track instance segmentation model according to claim 5, characterized in that, S3 includes: S31, the railway track image dataset after edge enhancement is divided into training set and validation set according to a preset ratio, and the images, masks and their enhancement results are loaded synchronously, normalized and batch packaged; S32, The training set is input into the SMDE-YOLO network model, the prediction result is obtained through forward inference, and the detection branch and the segmentation branch are jointly optimized based on the preset loss function; S33. The model weights are iteratively updated based on the performance metrics of the validation set. The network parameters are adjusted through backpropagation until the training process converges or reaches the preset training rounds, thereby obtaining the final trained model weights.

10. A method for segmenting railway track instances, characterized in that, include: S4. Input the test set images into the railway track instance segmentation model according to any one of claims 1 to 9 to obtain the instance segmentation result of the track region.

11. The method for establishing a railway track instance segmentation model according to claim 10, characterized in that, Step S4 specifically includes: S41, Input the railway track images and their corresponding augmented images from the test set into the trained SMDE-YOLO network model; S42, The detection branch and segmentation branch of the model are used to perform forward inference on the input image to generate target category prediction and orbital region mask prediction respectively; S43, The mask prediction and detection prediction are fused to generate the final instance segmentation result of the railway track area, and the result is visualized or saved as needed for subsequent analysis and application.

12. A device for establishing a railway track instance segmentation model, characterized in that, include: The data acquisition and processing module is used to acquire railway track image datasets and perform data preprocessing and enhancement. It adopts edge enhancement operation based on the Scharr edge operator method to highlight the edge features of the track area. The model building module is used to build an improved SMDE-YOLO network model based on YOLO11n-seg by introducing a multi-scale edge feature fusion module MSEFF in the backbone network, adopting a dual-path multi-branch feature pyramid DMBFPN in the neck network, and constructing a decoupled and efficient edge-aware detection and segmentation head DEED-Seg to achieve edge-guided multi-scale feature aggregation and high-precision instance segmentation in the orbital region. The model training and generation module is used to input the training set and validation set from the edge-enhanced railway track image dataset into the improved SMDE-YOLO model for training, thereby obtaining a railway track instance segmentation model.

13. An electronic device, characterized in that, include: At least one processor and a memory communicatively connected to the processor; The memory contains instructions that, when executed by the processor, implement the method according to any one of claims 1 to 9.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1 to 9.

15. A railway track segmentation device, characterized in that, include: The input module is used to input test set images into the railway track instance segmentation model of claim 12 to obtain the instance segmentation result of the track region.

Images