Track plate crack segmentation method and system based on multi-scale attention mechanism

By adopting a multi-scale attention mechanism MA-UNet model in the crack detection of railway track plates, combined with VGG16 pre-trained encoder and global attention mechanism, the problems of low detection accuracy and slow efficiency are solved, and higher fracture segmentation is achieved Accuracy and efficiency.

CN118918116APending Publication Date: 2024-11-08BEIJING JIAOTONG UNIV
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202410793929.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-19
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art has problems such as low detection accuracy, slow efficiency and weak system robustness in the detection of cracks of railway track plates, especially in complex environments, which are difficult to accurately identify crack details.

Method used

Using the MA-UNet model based on multi-scale attention mechanism, combined with VGG16 pre-trained encoder, a multi-scale attention gating mechanism and global attention mechanism are introduced, and Focal Loss and Dice loss loss functions are used to improve the model's extraction of crack features. and segmentation accuracy.

Benefits of technology

It improves the accuracy and recall of track plate crack segmentation, can effectively eliminate background noise interference, improve the segmentation performance of the model, and achieves a higher balance between pixel accuracy and recall.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918116A_ABST
    Figure CN118918116A_ABST
Patent Text Reader

Abstract

The invention provides a track plate crack segmentation method and system based on a multi-scale attention mechanism, belongs to the technical field of image segmentation based on computer vision, and aims to solve the problem that the pixel proportion of a track plate crack in an unmanned aerial vehicle image is small. A neural network pre-trained on a large-scale data set is used as an encoder, extraction and representation of crack detail information by a model are improved, a feature fusion module based on a mixed attention mechanism is adopted, the model can eliminate interference of background redundant information and focus on crack features so as to improve track slab crack segmentation precision, and track slab crack segmentation accuracy is improved. And Focal loss and Dice loss functions are introduced to solve the problems that positive and negative samples are unbalanced and the positive samples are difficult to learn in a crack segmentation task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation based on computer vision, and in particular to a track slab crack segmentation method and system based on a multi-scale attention mechanism. Background Art

[0002] With the increase in operating mileage, the increase in road network density, and the continuous increase in driving speed and load, railway safety risks are also increasing. In this case, it is increasingly important to improve the efficiency of railway inspection to ensure smooth and safe operation. Ballastless track is widely used in high-speed railway networks due to its advantages such as high smoothness, stability, durability and easy maintenance, and has become the main structural type. Its good working condition is the key to the smooth and safe operation of high-speed railways, and it is of great significance to the condition monitoring and defect identification of ballastless track area components. The main components of the ballastless track structure are as follows: Figure 1 As shown, there are rails, fasteners, track plates, CA mortar layers, and base plates. Due to the increase in train traffic and load, under the combined effects of various influencing factors such as ambient temperature, rain, train dynamic load, and geological settlement, the components in the ballastless track structure are constantly worn, resulting in cracks of varying degrees on the surface of the track plates, reducing the stability of the track structure and causing uneven track lines. In order to ensure the safety and efficiency of railway transportation, track plates need to be inspected regularly to promptly detect and eliminate abnormal conditions on the railway tracks.

[0003] At present, the maintenance and upkeep of high-speed railways mainly rely on two methods: manual inspection and track inspection vehicle inspection. Manual inspection refers to the professional and systematic training of line inspection staff in the early stage, and on-site staff inspect track components through visual observation or the use of simple inspection equipment. There are problems such as strong subjectivity, inefficiency, false detection and missed detection, especially in complex environments, attention should be paid to the safety of security personnel; track inspection vehicle inspection refers to the installation of industrial cameras and other equipment on the track inspection vehicle for image acquisition and the use of image processing and other technologies to detect defects, but it takes up train running time, has a long cycle, low efficiency and may affect components. At the same time, the system has weak robustness and a high false alarm rate.

[0004] In recent years, drone inspections have been widely used in the fields of power equipment, roads and bridges. The use of drones for railway inspections has the advantages of long-distance operation, no terrain restrictions, little impact from maintenance windows, cost savings, improved efficiency and expanded coverage. At the same time, the rapid development of artificial intelligence and computer vision technology and the promotion and application of deep learning algorithms have enabled related algorithms based on convolutional neural networks (CNNs) to efficiently extract image features and improve the accuracy of image classification, target detection and semantic segmentation tasks. Therefore, deep learning methods are also widely used in the field of automatic inspections. Summary of the invention

[0005] The object of the present invention is to provide a track slab crack segmentation method and system based on a multi-scale attention mechanism network (MA-UNet) to solve at least one technical problem existing in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a track slab crack segmentation method based on a multi-scale attention mechanism, comprising:

[0008] Acquire track plate image data;

[0009] The acquired track plate image data is processed using a pre-trained segmentation model to obtain a track plate crack segmentation result in the image; wherein, training the segmentation model includes: training a VGG16 network model on the ImageNet dataset, wherein a multi-scale attention gating mechanism is introduced in the jump connection, a global attention mechanism is introduced in the upsampling process, and Focal Loss and Dice loss functions are introduced.

[0010] Furthermore, skip connections are used to connect corresponding layers, and a multi-scale attention gating mechanism is introduced before the feature maps in each encoder are connected with the feature maps in the decoder. The multi-scale attention gating mechanism uses three convolution modules with different receptive fields, including point-by-point convolution, ordinary convolution and dilated convolution, to extract multi-scale feature information. After connecting the feature maps of the three convolutions, the global feature information and channel importance are calibrated through point-by-point convolution.

[0011] Furthermore, the decoder of U-Net consists of multiple convolutional layers and upsampling layers, which are responsible for gradually reducing the size of the feature map to the input image size and achieving the final pixel-level image segmentation. Before the feature map in each layer of the decoder is upsampled to the same size as the feature map in the encoder of the next layer, a global attention mechanism is introduced.

[0012] Furthermore, a global attention mechanism is introduced in the upsampling process, and the channel attention submodule and the spatial attention submodule are designed to include:

[0013] The structure of the channel attention submodule: First, a three-dimensional arrangement is used to preserve the dimensional information; then a two-layer multi-layer perceptron MLP is used to amplify the information dependency between channels and spaces across dimensions, where MLP is a codec structure with a clipping ratio;

[0014] Structure of the spatial attention submodule: Two convolution modules are used to achieve feature fusion to focus on spatial information. Using the same cropping ratio as the channel attention submodule for the maximum pooling operation will cause feature information loss and produce negative contributions, so the pooling operation is deleted. In order to alleviate the resulting parameter increase problem, a grouped convolution operation with channel shuffling is used.

[0015] Furthermore, the focus parameter γ and the modulation coefficient (1-y′) are added on the basis of the weighted cross entropy loss function. γ , Focal loss is defined as:

[0016]

[0017] Among them: y′ is the output category confidence. For background negative samples, the smaller the output probability y′, the smaller the loss value L; on the contrary, for foreground positive samples that the model needs to pay attention to, the smaller the output probability y′, the larger the loss L, and α is the weight parameter; when the sample is an easy-to-classify sample, its category confidence output is higher, so that the modulation factor tends to zero, thereby reducing the contribution of easy-to-classify samples to the overall loss; on the contrary, when the sample is a difficult-to-classify sample, its category confidence output is lower, so that the modulation coefficient tends to 1, without changing the original loss contribution.

[0018] Furthermore, the Dice similarity coefficient is used to calculate the degree of overlap between two samples. Dice loss can consider all pixels of the same category as a whole and calculate the proportion of the intersection in the total, which can reduce the impact of a large number of ballastless track background pixels.

[0019] Fusion of Focal loss and Dice loss gives a hybrid loss function:

[0020] L=λ1L Focal +λ2L Dice

[0021] Among them, λ1 and λ2 are the weight coefficients of Focal loss and Dice loss respectively.

[0022] In a second aspect, the present invention provides a track slab crack segmentation system based on a multi-scale attention mechanism, comprising:

[0023] An acquisition module, used for acquiring track plate image data;

[0024] A processing module is used to process the acquired track plate image data using a pre-trained segmentation model to obtain a track plate crack segmentation result in the image; wherein, training the segmentation model includes: training a VGG16 network model on the ImageNet dataset, wherein a multi-scale attention gating mechanism is introduced in the jump connection, a global attention mechanism is introduced in the upsampling process, and a Focal Loss and a Dice loss loss function are introduced.

[0025] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the track plate crack segmentation method based on MA-UNet as described in the first aspect is implemented.

[0026] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the track plate crack segmentation method based on MA-UNet as described in the first aspect.

[0027] In a fifth aspect, the present invention provides an electronic device, comprising: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the track plate crack segmentation method based on MA-UNet as described in the first aspect.

[0028] The beneficial effects of the present invention are as follows: the introduction of the VGG16 pre-trained encoder improves the segmentation performance and segmentation accuracy, the introduction of the multi-scale attention gating module improves the recall rate, and the introduction of the global attention mechanism improves the accuracy; the designed hybrid attention mechanism can help eliminate background noise interference during feature fusion, which is beneficial to the location establishment of cracks and detail recovery. Through the combination of the two, the segmentation performance of the model can be effectively improved, and a balance between accuracy and recall can be achieved.

[0029] Additional advantages of the present invention will be more clearly given in the following description or learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0031] Figure 1 This is the structural diagram of the main components of the ballastless track structure.

[0032] Figure 2 This is a flow chart of the track slab crack segmentation method based on MA-UNet according to an embodiment of the present invention.

[0033] Figure 3 This is a diagram of the VGG16 network structure described in an embodiment of the present invention.

[0034] Figure 4 Schematic diagram of the track slab crack migration learning scheme described in an embodiment of the present invention.

[0035] Figure 5 This is a diagram of the multi-scale attention gating mechanism described in an embodiment of the present invention.

[0036] Figure 6 This is a GAM diagram of the global attention mechanism described in an embodiment of the present invention.

[0037] Figure 7 This is a diagram of the channel attention submodule described in an embodiment of the present invention.

[0038] Figure 8 This is a diagram of the spatial attention submodule described in an embodiment of the present invention.

[0039] Fig. 9 This is a structural diagram of the MA_UNet model described in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below by the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0041] It should be understood by those skilled in the art that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0042] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with that in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0043] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or groups thereof.

[0044] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. Different embodiments or examples described in this specification and features of different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

[0045] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0046] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0047] The present invention clarifies the key problem of semantic segmentation of cracks in high-speed rail ballastless track slabs based on drone inspection: if the target detection algorithm is used for track slab crack detection, only rough detection results can be obtained, and the detailed features of the cracks cannot be accurately depicted. In addition, in the crack segmentation scenario, the problem of imbalance between positive and negative samples is particularly serious, and it is necessary to improve the algorithm implementation design so that the network can focus on learning crack features, thereby improving the crack segmentation accuracy. In view of the above problems, the present invention proposes a track slab crack segmentation method based on MA-UNet. According to the problem that the proportion of track slab crack pixels in drone images is small, the MA-UNet crack segmentation algorithm is adopted. Using the model-based transfer learning technology, the neural network pre-trained on a large-scale data set is used as the encoder of the model to enhance the network's ability to extract deep semantic features about cracks. At the same time, in order to make the network pay more attention to learning crack features, a hybrid attention feature fusion module based on a multi-scale attention gating mechanism and a global attention mechanism is designed. Finally, Focal loss and Dice loss are introduced to solve the problem that the positive sample (crack) features are difficult to learn, and the crack segmentation accuracy of the algorithm is further improved.

[0048] Example 1

[0049] In this embodiment 1, a track plate crack segmentation system based on a multi-scale attention mechanism is first provided, including: an acquisition module, used to acquire track plate image data; a processing module, used to process the acquired track plate image data using a pre-trained segmentation model to obtain a track plate crack segmentation result in the image; wherein, training the segmentation model includes: training a VGG16 network model on the ImageNet dataset, wherein a multi-scale attention gating mechanism is introduced in the jump connection, a global attention mechanism is introduced in the upsampling process, and Focal Loss and Dice loss functions are introduced.

[0050] In this embodiment, based on the above-mentioned system, a track plate crack segmentation method based on a multi-scale attention mechanism is implemented, including: acquiring track plate image data; using a pre-trained segmentation model to process the acquired track plate image data to obtain a track plate crack segmentation result in the image; wherein, training the segmentation model includes: training a VGG16 network model on the ImageNet dataset, wherein a multi-scale attention gating mechanism is introduced in the jump connection, a global attention mechanism is introduced in the upsampling process, and Focal Loss and Dice loss functions are introduced.

[0051] In this embodiment, the training method of the segmentation model specifically includes the following steps:

[0052] Step 1: Pre-train the VGG16 network model on the ImageNet dataset as the encoder of U-Net. Step 2: Introduce a multi-scale attention gating mechanism in the skip connection. Step 3: Introduce a global attention mechanism in the upsampling process. Step 4: Introduce the Focal Loss and Dice loss functions.

[0053] Step 1 uses the network model of VGG16 pre-trained on the ImageNet dataset as the encoder of U-Net. Its principle and operation are as follows:

[0054] Principle: Transfer learning refers to transferring a model trained in a certain task to another task. The basic principle of transfer learning is to improve the performance of the target domain by transferring the knowledge of the source domain to the target domain. The VGG16 model has a strong feature extraction capability and has been proven to be effective in extracting deep features of crack images. It is suitable for use as a feature extractor to optimize image segmentation tasks and improve accuracy and efficiency. In addition, the richness of the ImageNet dataset enables VGG16 to fully train its generalization ability, showing good robustness, and the model obtained through pre-training has high accuracy.

[0055] Operation: This embodiment intends to combine the model-based transfer learning strategy in deep transfer learning technology. The VGG16 network model pre-trained on the ImageNet dataset is used as the U-Net encoder to improve the feature extraction capability of the original U-Net encoder, thereby improving the network crack segmentation performance.

[0056] Step 2 introduces a multi-scale attention gating mechanism in the skip connection, and its principle and operation are as follows:

[0057] Principle: The multi-scale attention gating mechanism uses three convolution modules with different receptive fields, including point-by-point convolution, ordinary convolution, and dilated convolution with a dilation rate of 2, to extract multi-scale feature information. After connecting the feature maps of the three convolutions, the global feature information and channel importance are calibrated through point-by-point convolution. In this way, the number of parameters is reduced while maintaining good segmentation performance.

[0058] f Concat =σ2(Concat{BN{PointwiseConv(f)},

[0059] BN{OrdinaryConv(f)},BN{DilationConv(f)}})

[0060] f s =f×σ3(PointwiseConv(f Concat ))+f

[0061] In the formula, f represents the input feature map, f Concat represents the feature map connected after three convolution operations, f s represents the output feature map, σ2 and σ3 represent the ReLU and Sigmoid activation functions respectively.

[0062] Operation: U-Net uses skip connections to connect corresponding layers between the encoder and decoder. Before the feature map in each encoder layer is connected with the feature map in the decoder, a multi-scale attention gating mechanism is introduced.

[0063] Step 3 introduces a global attention mechanism in the upsampling process. Its principle and operation are as follows:

[0064] Principle: GAM adopts a sequential channel-spatial attention mechanism and redesigns the submodules based on it. Given the input feature map F1, the intermediate layer feature map F2 and the output feature map F3 are defined as:

[0065]

[0066] Where M C and M SThey are channel and spatial attention maps respectively; Represents an element-wise multiplication operation.

[0067] The structure of the channel attention submodule: First, a three-dimensional arrangement is used to retain the dimensional information; then a two-layer multi-layer perceptron (MLP) is used to amplify the information dependency between channels and spaces across dimensions, where MLP is a codec structure with an r reduction ratio.

[0068] The structure of the spatial attention submodule: Two convolution modules are used to achieve feature fusion to focus on spatial information, using the same crop ratio r as the channel attention submodule. The maximum pooling operation will cause feature information loss and produce negative contributions, so the pooling operation is deleted. In order to alleviate the problem of increased parameters caused by this, a grouped convolution operation with channel shuffling can be used.

[0069] Operation: The decoder of U-Net consists of multiple convolutional layers and upsampling layers, which are responsible for gradually reducing the size of the feature map to the input image size and achieving the final pixel-level image segmentation. Before the feature map in each decoder layer is upsampled to the same size as the feature map in the encoder of the next layer, a global attention mechanism is introduced.

[0070] Step 4 introduces the Focal Loss and Dice loss functions, and their principles and operations are as follows:

[0071] principle:

[0072] i. The original U-Net model generally uses a binary cross entropy loss function, and its formula is:

[0073]

[0074] Where: y′ is the confidence of the output category, ranging from 0 to 1. As can be seen from the above formula, for background negative samples, the smaller the output probability y′, the smaller the loss value L; on the contrary, for foreground positive samples that the model needs to pay attention to, the smaller the output probability y′, the greater the loss L. Sample imbalance will cause the model to be dominated by a large number of easily distinguishable negative samples, too many invalid iterations, slow training, and may not be able to find the optimal solution. To alleviate this problem, the weight parameter α is often added before the formula:

[0075]

[0076] Therefore, α plays a certain balancing role in the weight of positive and negative samples in the loss function, and does not improve the loss contribution distribution of difficult and easy samples. In the process of network learning, the target of the candidate box is filled with a large number of easy-to-classify samples, which ultimately play a dominant role in the overall loss function. Therefore, this embodiment uses Focal loss, adding a focus parameter γ and a modulation coefficient (1-y′) on the basis of the weighted cross entropy loss function. γ Focal loss is defined as:

[0077]

[0078] When the sample is easy to classify, its category confidence output is high, making the modulation factor approach zero, thereby reducing the contribution of the easy-to-classify sample to the overall loss. Conversely, when the sample is difficult to classify, its category confidence output is low, making the modulation coefficient approach 1, without changing the original loss contribution. In summary, Focal loss is equivalent to increasing the loss contribution of difficult-to-classify samples, making the loss function tilt toward samples that are difficult to classify, thereby increasing the model's ability to learn crack pixel features.

[0079] ii. Dice Similarity Coefficient (DSC) is a set similarity measurement function used to calculate the degree of overlap between two samples. Its value is [0,1] and the calculation formula is:

[0080]

[0081] In the formula, X represents the real sample label, Y represents the segmentation result of the network, and |X∩Y| represents the intersection of X and X, that is, the position where the segmentation result overlaps with the sample label. |X| and |Y| represent the number of elements in X and Y respectively. The Dice loss function calculation formula is:

[0082] L Dice =1-Dice

[0083] Dice loss can consider all pixels of the same category as a whole and calculate the proportion of the intersection in the whole, which can reduce the impact of a large number of ballastless track background pixels.

[0084] Operation: This example combines Focal loss and Dice loss and designs a hybrid loss function:

[0085] L=λ1L Focal +λ2L Dice

[0086] Wherein, λ1 and λ2 are weight coefficients of Focal loss and Dice loss respectively, and in the present invention, both of them are taken as 0.5.

[0087] Example 2

[0088] like Figure 2 As shown, this embodiment provides a track plate crack segmentation method, which is implemented by the following technical solution, including the following steps: Step 1: Pre-train the VGG16 network model on the ImageNet dataset as the encoder of U-Net. Among them, in the training model, it includes: Step 2: Introduce a multi-scale attention gating mechanism in the jump connection. Step 3: Introduce a global attention mechanism in the upsampling process. Step 4: Introduce Focal Loss and Dice loss functions. Finally, use the trained segmentation model to segment the cracks.

[0089] The VGG16 in step 1 is named after the VGG (Visual Geometry Group) laboratory and can be used for computer vision tasks such as image classification and semantic segmentation.

[0090] The VGG16 network model architecture is as follows Figure 3 As shown in the figure, it consists of 5 convolutional blocks, each of which contains a convolutional layer and a maximum pooling layer, and finally three fully connected layers for multi-classification. Among them, the first 13 layers use 3×3 convolution kernels and ReLU activation functions to process the results. The convolutional layers and pooling layers are stacked on each other to obtain the same receptive field, which reduces the model parameters; and more ReLU activation functions increase the nonlinear fitting ability of the network through nonlinear mapping. In addition, the superposition of multiple convolutional kernels makes the model generate more intermediate features, further promoting feature fusion in transfer learning. The last 3 fully connected layers map high-level abstract features and output classification results. In this network, 5 convolutional layers are used to extract features at different levels of the image. Shallow convolution extracts underlying general feature information such as texture, color, edge, brightness, etc., while deep convolution extracts more abstract semantic features. The crack segmentation network model built in this embodiment only uses the feature extraction part of VGG16 (i.e., Con_v1 to Conv_6). With the help of the weight parameters of the feature extraction layer of the model that has been pre-trained on a large-scale data set, the model parameters for the track plate crack segmentation task are fine-tuned and trained on this basis.

[0091] The VGG16 model has a strong feature extraction capability and has been proven to be effective in extracting deep features of crack images. As a baseline model in the field of computer vision, it is suitable for use as a feature extractor to optimize image segmentation tasks and improve accuracy and efficiency. In addition, the richness of the ImageNet dataset enables VGG16 to fully train its generalization ability, showing good robustness, and the model obtained through pre-training has high accuracy.

[0092] In this embodiment, the model-based transfer learning strategy in the deep transfer learning technology is combined, such as Figure 4 As shown in the figure, the VGG16 network model pre-trained on the ImageNet dataset is used as the U-Net encoder to improve the feature extraction capability of the original U-Net encoder, thereby improving the network crack segmentation performance.

[0093] The jump connection in step 2 is an important part of the U-Net network structure in step 1, which mainly realizes the fusion of deep low-resolution semantic information and shallow high-resolution local spatial information. Specifically, by performing concat operation on the channel, the intermediate features of each layer of the encoder are combined with the features of the same resolution layer of the decoder, thereby realizing the integration of global content and local resolution position information.

[0094] The multi-scale attention gating mechanism in step 2 and the global attention mechanism in step 3 belong to the hybrid attention mechanism introduced in the process of high-level and low-level feature fusion, which can realize the fusion of channel information and spatial information, reduce information loss, accurately locate cracks and restore crack details, so as to improve the accuracy of crack segmentation.

[0095] In order to solve the problem of different sizes of track slab cracks, this embodiment introduces a multi-scale attention gating mechanism in the feature fusion process to obtain richer semantic and spatial information. Its structure is as follows: Figure 5 As shown in the figure. This module uses three convolution modules with different receptive fields, including point-by-point convolution, ordinary convolution, and dilation rate 2 dilation convolution, to extract multi-scale feature information. After connecting the feature maps of the three convolutions, the global feature information and channel importance are calibrated through point-by-point convolution. In this way, the number of parameters is reduced while maintaining good segmentation performance.

[0096] f Concat =σ2(Concat{BN{PointwiseConv(f)},

[0097] BN{OrdinaryConv(f)},BN{DilationConv(f)}})

[0098] f s=f×σ3(PointwiseConv(f Concat ))+f

[0099] In the formula, f represents the input feature map, f Concat represents the feature map connected after three convolution operations, f s represents the output feature map, σ2 and σ3 represent the ReLU and Sigmoid activation functions respectively.

[0100] Attention mechanisms have been shown to significantly improve the performance of neural networks in various computer vision tasks. SENet is the first network to use channel attention and channel-level feature fusion to suppress unimportant channels. However, it is inefficient in suppressing unimportant pixels. CBAM considers both space and channels, but only captures local feature relationships and cannot obtain long-distance spatial dependencies. These two methods ignore the significance of retaining both channel and spatial information to enhance cross-dimensional interaction, and can only obtain limited receptive field features. Based on the above problems, this embodiment adopts a global attention mechanism (Global Attention Mechanism, GAM) to enable channel information representing global features of the image and spatial information representing detailed features of the image such as edges and textures to achieve cross-dimensional interaction to improve the segmentation performance of the U-Net neural network.

[0101] GAM adopts a sequential channel-spatial attention mechanism, on which the submodules are redesigned. The entire GAM attention mechanism framework is as follows: Figure 7 As shown. Given the input feature map F1, the intermediate layer feature map F2 and the output feature map F3 are defined as:

[0102]

[0103] Where M C and M S They are channel and spatial attention maps respectively; Represents an element-wise multiplication operation.

[0104] The structure of the channel attention submodule is as follows Figure 7 As shown in the figure, firstly, a three-dimensional arrangement is used to preserve the dimensional information. Then, a two-layer multi-layer perceptron (MLP) is used to amplify the information dependency between channels and spaces across dimensions. MLP is a codec structure with a reduction ratio of r.

[0105] The structure of the spatial attention submodule is as follows Figure 8As shown in Figure 2, two convolution modules are used to achieve feature fusion to focus on spatial information. The same crop ratio r as the channel attention submodule is used. The maximum pooling operation will cause feature information loss and produce negative contributions, so the pooling operation is deleted. In order to alleviate the resulting parameter increase problem, a grouped convolution operation with channel shuffling can be used.

[0106] Based on steps 1 to 3, the MA_UNet model constructed in this embodiment is as follows: Fig. 9 As shown in the figure. The VGG16 model pre-trained on the ImageNet dataset is used as the encoder, because the convolutional module of the model after removing the fully connected layer can be regarded as a feature extractor. The pre-training technology enables the network to learn more refined features. The migrated feature layer contains the underlying abstract general features such as edges, textures, and colors, which improves the network's ability to recognize and extract crack feature information, and enhances generalization and robustness. In the feature fusion process, a multi-scale attention gating mechanism and a global attention mechanism are introduced to eliminate redundant information interference and extract crack pixel features in a targeted manner.

[0107] The Focal Loss and Dice loss functions in step 4 are used to alleviate the problem of imbalance between positive and negative samples in the crack segmentation task. The original U-Net model generally uses a binary cross entropy loss function, and its formula is:

[0108]

[0109] Where: y′ is the confidence of the output category, ranging from 0 to 1. As can be seen from the above formula, for background negative samples, the smaller the output probability y′, the smaller the loss value L; on the contrary, for foreground positive samples that the model needs to pay attention to, the smaller the output probability y′, the greater the loss L. Sample imbalance will cause the model to be dominated by a large number of easily distinguishable negative samples, too many invalid iterations, slow training, and may not be able to find the optimal solution. To alleviate this problem, the weight parameter α is often added before the formula:

[0110]

[0111] Therefore, α plays a certain balancing role in the weight of positive and negative samples in the loss function, and does not improve the loss contribution distribution of difficult and easy samples. In the process of network learning, the target of the candidate box is filled with a large number of easy-to-classify samples, which ultimately play a dominant role in the overall loss function. Based on the above observations, this embodiment uses Focal loss, adding a focus parameter γ and a modulation coefficient (1-y′) on the basis of the weighted cross entropy loss function. γ Focal loss is defined as:

[0112]

[0113] When the sample is easy to classify, its category confidence output is high, making the modulation factor approach zero, thereby reducing the contribution of the easy-to-classify sample to the overall loss. Conversely, when the sample is difficult to classify, its category confidence output is low, making the modulation coefficient approach 1, without changing the original loss contribution. In summary, Focal loss is equivalent to increasing the loss contribution of difficult-to-classify samples, making the loss function tilt toward samples that are difficult to classify, thereby increasing the model's ability to learn crack pixel features.

[0114] Dice Similarity Coefficient (DSC) is a set similarity measurement function used to calculate the overlap between two samples. Its value is [0,1] and the calculation formula is:

[0115]

[0116] In the formula, X represents the real sample label, Y represents the segmentation result of the network, and |X∩Y| represents the intersection of X and X, that is, the position where the segmentation result overlaps with the sample label. |X and |Y| represent the number of elements in X and Y respectively. The Dice loss function calculation formula is:

[0117] L Dice =1-Dice

[0118] Dice loss can consider all pixels of the same category as a whole and calculate the proportion of the intersection in the whole, which can reduce the influence of a large number of ballastless track background pixels. The present invention combines Focal loss and Dice loss to design a hybrid loss function:

[0119] L=λ1L Focal +λ2L Dice ; In the formula, λ1 and λ2 are the weight coefficients of Focal loss and Dice loss respectively, and in this embodiment, both of them are taken as 0.5.

[0120] Example 3

[0121] This embodiment 3 provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the track plate crack segmentation method based on MA-UNet as described above is implemented.

[0122] Example 4

[0123] This embodiment 4 provides a computer device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the track plate crack segmentation method based on MA-UNet as described above.

[0124] Example 5

[0125] This embodiment 5 provides an electronic device, including: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the track plate crack segmentation method based on MA-UNet as described above.

[0126] In summary, the track plate crack segmentation method based on MA-UNet described in the embodiment of the present invention has a significant improvement in model segmentation performance after the introduction of the VGG16 pre-trained encoder, the intersection over union (mIoU) value increased by 3.68%, and the pixel accuracy (PA) value increased by 6.03%. After the introduction of the multi-scale attention gating module, the recall rate (R) increased by 4.79%. After the introduction of the global attention mechanism, the precision (P) increased by 4.23%. The hybrid attention mechanism designed by the present invention can help eliminate background noise interference during feature fusion, which is conducive to the location establishment and detail recovery of the crack. Through the combination of the two, the segmentation performance of the model can be effectively improved. When using Dice loss as the loss function, the recall rate of the model is high, because the Dice loss pays more attention to the crack pixel target and pays less attention to the background pixel. When using Focal loss as the loss function, the precision rate of the model is high, because the Focal loss focuses more on the learning of crack pixels. By using the hybrid loss function, a balance between precision and recall can be achieved, and higher mIoU and PA values ​​can be obtained.

[0127] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A track slab crack segmentation method based on a multi-scale attention mechanism, characterized in that: include: Acquire track plate image data; The acquired track plate image data is processed using a pre-trained segmentation model to obtain a track plate crack segmentation result in the image; wherein, training the segmentation model includes: training a VGG16 network model on the ImageNet dataset, wherein a multi-scale attention gating mechanism is introduced in the jump connection, a global attention mechanism is introduced in the upsampling process, and FocalLoss and Diceloss loss functions are introduced.

2. The track slab crack segmentation method based on multi-scale attention mechanism according to claim 1 is characterized in that: Skip connections are used to connect corresponding layers. Before the feature maps in each encoder layer are connected with the feature maps in the decoder, a multi-scale attention gating mechanism is introduced. The multi-scale attention gating mechanism uses three convolution modules with different receptive fields, including point-by-point convolution, ordinary convolution and dilated convolution, to extract multi-scale feature information. After connecting the feature maps of the three convolutions, the global feature information and channel importance are calibrated through point-by-point convolution.

3. The track slab crack segmentation method based on multi-scale attention mechanism according to claim 1, characterized in that: The decoder of U-Net consists of multiple convolutional layers and upsampling layers, which are responsible for gradually reducing the size of the feature map to the input image size and achieving the final pixel-level image segmentation. Before the feature map in each layer of the decoder is upsampled to the same size as the feature map in the encoder of the next layer, a global attention mechanism is introduced.

4. The track slab crack segmentation method based on multi-scale attention mechanism according to claim 3 is characterized in that: In the upsampling process, a global attention mechanism is introduced, and the channel attention submodule and the spatial attention submodule are designed including: The structure of the channel attention submodule: First, a three-dimensional arrangement is used to preserve the dimensional information; then a two-layer multi-layer perceptron MLP is used to amplify the information dependency between channels and spaces across dimensions, where MLP is a codec structure with a clipping ratio; Structure of the spatial attention submodule: Two convolution modules are used to achieve feature fusion to focus on spatial information. Using the same cropping ratio as the channel attention submodule for the maximum pooling operation will cause feature information loss and produce negative contributions, so the pooling operation is deleted. In order to alleviate the resulting parameter increase problem, a grouped convolution operation with channel shuffling is used.

5. The track slab crack segmentation method based on multi-scale attention mechanism according to claim 1, characterized in that: Based on the weighted cross entropy loss function, the focus parameter γ and the modulation coefficient (1-y′) are added. γ , Focalloss is defined as: Among them: y′ is the output category confidence. For background negative samples, the smaller the output probability y′, the smaller the loss value L; on the contrary, for foreground positive samples that the model needs to pay attention to, the smaller the output probability y′, the larger the loss L, and α is the weight parameter; when the sample is an easy-to-classify sample, its category confidence output is higher, so that the modulation factor tends to zero, thereby reducing the contribution of easy-to-classify samples to the overall loss; on the contrary, when the sample is a difficult-to-classify sample, its category confidence output is lower, so that the modulation coefficient tends to 1, without changing the original loss contribution.

6. The track slab crack segmentation method based on multi-scale attention mechanism according to claim 1, characterized in that: The Dice similarity coefficient is used to calculate the degree of overlap between two samples. Diceloss can consider all pixels of the same category as a whole and calculate the proportion of the intersection in the whole, which can reduce the influence of a large number of ballastless track background pixels. Fusion of Focal loss and Dice loss gives a hybrid loss function: L=λ1L Focal +λ2L Dice Among them, λ1 and λ2 are the weight coefficients of Focalloss and Dice loss respectively.

7. A track slab crack segmentation system based on a multi-scale attention mechanism, characterized in that: include: An acquisition module, used for acquiring track plate image data; A processing module is used to process the acquired track plate image data using a pre-trained segmentation model to obtain a track plate crack segmentation result in the image; wherein, training the segmentation model includes: training a VGG16 network model on the ImageNet dataset, wherein a multi-scale attention gating mechanism is introduced in the jump connection, a global attention mechanism is introduced in the upsampling process, and a Focal Loss and a Dice loss loss function are introduced.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by the processor, the track plate crack segmentation method based on the multi-scale attention mechanism as described in any one of claims 1-6 is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the track plate crack segmentation method based on the multi-scale attention mechanism as described in any one of claims 1-6.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the track plate crack segmentation method based on the multi-scale attention mechanism as described in any one of claims 1-6.

Citation Information

Cited By

  • Coal rock fracture intelligent extraction method based on improved U-Net

    CN120563993A

  • An intelligent extraction method for coal and rock fractures based on improved U-Net

    CN120563993B

  • High-temperature workpiece image background removal method based on improved U-Net

    CN120634957A

  • Method for identifying small target features in radiographic detection image

    CN120747624A

  • Underwater crack segmentation-oriented color correction and texture sharpening double-branch enhancement system

    CN120766127A