Image segmentation method suitable for railway track images in coal mines

Abstract

The present invention relates to an image segmentation method suitable for underground railway track images in coal mines. The method comprises: obtaining an underground track target image to be segmented, and loading the obtained underground track target image into a constructed image segmentation network, so as to use the image segmentation network to perform image segmentation on the underground track target image, and generating an underground track segmentation image after image segmentation, wherein the underground track segmentation image includes a railway track segmentation labeled area and a non-railway track segmentation labeled area, wherein the image segmentation network includes a segmentation network main unit and a segmentation network auxiliary unit, wherein the segmentation network main unit includes a main encoding unit and a main decoding unit adaptively connected to the main encoding unit, and the segmentation network auxiliary unit is adaptively connected to the main decoding unit. The present invention can effectively segment underground railway track images in coal mines and improve the segmentation accuracy of underground railway track images in coal mines.

Description

Technical Field

[0001] This invention relates to an image segmentation method, and more particularly to an image segmentation method suitable for railway track images in underground coal mines. Background Technology

[0002] Semantic segmentation aims to accurately classify each pixel in an image into different categories, thereby improving the accuracy and reliability of image analysis. In recent years, significant progress has been made in semantic segmentation through deep learning technology.

[0003] To meet the operational needs of underground coal mines, railway tracks are typically laid underground. These tracks are complex, including numerous curves and intersections. Due to the unique nature of underground coal mine work, comprehensive, real-time, and continuous analysis of surveillance video data from these tracks is generally required. This analysis typically employs semantic segmentation for image processing. To meet the demands of analyzing surveillance video data from underground coal mine tracks, semantic segmentation methods must possess superior real-time processing capabilities.

[0004] Currently, semantic segmentation often utilizes the U-Net architecture based on convolutional neural networks (CNNs), primarily focusing on the U-Net architecture and fully convolutional networks (FCNs), accompanied by various subsequent adaptive improvements. Specifically, the U-Net architecture shares significant similarities with the Feature Pyramid Network (FPN) framework. In the U-Net encoder, a divide-and-conquer strategy is employed, decomposing the input image into feature maps of different scales, which are then passed to the decoder. Simultaneously, the decoder in the U-Net architecture exhibits an effective feature fusion mechanism, integrating multi-scale feature maps from the encoder into a unified single-scale feature map by progressively fusing features at corresponding resolutions.

[0005] In existing technologies, the U-Net architecture relies primarily on convolution and pooling operations when constructing the encoder and decoder architecture. This dependence limits its ability to effectively capture long-range dependencies, thereby limiting the image segmentation accuracy of coal mine underground railway track images. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an image segmentation method suitable for underground railway track images in coal mines. This method can effectively segment underground railway track images in coal mines and improve the segmentation accuracy.

[0007] According to the technical solution provided by the present invention, an image segmentation method suitable for underground railway track images in coal mines is provided, the image segmentation method comprising:

[0008] The process involves acquiring an image of the underground track target to be segmented, and then loading this image into a constructed image segmentation network. The network is used to segment the underground track target image, generating a segmented underground track image. This segmented image includes both railway track segmentation annotation regions and non-railway track segmentation annotation regions.

[0009] The image segmentation network includes a segmentation network main unit based on the U-Net architecture and a segmentation network auxiliary unit based on SAM coding. The segmentation network main unit includes a main coding unit and a main decoding unit adapted and connected to the main coding unit, and the segmentation network auxiliary unit is adapted and connected to the main decoding unit.

[0010] When performing image segmentation on the underground track target image, the main coding unit in the main unit of the segmentation network is used to perform the first coding process to generate the main coding processing feature map after the first coding process. At the same time, the auxiliary unit of the segmentation network is used to perform the second coding process on the underground track target image to generate the auxiliary coding embedding feature map after the second coding.

[0011] The main encoding feature map and the auxiliary encoding embedded feature map are transmitted to the main decoding unit for decoding processing. After decoding, a subsurface track segmentation image is generated.

[0012] When the main decoding unit performs decoding, it includes at least a first decoding process and several second decoding processes performed sequentially.

[0013] During the first decoding process, the auxiliary encoding embedding feature map is concatenated with the main encoding deconvolution processing feature map and the main encoding attention processing feature map generated based on the main encoding processing feature map. This concatenation process generates the decoding baseline feature map.

[0014] The main encoding feature map is deconvolved, and the deconvolutioned feature map of the main encoding is generated after the deconvolution process.

[0015] The main encoding feature map is processed by an attention mechanism, and a main encoding attention-processed feature map is generated after the attention mechanism processing.

[0016] The generated decoding baseline feature map is subjected to a second decoding process, and the resulting underground track segmentation image is generated after the second decoding process.

[0017] The main coding unit includes several main coding network layers connected in sequence. Within the main coding unit, the main coding network layer at the bottom of the U-Net architecture is configured as the main coding transition connection layer, and the remaining main coding network layers are respectively used as main coding processing layers.

[0018] The main decoding unit includes several main decoding processing layers connected in sequence, wherein...

[0019] Within the main unit of the segmentation network, the main encoding processing layer in the main encoding unit and the main decoding processing layer in the main decoding unit are in one-to-one correspondence, and the main encoding processing layer is connected to the corresponding main decoding processing layer by a channel-spatial attention module.

[0020] The main encoding transition connection layer is adapted to the main decoding processing layer at the bottom of the U-Net architecture within the main decoding unit, and the generated main encoding processing feature map is transmitted to the corresponding connected main decoding processing layer through the main encoding transition connection layer. The main decoding processing layer at the bottom of the U-Net architecture also receives the auxiliary encoding embedded feature map.

[0021] When the main decoding unit performs decoding, it uses the main decoding processing layer at the bottom of the U-Net architecture to perform the first decoding process, and configures the remaining main decoding processing layers to perform the second decoding process.

[0022] The main coding network layer includes a main coding convolutional block and a main coding Ghost module connected in sequence, wherein,

[0023] When the main coding unit performs the first coding process, it sequentially extracts coding features through the main coding network layers to perform the first coding subprocessing using each main coding network layer. Specifically, during coding feature extraction, the main coding convolution block performs coding convolution processing first, and after coding convolution processing, the main coding Ghost module performs feature extraction. After feature extraction by the main coding Ghost module, the corresponding network layer feature map is generated.

[0024] Within the main coding unit, for two adjacent main coding network layers, along the direction from the opening of the U-Net architecture towards the bottom, the network layer feature map generated by the upper main coding network layer is subjected to max pooling to generate a max pooled feature map, and the max pooled feature map is loaded into the lower main coding network layer.

[0025] When the first decoding process is performed using the main decoding processing layer at the bottom of the U-Net architecture, then:

[0026] The main decoding processing layer receives the main encoding deconvolution processing feature map and loads the received main encoding sampled feature map into the corresponding connected channel-space attention module. The channel-space attention module also receives the network layer feature map output by the corresponding connected main encoding network layer.

[0027] Based on the received main encoding deconvolution processing feature map and network layer feature map, the channel-space attention module performs attention mechanism processing to generate the main encoding attention processing feature map after the attention mechanism processing.

[0028] The auxiliary encoding embedded feature map, the main encoding deconvolution processed feature map, and the main encoding attention processed feature map are concatenated. After feature concatenation, the feature map is processed by Ghost to generate the decoding baseline feature map.

[0029] The main decoding processing layer, which performs the second decoding process, includes a decoding deconvolution block and a main decoding Ghost module connected in sequence.

[0030] When performing the second decoding process, the basic decoding feature map to be decoded is received. Then, the basic decoding feature map is deconvolved using a decoding deconvolution block to generate a deconvolved decoding feature map.

[0031] The deconvolutioned decoded feature map is loaded into the channel-spatial attention module of the current main decoding processing layer, and the channel-spatial attention module also receives the network layer feature map corresponding to the output of the main coding network layer;

[0032] Based on the received deconvolution-decoded feature map and network layer feature map, the channel-space attention module performs attention mechanism processing to generate a decoded attention processing feature map after the attention mechanism processing.

[0033] The main decoding Ghost module is used to extract features from the decoding attention processing feature map to generate a basic decoding feature map after feature extraction.

[0034] Within the main decoding unit, for two adjacent main decoding processing layers, along the direction from the bottom of the U-Net architecture toward the opening, the basic decoding feature map is generated by the main decoding processing layer below and transmitted to the main decoding processing layer above.

[0035] The channel-space attention module includes a first attention splicer, a channel attention module, a spatial attention module, and an attention linear layer, wherein...

[0036] The attention-based first splicer is connected to the corresponding main encoding and decoding processing layers;

[0037] The channel attention module is connected to the output of the first attention splicer, and the channel attention module uses a residual connection;

[0038] The spatial attention module is connected to the output of the channel attention module, and the spatial attention module uses residual connection;

[0039] The output of the spatial attention module is connected to the attention linear layer, and the attention linear layer is used as the output layer of the channel-spatial attention module.

[0040] The channel attention module includes a first max pooling module for channel attention and a first average pooling module for channel attention, wherein...

[0041] Both the first channel attention max pooling module and the first channel attention average pooling module are connected to the output of the first attention splicer.

[0042] The outputs of the first maximum pooling module and the first average pooling module of channel attention are connected to the first linear module of channel attention. The first linear module of channel attention is connected to the second linear module of channel attention through the channel attention LeakyReLu activation function.

[0043] The output of the second linear channel attention module is connected to the second maximum pooling module and the second average pooling module of channel attention, and both the second maximum pooling module and the second average pooling module of channel attention are connected to the channel attention splicer.

[0044] The output of the channel attention splicer is connected to the channel attention adder, which is used to form the residual connection of the channel attention module, through the channel attention Sigmoid function, and is adapted to the spatial attention module through the channel attention adder.

[0045] The spatial attention module includes a first spatial attention convolutional block, a normalized activation function module, a second spatial attention convolutional block, a normalization module, and a spatial attention sigmoid function connected sequentially.

[0046] The first convolutional block of spatial attention is connected to the output of the channel attention adder;

[0047] The first and second convolutional blocks of spatial attention use the same kernel size;

[0048] When the spatial attention module uses residual connections, the spatial attention sigmoid function is connected to the input of the spatial attention adder, and the input of the first spatial attention convolutional block is also connected to the input of the spatial attention adder. The output of the spatial attention adder is connected to the attention linear layer.

[0049] The segmentation network auxiliary unit includes a SAM encoding unit and a size adjustment unit adapted and connected to the SAM encoding unit, wherein...

[0050] The underground track target image is encoded using SAM coding units, and a basic coded embedding feature map is generated after the encoding process.

[0051] The generated basic encoding embedding feature map is resized using a size adjustment unit to generate an auxiliary encoding embedding feature map. The feature map size of the auxiliary encoding embedding feature map is consistent with the feature map size of the main encoding deconvolution processing feature map and the main encoding attention processing feature map.

[0052] When acquiring images of underground orbital targets, the following steps are included:

[0053] The source image of the underground track is acquired, and the acquired source image is preprocessed to generate the target image of the underground track.

[0054] Preprocessing of the underground track source images includes bilateral filtering and / or contrast-limited adaptive histogram equalization enhancement.

[0055] When performing contrast-limited adaptive histogram equalization enhancement processing, including

[0056] Obtain the source image to be balanced and enhanced, and divide the source image into several source image sub-blocks. For any source image sub-block, then:

[0057]

[0058] Where (b1,b2) is the size of each source image sub-block, (H,W) is the size of the source image to be equalized and enhanced, K is the number of source image sub-blocks, and the source image for equalization and enhancement is the underground track source image or the underground track filtered image generated after bilateral filtering.

[0059] Contrast limiting is applied to each source image sub-block, and the contrast limiting process includes:

[0060] For a source image sub-block T, a grayscale histogram of source image sub-block T is generated, and the frequency of the i-th grayscale level is restricted based on the generated grayscale histogram, then:

[0061]

[0062] Where DB is the frequency limiting threshold, and H T (i) represents the frequency of the source image sub-block T at the i-th gray level, and L represents the number of gray levels of the source image to be equalized and enhanced.

[0063] After frequency limiting and mirroring grayscale mapping, we have:

[0064]

[0065] in, Let i be the gray level mapping for the i-th gray level; Let be the cumulative distribution function of the i-th gray level;

[0066] After the mirrored grayscale mapping process, a mapped grayscale histogram is generated based on the grayscale mapping information, and a corresponding contrast-limited sub-block is generated based on the mapped grayscale histogram.

[0067] Any two contrast-limited sub-blocks are merged using linear interpolation to generate an image of the underground track target.

[0068] Advantages of this invention: Based on the characteristics of underground track target images, the constructed image segmentation network includes a main segmentation network unit and an auxiliary segmentation network unit. The main segmentation network unit performs a first encoding process through its main encoding unit, and simultaneously performs a second encoding process through the auxiliary segmentation network unit. Subsequently, the main encoding feature map generated from the first encoding process and the auxiliary encoding embedding feature map generated from the second encoding process are fed into the main decoding unit. After decoding, an underground track segmentation image can be generated, thereby improving the accuracy of image segmentation for underground track targets.

[0069] By using the Ghost module for feature extraction within the main unit of the segmentation network, the overhead of feature extraction operations performed by the main unit during image segmentation can be reduced, and the impact of secondary encoding processing by the auxiliary units of the segmentation network can be mitigated, thereby improving image segmentation efficiency. Attached Figure Description

[0070] Figure 1 This is a flowchart illustrating an embodiment of the present invention for image segmentation of underground railway track images in coal mines.

[0071] Figure 2 This is one embodiment of the present invention for acquiring images of underground track sources.

[0072] Figure 3 This is a structural block diagram of an embodiment of the image segmentation network of the present invention.

[0073] Figure 4 This is a structural block diagram of one embodiment of the channel-space attention module of the present invention. Detailed Implementation

[0074] The present invention will be further described below with reference to specific accompanying drawings and embodiments.

[0075] To effectively segment images of underground railway tracks in coal mines and improve the segmentation accuracy, this invention provides an image segmentation method suitable for underground railway track images in coal mines. Specifically, the image segmentation method includes:

[0076] The process involves acquiring an image of the underground track target to be segmented, and then loading this image into a constructed image segmentation network. The network is used to segment the underground track target image, generating a segmented underground track image. This segmented image includes both railway track segmentation annotation regions and non-railway track segmentation annotation regions.

[0077] The image segmentation network includes a segmentation network main unit based on the U-Net architecture and a segmentation network auxiliary unit based on SAM coding. The segmentation network main unit includes a main coding unit and a main decoding unit adapted and connected to the main coding unit, and the segmentation network auxiliary unit is adapted and connected to the main decoding unit.

[0078] When performing image segmentation on the underground track target image, the main coding unit in the main unit of the segmentation network is used to perform the first coding process to generate the main coding processing feature map after the first coding process. At the same time, the auxiliary unit of the segmentation network is used to perform the second coding process on the underground track target image to generate the auxiliary coding embedding feature map after the second coding.

[0079] The main encoding feature map and the auxiliary encoding embedded feature map are transmitted to the main decoding unit for decoding processing. After decoding, a subsurface track segmentation image is generated.

[0080] When the main decoding unit performs decoding, it includes at least a first decoding process and several second decoding processes performed sequentially.

[0081] During the first decoding process, the auxiliary encoding embedding feature map is concatenated with the main encoding deconvolution processing feature map and the main encoding attention processing feature map generated based on the main encoding processing feature map. This concatenation process generates the decoding baseline feature map.

[0082] The main encoding feature map is deconvolved, and the deconvolutioned feature map of the main encoding is generated after the deconvolution process.

[0083] The main encoding feature map is processed by an attention mechanism, and a main encoding attention-processed feature map is generated after the attention mechanism processing.

[0084] The generated decoding baseline feature map is subjected to a second decoding process, and the resulting underground track segmentation image is generated after the second decoding process.

[0085] Figure 1 The diagram illustrates a flowchart of an embodiment of the image segmentation method of the present invention. As shown, during image segmentation, an image of the underground track target to be segmented should be acquired. Subsequently, the acquired underground track target image should be loaded into an image segmentation network for image segmentation, generating a segmented underground track image. It should be understood that the image segmentation network should be pre-constructed. The construction of the image segmentation network, the method and process of generating the segmented underground track image using the image segmentation network, will be described in detail below.

[0086] Figure 2 The diagram shows a working scenario for acquiring images of underground track targets according to the present invention. As shown in the diagram, an underground railway is laid in a coal mine. A signal receiver and an underground monitoring camera can be installed on the side of the underground railway. The signal receiver and the underground monitoring camera can be installed at equal intervals. In the diagram, the underground monitoring camera can be located above the underground railway through a camera mounting bracket.

[0087] In practice, vehicle-mounted cameras can be installed on the transport trolleys running on the underground railway in the coal mine. After the vehicle-mounted cameras are installed on the transport trolleys, images of the transport trolleys' forward direction can be obtained. Thus, the obtained images should include the underground railway in the coal mine and the area outside the underground railway. Specifically, the underground railway in the coal mine refers to the two tracks and the area between the two tracks, while the area outside the underground railway in the coal mine refers to the area outside the two tracks of the underground railway in the coal mine.

[0088] In one embodiment of the present invention, the underground track target image can be generated based on images captured by an onboard camera. Therefore, the underground track target image should include the underground coal mine railway and the area outside the underground coal mine railway. It should be noted that when using an image segmentation network for image segmentation, the underground coal mine railway and the area outside the underground coal mine railway included in the underground track target image are mainly segmented and labeled. Therefore, the segmented underground track image includes railway track segmentation and labeling areas and non-railway track segmentation and labeling areas. Specifically, the railway track segmentation and labeling area refers to the area labeled after segmenting the underground coal mine railway, and the non-railway track segmentation and labeling area specifically refers to the labeled area outside the two tracks.

[0089] As those skilled in the art will know, due to the low illumination and high dust levels in underground coal mine environments, it is difficult to achieve ideal image segmentation results if images captured by vehicle-mounted cameras are directly used for image segmentation. Therefore, the acquired underground track target images generally need to be processed before generation. In one embodiment of the present invention, an underground track source image is acquired and preprocessed to generate the underground track target image. As can be seen from the above description, the underground track source image is an image directly captured by a vehicle-mounted camera, and the underground track source image is generally an RGB image.

[0090] In practice, the preprocessing of the underground track source image includes bilateral filtering and / or contrast-limited adaptive histogram equalization enhancement. Therefore, when preprocessing the underground track source image, bilateral filtering, contrast-limited adaptive histogram equalization enhancement, or both can be performed simultaneously. It should be noted that when performing both bilateral filtering and contrast-limited adaptive histogram equalization enhancement simultaneously, bilateral filtering can be performed first, followed by contrast-limited adaptive histogram equalization enhancement; alternatively, contrast-limited adaptive histogram equalization enhancement can be performed first, followed by bilateral filtering, depending on the specific requirements.

[0091] In one embodiment of the present invention, when performing bilateral filtering, the following applies:

[0092]

[0093] Among them, I filered (x,y) are the filtered pixel values, I(x,y) are the pixel values ​​in the original image, Ω is the neighborhood window during bilateral filtering, and W... p (x,y) represents the normalized weights of the pixel values ​​within the neighborhood window Ω, F spatial (i,j) is the Gaussian kernel function in the spatial domain, F rang (I(x,y),I(x+i,y+j)) is the pixel-domain Gaussian kernel function;

[0094] When configuring bilateral filter parameters based on mine environmental parameters, at least the size of the neighborhood window Ω and the spatial domain Gaussian kernel function F should be configured based on the mine environmental parameters. spatial (i,j), pixel-domain Gaussian kernel function F rang The width of the corresponding kernel function (I(x,y),I(x+i,y+j)).

[0095] It is understood that the original image mentioned above can be an image of the underground track source, or an image enhanced by contrast-limited adaptive histogram equalization. An example is given of configuring bilateral filter parameters based on mine environment parameters, such as a mine environment brightness greater than >100 (cd / m²). 2 And the dust concentration in the mine is <1000 (mg / m³) 3 When ), the size of the neighborhood window Ω can be 5, and the spatial domain Gaussian kernel function F spatial The width of the (i,j) kernel function can be 3, and the pixel-domain Gaussian kernel function F rang The width of the kernel function corresponding to (I(x,y),I(x+i,y+j)) can be 3; in other cases, the size of the neighborhood window Ω can be 5, and the spatial domain Gaussian kernel function F spatial The width of the (i,j) kernel function can be 9, and the pixel-domain Gaussian kernel function F rang The width of the kernel function (I(x,y),I(x+i,y+j)) can be 9. It is understandable that, for a spatial domain Gaussian kernel function, once the width of the kernel function is determined, those skilled in the art can determine how to perform the corresponding filtering process using the spatial domain Gaussian kernel function.

[0096] As explained above, the size of the neighborhood window Ω can be configured based on the mine environment parameters. Subsequently, during filtering, the normalized weight W can be calculated based on the pixel values ​​of the current neighborhood window Ω. p (x, y). Subsequently, bilateral filtering can be performed on the low-light source image based on the above bilateral filtering formula. Of course, other methods can also be used to perform bilateral filtering on the low-light source image; the specific bilateral filtering method can be selected according to actual needs, and will not be listed here.

[0097] In one embodiment of the present invention, when performing contrast-limited adaptive histogram equalization enhancement processing, it includes:

[0098] Obtain the source image to be balanced and enhanced, and divide the source image into several source image sub-blocks. For any source image sub-block, then:

[0099]

[0100] Where (b1,b2) is the size of each source image sub-block, (H,W) is the size of the source image to be equalized and enhanced, K is the number of source image sub-blocks, and the source image for equalization and enhancement is the underground track source image or the underground track filtered image generated after bilateral filtering.

[0101] Contrast limiting is applied to each source image sub-block, and the contrast limiting process includes:

[0102] For a source image sub-block T, a grayscale histogram of source image sub-block T is generated, and the frequency of the i-th grayscale level is restricted based on the generated grayscale histogram, then:

[0103]

[0104] Where DB is the frequency limiting threshold, and H T (i) represents the frequency of the source image sub-block T at the i-th gray level, and L represents the number of gray levels of the source image to be equalized and enhanced.

[0105] After frequency limiting and mirroring grayscale mapping, we have:

[0106]

[0107] in, Let i be the gray level mapping for the i-th gray level; Let be the cumulative distribution function of the i-th gray level;

[0108] After the mirrored grayscale mapping process, a mapped grayscale histogram is generated based on the grayscale mapping information, and a corresponding contrast-limited sub-block is generated based on the mapped grayscale histogram.

[0109] Any two contrast-limited sub-blocks are merged using linear interpolation to generate an image of the underground track target.

[0110] In practice, after dividing the source image to be enhanced into corresponding sub-blocks, contrast limiting processing should be applied to each sub-block. This contrast limiting processing involves first generating a grayscale histogram for each sub-block using existing techniques. In the generated grayscale histogram, the horizontal axis represents the grayscale level, and the vertical axis represents the proportion of pixels at any given grayscale level to the total number of pixels in the current sub-block. Therefore, for any sub-block, the frequency H of the i-th grayscale level can be obtained from the generated grayscale histogram. T (i) The corresponding frequency can be obtained by taking the reciprocal of the proportion of the i-th gray level in the gray histogram. The gray level L of the source image to be equalized and enhanced can be 256.

[0111] After determining the frequency of each grayscale level, a frequency limiting threshold DB can be used for contrast limiting. Therefore, when limiting contrast, a frequency limiting threshold DB should be configured. It should be noted that the frequency limiting threshold DB should be related to the acquisition conditions of the source image to be equalized and enhanced in the coal mine, in order to improve the accuracy of generating the underground track target image. Specifically, if the brightness of the mine environment is greater than >100 cd / m², then... 2 And the dust concentration in the mine is <1000 (mg / m³) 3When the number of divisions K is 32, the frequency limit threshold DB can be 10; in other cases, the number of divisions K is 64, and the frequency limit threshold DB can be 30.

[0112] After frequency limiting for each gray level within the grayscale histogram, a mirrored grayscale mapping process is required. This process determines the grayscale mapping for each gray level, allowing grayscale mapping information to be obtained based on all grayscale mappings. Subsequently, a mapped grayscale histogram can be generated based on this information. Since there is a correspondence between the grayscale histogram and the image, contrast-limited sub-blocks can be generated. In other words, for each source image sub-block, after the aforementioned contrast-limiting process, a corresponding contrast-limited sub-block can be generated.

[0113] For any source image sub-block, after generating a corresponding contrast-limited sub-block using the above-mentioned contrast limitation, any two contrast-limited sub-blocks can be merged using bilinear interpolation to generate an underground track target image. Specifically, when using bilinear interpolation for merging, contrast-limited sub-blocks in the same arrangement direction can be merged. The bilinear interpolation method can be consistent with existing technologies, and the merging method should be based on the ability to achieve linear interpolation.

[0114] To minimize the impact of the underground coal mine environment, during preprocessing, it is preferable to simultaneously perform bilateral filtering and contrast-limited adaptive histogram equalization enhancement. Of course, other preprocessing methods can also be used, and the form of preprocessing can be selected as needed to effectively improve the segmentation accuracy of the underground track target image; these will not be listed here.

[0115] To improve segmentation accuracy, in one embodiment of the present invention, the image segmentation network includes a segmentation network master unit based on a U-Net architecture and a segmentation network auxiliary unit based on SAM encoding. Specifically, image segmentation can be achieved using the segmentation network master unit based on a U-Net architecture, and segmentation accuracy can be effectively improved using the segmentation network auxiliary unit based on SAM encoding. The segmentation network master unit based on the U-Net architecture may include a master encoding unit and a master decoding unit. The master encoding unit and the master decoding unit form a U-shaped network; that is, the master encoding unit can form the encoder of the U-shaped network, and the master decoding unit can form the decoder of the U-shaped network. Therefore, the roles of the master encoding unit and the master decoding unit in image segmentation are consistent with existing technologies.

[0116] In specific implementation, when segmenting the underground track target image, the main encoding unit within the main unit of the segmentation network performs a first encoding process to generate a main encoding processing feature map. Simultaneously, the auxiliary unit of the segmentation network performs a second encoding process on the underground track target image to generate an auxiliary encoding embedding feature map. Subsequently, the main encoding processing feature map and the auxiliary encoding embedding feature map are transmitted to the main decoding unit for decoding processing, and an underground track segmentation image is generated after decoding processing.

[0117] It should be understood that when using the main decoding unit for decoding, since the main encoding unit generates the main encoding processing feature map and the auxiliary encoding embedding feature map generated by the auxiliary unit of the segmentation network are used simultaneously, the underground track segmentation image generated by the decoding process has high segmentation accuracy.

[0118] In one embodiment of the present invention, the main coding unit includes a plurality of main coding network layers connected in sequence, wherein, within the main coding unit, the main coding network layer at the bottom of the U-Net architecture is configured as a main coding transition connection layer, and the remaining main coding network layers are respectively used as main coding processing layers.

[0119] The main decoding unit includes several main decoding processing layers connected in sequence, wherein...

[0120] Within the main unit of the segmentation network, the main encoding processing layer in the main encoding unit and the main decoding processing layer in the main decoding unit are in one-to-one correspondence, and the main encoding processing layer is connected to the corresponding main decoding processing layer by a channel-spatial attention module.

[0121] The main encoding transition connection layer is adapted to the main decoding processing layer at the bottom of the U-Net architecture within the main decoding unit, and the generated main encoding processing feature map is transmitted to the corresponding connected main decoding processing layer through the main encoding transition connection layer. The main decoding processing layer at the bottom of the U-Net architecture also receives the auxiliary encoding embedded feature map.

[0122] When the main decoding unit performs decoding, it uses the main decoding processing layer at the bottom of the U-Net architecture to perform the first decoding process, and configures the remaining main decoding processing layers to perform the second decoding process.

[0123] To achieve the first encoding process, a main encoding unit can include multiple main encoding network layers. When these main encoding network layers are sequentially connected to form a U-Net architecture, there is one main encoding network layer located at the opening of the U-Net architecture, one main encoding network layer located at the bottom of the U-Net architecture, and the remaining main encoding network layers located between the opening and the bottom of the U-Net architecture. In specific implementation, the main encoding network layer at the bottom of the U-Net architecture is used as a main encoding transition connection layer, and the remaining main encoding network layers are used as main encoding processing layers. That is, apart from the main encoding network layer used as the main encoding transition connection layer, each of the remaining main encoding network layers forms a main encoding processing layer.

[0124] In practical implementation, the main decoding unit should include multiple sequentially connected main decoding processing layers. Each main decoding processing layer corresponds to one encoding processing layer, and each main encoding processing layer is connected to its corresponding main decoding processing layer via a channel-spatial attention module. Therefore, the number of main decoding processing layers within the main decoding unit is less than the number of main encoding network layers within the main encoding unit. Thus, for the constructed segmentation network main unit, the main decoding processing layer at the bottom of the U-Net architecture is connected to a corresponding main encoding processing layer, and also adapted to a main encoding transition connection layer. This transition connection layer outputs the main encoding processing feature map generated by the first encoding process.

[0125] Furthermore, the auxiliary coding embedding feature maps generated by the auxiliary units of the segmentation network should also be transmitted to the main decoding processing layer at the bottom of the U-Net architecture, so that the first decoding process can be performed by the main decoding processing layer at the bottom of the U-Net architecture. Therefore, the adaptation connection between the auxiliary units of the segmentation network and the main decoding unit specifically refers to the connection between the auxiliary units of the segmentation network and the main decoding processing layer at the bottom of the U-Net architecture within the main decoding unit. It should be noted that, for the main decoding processing layers within the main decoding unit, in addition to the main decoding processing layer at the bottom of the U-Net architecture performing the first decoding process, the remaining main decoding processing layers should be configured to perform the second decoding process. When performing the second decoding process, each main decoding processing layer performs the same second decoding sub-process.

[0126] Depend on Figure 3As can be seen from the existing U-Net architecture, the main encoding transition connection layer should be located at the bottom of the U-Net architecture. At this time, the main decoding processing layer connected to the main encoding transition connection layer is also located at the bottom of the U-Net architecture. When performing the second decoding process, multiple main decoding processing layers sequentially execute the corresponding second decoding sub-processes along the direction from the bottom of the U-Net architecture to the opening of the U-Net architecture. For example, the main decoding processing layer that is adjacent to the main decoding processing layer at the bottom of the U-Net architecture executes the second decoding sub-process first, while the main decoding processing layer at the opening of the U-Net architecture executes the second decoding sub-process last. After executing the corresponding second decoding sub-processes, a well track segmentation image can be generated.

[0127] In one embodiment of the present invention, the main coding network layer includes a main coding convolutional block and a main coding Ghost module connected in sequence, wherein,

[0128] When the main coding unit performs the first coding process, it sequentially extracts coding features through the main coding network layers to perform the first coding subprocessing using each main coding network layer. Specifically, during coding feature extraction, the main coding convolution block performs coding convolution processing first, and after coding convolution processing, the main coding Ghost module performs feature extraction. After feature extraction by the main coding Ghost module, the corresponding network layer feature map is generated.

[0129] Within the main coding unit, for two adjacent main coding network layers, along the direction from the opening of the U-Net architecture towards the bottom, the network layer feature map generated by the upper main coding network layer is subjected to max pooling to generate a max pooled feature map, and the max pooled feature map is loaded into the lower main coding network layer.

[0130] In specific implementation, the main coding network layers within the main coding unit can adopt the same structural form, such as each main coding network layer adopting the form of a main coding convolutional block and a main coding Ghost module. When the main coding unit performs the first coding process, along the direction from the opening of the U-Net architecture to the bottom, each main coding network layer sequentially performs the first coding sub-process. That is, when performing the first coding process, all main coding network layers perform the first coding sub-process once. Thus, when all main coding network layers have performed the corresponding first coding sub-process, the first coding process is considered to be completed.

[0131] For any master coding network layer, when performing the first coding sub-process, coding convolution is first performed using the master coding convolution block, followed by feature extraction using the master coding Ghost module. The resulting network layer feature map is generated after feature extraction by the master coding Ghost module. If the current master coding network layer is not used as a master coding transition connection layer, the generated network layer feature map should be transmitted to the adjacent master coding network layer. Simultaneously, the network layer feature map should be transmitted to the corresponding channel-spatial attention module. It should be noted that when transmitting the network layer feature map to the adjacent master coding network layer, the transmission direction is the sequential direction in which the master coding network layer performs the first coding sub-process. Specifically, the transmission direction is along the opening of the U-Net architecture pointing towards the bottom, i.e., towards the master coding transition connection layer. Finally, the master coding transition connection layer outputs the master coding processed feature map.

[0132] Figure 3 The figure illustrates one embodiment of the main coding unit of the present invention. The figure shows an embodiment in which the main coding unit includes five main coding network layers. As can be seen from the above description, one main coding network layer is configured as a main coding transition connection layer, and the remaining four main coding network layers are configured as main coding processing layers. In this case, the main decoding unit should include four main decoding processing layers. Specifically, the four main decoding processing layers correspond one-to-one with the four main coding processing layers, and each main coding processing layer achieves a skip connection with the corresponding main decoding processing layer through a channel-spatial attention module.

[0133] Figure 3 In this context, GXT refers to the underground track target image. After acquiring the underground track target image, the main coding network layer that performs the first coding sub-processing on the underground track target image is configured as the fifth main coding network layer. At this time, the fifth main coding network layer should serve as the main coding processing layer, and the fifth main coding network layer is located at the opening of the U-Net architecture, such as... Figure 3 As shown. Figure 3 In this context, CR4 is the main coding convolutional block within the fifth main coding network layer, and BGh4 is the main coding Ghost module within the fifth main coding network layer. In specific implementations, the main coding convolutional block may include coding convolutional units and coding activation functions. Within the main coding convolutional block CR4, the number of convolutional kernels in the coding convolutional units can be 64, and the size of each convolutional kernel is 3*3. The coding activation function unit can use the ReLU activation function, and the main coding Ghost module can use the Ghost module commonly used in the prior art. The Ghost module used can be found in the relevant description in GhostNet: More Features From Cheap Operations.

[0134] When the fifth master coding network layer adopts the above parameter configuration, when performing the first coding subprocessing on the underground track target image, it first performs coding convolution processing through the master coding convolution block CR4 to generate feature map E4-0. At this time, the number of channels in feature map E4-0 is 64-dimensional, and the feature size of feature map E4-0 is 1 / 2 of the corresponding feature size of the underground track target image. Feature map E4-0 is then used for feature extraction by the master coding Ghost module, and after feature extraction, network layer feature map E4 can be generated. The feature size of network layer feature map E4 is consistent with the feature size of feature map E4-0, and the number of channels in network layer feature map E4 is also 64-dimensional. It can be understood that using the master coding Ghost module to extract features from feature map E4-0 can effectively improve the efficiency of feature extraction by the master coding network layer.

[0135] Furthermore, Figure 3 The image shows an embodiment where the main coding network layer uses the same structural form. Figure 3 In the diagram, CR3 is the main coding convolutional block within the fourth main coding network layer, and BGh3 is the main coding Ghost module within the fourth main coding network layer. As shown in the diagram, the fourth and fifth main coding network layers are adjacent. After the fifth main coding network layer performs the first coding sub-processing on the underground track target image, the fourth main coding network layer then performs the first coding sub-processing. From the above explanation, it can be seen that after the fifth main coding network layer performs the first coding sub-processing, the generated network layer feature map E4 should be transmitted to the fourth main coding network layer, and after max pooling processing, a max pooled feature map is generated. Figure 3 In this context, MP4 is the Maxpooling module that performs max pooling on the network layer feature map E4, and E3-0 is the max pooled feature map generated after max pooling on the network layer feature map E4.

[0136] In specific implementation, the max pooling module MP4 can perform 2*2 max pooling, that is, the pooling window size of the max pooling module MP4 is 2*2 when performing max pooling. Therefore, the number of channels of the generated max pooling feature map E3-0 is 64-dimensional, and the feature size of the max pooling feature map E3-0 should be 1 / 4 of the corresponding feature size of the underground track target image.

[0137] The parameter configurations for the main encoding convolutional block CR3 and the main encoding Ghost module BGh3 can be found in the descriptions of the main encoding convolutional block CR4 and the main encoding Ghost module BGh4, respectively. Of course, parameter configuration should aim to achieve the corresponding number of channels and feature sizes described below. Specifically, when the fourth main encoding network layer performs the first encoder subprocessing, it uses the main encoding convolutional block CR3 for encoding convolution processing to generate feature map E3-1. At this time, feature map E3-1 has 128 channels, and its feature size is consistent with that of the max-pooling feature map E3-0. Feature map E3-1 undergoes feature extraction via the main encoding Ghost module BGh3 to generate network layer feature map E3. Network layer feature map E3 has 128 channels, and its feature size is consistent with that of feature map E3-1.

[0138] Referring to the above explanation, we can obtain Figure 3 In this context, MP3 is a max pooling module that performs max pooling on the network layer feature map E3. After performing max pooling on the network layer feature map E3, the max pooling module MP3 can generate a max pooling feature map E2-0. The max pooling feature map E2-0 has 128 channels, and its feature size should be 1 / 8 of the corresponding feature size of the underground track target image.

[0139] Figure 3 In this diagram, CR2 is the main coding convolutional block within the third main coding network layer, and BGh2 is the main coding Ghost module within the third main coding network layer. When the third main coding network layer performs the first coding subprocessing, it first performs coding convolution processing on the max-pooling feature map E2-0 through the main coding convolutional block CR2, generating feature map E2-1. At this point, feature map E2-1 has 256 channels, and its feature size is consistent with that of the max-pooling feature map E2-0. Then, the main coding Ghost module BGh2 extracts features from feature map E2-1, generating network layer feature map E2. Network layer feature map E2 also has 256 channels, and its feature size is consistent with that of feature map E2-1.

[0140] Figure 3 In this context, MP2 is a max pooling module that performs max pooling on the network layer feature map E2. After performing max pooling on the network layer feature map E2, the max pooling module MP2 can generate a max pooling feature map E1-0. The max pooling feature map E1-0 has 256 channels, and its feature size should be 1 / 16 of the corresponding feature size of the underground track target image.

[0141] Figure 3 In this diagram, CR1 is the main coding convolutional block within the second main coding network layer, and BGh1 is the main coding Ghost module within the second main coding network layer. When the second main coding network layer performs the first coding subprocessing, it first performs coding convolution processing on the max-pooling feature map E1-0 through the main coding convolutional block CR1, generating feature map E1-1. At this point, feature map E1-1 has 512 channels, and its feature size is consistent with that of the max-pooling feature map E1-0. Then, the main coding Ghost module BGh1 extracts features from feature map E1-1, generating network layer feature map E1. Network layer feature map E1 also has 512 channels, and its feature size is consistent with that of feature map E1-1.

[0142] Figure 3 In this context, MP1 is a max pooling module that performs max pooling on the network layer feature map E1. After performing max pooling on the network layer feature map E1, the max pooling module MP1 can generate a max pooled feature map E0-0. The max pooled feature map E0-0 has 512 channels, and its feature size should be 1 / 32 of the corresponding feature size of the underground track target image.

[0143] CR0 is the main coding convolutional block within the first main coding network layer, and BGh0 is the main coding Ghost module within the first main coding network layer. When the first main coding network layer performs the first coding subprocessing, it first performs coding convolution processing on the max-pooling feature map E0-0 through the main coding convolutional block CR0, generating feature map E0-1. At this point, feature map E0-1 has 1024 channels, and its feature size is consistent with that of the max-pooling feature map E0-0. The main coding Ghost module BGh0 then extracts features from feature map E0-1, generating network layer feature map E0. Network layer feature map E0 also has 1024 channels, and its feature size is consistent with that of feature map E0-1.

[0144] It should be noted that the first master coding network layer is located at the bottom of the U-Net architecture. Therefore, there is no need to perform the above-mentioned max pooling process on the network layer feature map E0. That is, the network layer feature map E0 is the master coding feature map.

[0145] In one embodiment of the present invention, the segmentation network auxiliary unit includes a SAM encoding unit and a size adjustment unit adapted and connected to the SAM encoding unit, wherein,

[0146] The underground track target image is encoded using SAM coding units, and a basic coded embedding feature map is generated after the encoding process.

[0147] The generated basic encoding embedding feature map is resized using a size adjustment unit to generate an auxiliary encoding embedding feature map. The feature map size of the auxiliary encoding embedding feature map is consistent with the feature map size of the main encoding deconvolution processing feature map and the main encoding attention processing feature map.

[0148] Figure 3 In this context, SAM stands for SAM coding unit. The SAM coding unit can adopt commonly used forms, such as those described in *Faster segment anything: Towards lightweight SAM for mobile applications*. When using the auxiliary unit of the segmentation network to perform secondary encoding processing on the underground track target image, the SAM coding unit is used first for encoding, and a basic encoded embedding feature map is generated after the encoding process. Figure 3 S in the SAM encoding unit is the basic encoded embedding feature map generated by the SAM encoding unit. The basic encoded embedding feature map can have 512 channels.

[0149] The generated basic encoded embedding feature map is then resized by a resizing unit to ensure that the feature size of the auxiliary encoded embedding feature map matches the corresponding feature map sizes of the main encoded deconvolution processing feature map and the main encoded attention processing feature map. For example, the feature size of the auxiliary encoded embedding feature map can be 1 / 16 of the feature size of the underground track target image. Figure 3 In this context, ZC3 is the auxiliary encoding embedding feature map.

[0150] It should be noted that the size adjustment unit only adjusts the feature size of the size adjustment unit. The size adjustment unit can adopt the commonly used form. The specific method and process of implementing the feature size adjustment can be consistent with the existing technology, and will not be described in detail here.

[0151] In one embodiment of the present invention, when the first decoding process is performed using the main decoding processing layer at the bottom of the U-Net architecture, then:

[0152] The main decoding processing layer receives the main encoding deconvolution processing feature map and loads the received main encoding sampled feature map into the corresponding connected channel-space attention module. The channel-space attention module also receives the network layer feature map output by the corresponding connected main encoding network layer.

[0153] Based on the received main encoding deconvolution processing feature map and network layer feature map, the channel-space attention module performs attention mechanism processing to generate the main encoding attention processing feature map after the attention mechanism processing.

[0154] The auxiliary encoding embedded feature map, the main encoding deconvolution processed feature map, and the main encoding attention processed feature map are concatenated. After feature concatenation, the feature map is processed by Ghost to generate the decoding baseline feature map.

[0155] As can be seen from the above description, the number of channel-spatial attention modules should be consistent with the number of main coding processing layers in the main coding unit. When the main coding unit includes four main coding processing layers, when the main decoding processing layer at the bottom of the U-Net architecture is the first main decoding processing layer, the remaining main decoding processing layers are the second, third, and fourth main decoding processing layers in sequence, along the direction from the bottom of the U-Net architecture to the opening of the U-Net architecture.

[0156] As explained above, the main unit of the segmentation network should include four channel-spatial attention modules. Figure 3 The diagram illustrates an embodiment in which the main unit of the segmentation network includes four channel-spatial attention modules, namely channel-spatial attention modules SCCBAM1 to SCCBAM4. Specifically, channel-spatial attention module SCCBAM1 is adapted and connected to the second main encoding network layer and the first main decoding processing layer; channel-spatial attention module SCCBAM2 is adapted and connected to the third main encoding network layer and the second main decoding processing layer; channel-spatial attention module SCCBAM3 is adapted and connected to the fourth main encoding network layer and the third main decoding processing layer; and channel-spatial attention module SCCBAM4 is adapted and connected to the reading material main encoding network layer and the fourth main decoding processing layer.

[0157] In specific implementation, the first main decoding processing layer should be configured to perform the first decoding processing. When the first main decoding processing layer performs the first decoding processing, it first performs deconvolution processing on the main encoding processing feature map. For example, a deconvolution module with a kernel size of 2*2 can be used to perform deconvolution processing on the main encoding processing feature map so that the main encoding deconvolution processing feature map can be obtained after deconvolution processing. After deconvolution processing, the number of channels of the main encoding deconvolution processing feature map can be 512 dimensions, and the feature size of the main encoding deconvolution processing feature map is 1 / 16 of the corresponding feature size of the underground track target image. Figure 3 In the diagram, ZC1 is the main encoding deconvolution processing feature map, and UC0 is the decoding deconvolution block that performs deconvolution processing on the main encoding processing feature map. The decoding deconvolution block can use existing commonly used deconvolution modules, such as a deconvolution module with a kernel size of 2*2.

[0158] Specifically, after obtaining the main encoding deconvolution processing feature map, it should be loaded into the channel-spatial attention module SCCBAM1. SCCBAM1 also receives the network layer feature map E1 generated by the second main encoding network layer. SCCBAM1 performs attention mechanism processing on the main encoding deconvolution processing feature map and the network layer feature map E1 to generate the main encoding attention processing feature map. It should be noted that the main encoding attention processing feature map has 512 channels, and its feature size is 1 / 16 of the corresponding feature size of the underground track target image. Figure 3 In this context, ZC2 is the main encoding attention processing feature map.

[0159] Within the first main decoding layer, the feature maps embedded in the auxiliary encoder, processed by deconvolution of the main encoder, and processed by attention of the main encoder are concatenated along the channel number dimension. After concatenation, these features are then processed by Ghost to generate the decoding baseline feature map. As explained above, the generated decoding baseline feature map should be transmitted to the second main decoding layer. Figure 3 In the diagram, D0 is the decoding reference feature map. The number of channels in the decoding reference feature map should be 512 dimensions, and the feature size of the decoding reference feature map is 1 / 16 of the corresponding feature size of the underground track target image.

[0160] It should be understood that, in order to perform the first decoding process described above, the first main decoding layer should include a decoding deconvolution block, a feature map stitcher, and a decoding Ghsot module. Figure 3 In the diagram, JGh0 is the decoding Ghost module within the first main decoding processing layer, but the feature map stitcher is not included. Figure 3 As shown in the figure, the parameter settings for the decoding deconvolution block can be referred to the above description, the feature map stitcher can adopt the form in the prior art, and the decoding Ghost module can refer to the above description of the main encoding Ghost module.

[0161] In one embodiment of the present invention, the main decoding processing layer performing the second decoding process includes a decoding deconvolution block and a main decoding Ghost module connected in sequence, wherein...

[0162] When performing the second decoding process, the basic decoding feature map to be decoded is received. Then, the basic decoding feature map is deconvolved using a decoding deconvolution block to generate a deconvolved decoding feature map.

[0163] The deconvolutioned decoded feature map is loaded into the channel-spatial attention module of the current main decoding processing layer, and the channel-spatial attention module also receives the network layer feature map corresponding to the output of the main coding network layer;

[0164] Based on the received deconvolution-decoded feature map and network layer feature map, the channel-space attention module performs attention mechanism processing to generate a decoded attention processing feature map after the attention mechanism processing.

[0165] The main decoding Ghost module is used to extract features from the decoding attention processing feature map to generate a basic decoding feature map after feature extraction.

[0166] Within the main decoding unit, for two adjacent main decoding processing layers, along the direction from the bottom of the U-Net architecture toward the opening, the basic decoding feature map is generated by the main decoding processing layer below and transmitted to the main decoding processing layer above.

[0167] In practice, the second decoding process performed by all main decoding processing layers is the same. The difference from the first decoding process performed by the first main decoding processing layer is that the feature map splicing using the feature map splicer can be omitted. As explained above, for the second main decoding processing layer, the received basic decoding feature map should be the decoding reference feature map generated by the first decoding process performed by the first main decoding processing layer. For the third main decoding processing layer, the basic decoding feature map should be generated by the second main decoding processing layer. For the fourth main decoding processing layer, the basic decoding feature map should be generated by the third main decoding processing layer.

[0168] Figure 3 In this diagram, UC1 is the decoding deconvolution block within the second main decoding processing layer, JGh1 is the main decoding Ghost module within the second main decoding processing layer, D1-0 is the deconvolutional decoded feature map generated after upsampling the decoding baseline feature map using the decoding deconvolution block UC1, and D1-1 is the decoded attention-processed feature map generated by the channel-spatial attention module SCCBMA2. The deconvolutional decoded feature map D1-0 has 256 channels, and its feature size is 1 / 16 of the corresponding feature size of the underground track target image. The channel-spatial attention module SCCBMA2 performs attention mechanism processing on the deconvolutional decoded feature map D1-0 and the network layer feature map E2 to generate the decoded attention-processed feature map D1-1, which also has 256 channels.

[0169] By extracting features from the decoding attention processing feature map D1-1 using the main decoding Ghost module JGh1, a basic decoding feature map D1 can be generated. The basic decoding feature map D1 has 512 channels and its feature size is 1 / 16 of the corresponding feature size of the underground track target image.

[0170] Figure 3 In the diagram, UC2 is the decoding deconvolution block within the third main decoding processing layer, JGh2 is the main decoding Ghost module within the third main decoding processing layer, D2-0 is the deconvolutioned decoding feature map generated after upsampling the basic decoding feature map D1 using the decoding deconvolution block UC2, and D2-1 is the decoding attention processing feature map generated by the channel-spatial attention module SCCBMA3.

[0171] The number of channels in the deconvolution-decoded feature map D2-0 is 256-dimensional, and its feature size is 1 / 8 of the corresponding feature size of the underground track target image. The channel-spatial attention module SCCBMA3 is used to process the deconvolution-decoded feature map D2-0 and the network layer feature map E3 through an attention mechanism to generate the decoded attention-processed feature map D2-1. The main decoding Ghost module JGh2 is used to extract features from the decoded attention-processed feature map D2-1 to generate the basic decoded feature map D2. The number of channels in the basic decoded feature map D2 is 256-dimensional, and its feature size is 1 / 8 of the corresponding feature size of the underground track target image.

[0172] Figure 3 In the diagram, UC3 is the decoding deconvolution block within the fourth main decoding processing layer, JGh3 is the main decoding Ghost module within the fourth main decoding processing layer, D3-0 is the deconvolutioned decoding feature map generated after upsampling the basic decoding feature map D2 using the decoding deconvolution block UC3, and D3-1 is the decoding attention processing feature map generated by the channel-spatial attention module SCCBMA4.

[0173] The channel-spatial attention module SCCBMA4 is used to process the deconvolution-decoded feature map D3-0 and the network layer feature map E4 with an attention mechanism to generate the decoded attention-processed feature map D3-1. The main decoding Ghost module JGh4 is used to extract features from the decoded attention-processed feature map D3-1 to generate the basic decoded feature map D3. The number of channels in the basic decoded feature map D3 can be 64 dimensions, and the feature size of the basic decoded feature map D3 can be half the feature size of the corresponding feature size of the underground track target image.

[0174] It should be noted that after generating the basic decoded feature map D3, it can be further processed by the decoded output processing module to generate a segmented image of the underground railway track. The decoded output processing module can consist of a decoded output processing convolutional block and a decoded processing sigmoid activation function. Specifically, the decoded output processing convolutional block can use a 1*1 convolutional kernel. During output processing, the basic decoded feature map D3 should first undergo convolution processing through the decoded output convolutional block. This block adds non-linear characteristics while maintaining the feature size of the basic decoded feature map D3. Subsequently, it is processed by the decoded processing sigmoid activation function to map the output to between 0 and 1, thereby achieving image segmentation as described in this invention and generating a segmented image of the underground railway track. Figure 3 The UCS in the middle is the decoding output processing module.

[0175] In one embodiment of the present invention, the channel-spatial attention module includes a first attention splicer, a channel attention module, a spatial attention module, and an attention linear layer, wherein,

[0176] The attention-based first splicer is connected to the corresponding main encoding and decoding processing layers;

[0177] The channel attention module is connected to the output of the first attention splicer, and the channel attention module uses a residual connection;

[0178] The spatial attention module is connected to the output of the channel attention module, and the spatial attention module uses residual connection;

[0179] The output of the spatial attention module is connected to the attention linear layer, and the attention linear layer is used as the output layer of the channel-spatial attention module.

[0180] In practice, the channel-spatial attention modules SCCBAM1 to SCCBAM4 can adopt the same form. Figure 4 An embodiment of the channel-space attention module is shown in the figure. Figure 4 In this diagram, Cat1 is the first attention stitcher, Channel Attention is the channel attention module, Spatial Attention is the spatial attention module, and XN3 is the linear attention layer. The linear attention layer XN3 can be a multilayer perceptron. By using the linear attention layer, the dimensionality of the output feature map can be reduced to half the number of channels in the input feature map. Figure 4In this context, "From Encoder Layer" specifically refers to the network layer feature map output from the main encoding layer, and "From Lower Decoder Layer" specifically refers to the feature map from the main decoding layer. For example, when the channel-spatial attention module SCCBAM1 is connected to the first main decoding layer, the feature map from "From Lower Decoder Layer" should be the main encoding deconvolution processing feature map. When the channel-spatial attention module is connected to the second to fourth main decoding layers, the feature map from "From Lower Decoder Layer" should be the deconvolution-decoded feature map. The cases of main encoding deconvolution processing feature maps and deconvolution-decoded feature maps can be referred to the corresponding explanations above, and will not be repeated here.

[0181] For any channel-spatial attention module, the channel-spatial attention module is adapted and connected to the corresponding main encoding processing layer and main decoding processing layer through the attention first splicer. For example, for the channel-spatial attention module SCCBAM1, it is connected to the first main decoding processing layer and the main encoding processing layer formed by the fourth main encoding network layer through the attention first splicer. Other connection situations can be referred to here and the corresponding descriptions above.

[0182] It is understandable that feature stitching can be performed through the first attention stitcher, channel attention processing can be achieved through the channel attention module, and spatial attention processing can be achieved through the spatial attention module. That is, the above attention mechanism processing should include both channel attention processing and spatial attention processing.

[0183] It should be noted that when the main encoding and decoding layers are connected by the channel-spatial attention module, both channel attention and spatial attention are performed simultaneously. Furthermore, when the attention mechanism is executed, channel attention and spatial attention are performed sequentially. Compared to relying on a single attention mechanism, this significantly improves the multidimensional interaction in the channel and spatial domains during image segmentation, enhances the ability to identify and extract key information features, effectively reduces the spatial information loss commonly seen in encoding convolution operations, and further improves the accuracy of image segmentation.

[0184] In one embodiment of the present invention, the channel attention module includes a channel attention first max pooling module and a channel attention first average pooling module, wherein,

[0185] Both the first channel attention max pooling module and the first channel attention average pooling module are connected to the output of the first attention splicer.

[0186] The outputs of the first maximum pooling module and the first average pooling module of channel attention are connected to the first linear module of channel attention. The first linear module of channel attention is connected to the second linear module of channel attention through the channel attention LeakyReLu activation function.

[0187] The output of the second linear channel attention module is connected to the second maximum pooling module and the second average pooling module of channel attention, and both the second maximum pooling module and the second average pooling module of channel attention are connected to the channel attention splicer.

[0188] The output of the channel attention splicer is connected to the channel attention adder, which is used to form the residual connection of the channel attention module, through the channel attention Sigmoid function, and is adapted to the spatial attention module through the channel attention adder.

[0189] Figure 4 In this framework, Maxpool1 is the first max pooling module for channel attention, and Maxpool2 is the second max pooling module for channel attention. Max pooling can be performed through the first and second max pooling modules for channel attention, respectively. Avgpool1 is the first average pooling module for channel attention, and Avgpool2 is the second average pooling module for channel attention. Average pooling can be performed through the first and second average pooling modules for channel attention, respectively. Figure 4 In this diagram, XN1 is the first linear module for channel attention, LeakyReLu is the LeakyReLu activation function for channel attention, XN2 is the second linear module for channel attention, Cat2 is the channel attention splicer, S1 is the channel attention Sigmoid function, and Ad1 is the channel attention adder. Both the first and second linear modules for channel attention can be implemented using a multilayer perceptron for linear transformation processing.

[0190] Figure 4 In this diagram, F1 is the feature map output by the first attention stitcher, and F2 is the feature map output by the channel attention adder. In practice, after generating feature map F1 through the first attention stitcher, two different spatial context representations can be generated through the channel attention first max pooling module and the channel attention first average pooling module: and in, This is the spatial context representation generated after the feature map F1 undergoes max pooling processing via the first max pooling module of channel attention. The spatial context representation is generated after the feature map F1 undergoes average pooling processing by the first average pooling module of channel attention. Subsequently, dimensionality compression is performed by the first linear module of channel attention to reduce parameter computation. This first linear module reduces the number of channels in the feature map by half, and the LeakyReLu activation function enhances the expressive power of non-linear relationships between channels, improving feature interaction capabilities. The second linear module of channel attention restores the feature map's feature dimensions to their original size before processing by the first linear module.

[0191] In one embodiment of the present invention, the spatial attention module includes a spatial attention first convolutional block, a normalized activation function module, a spatial attention second convolutional block, a normalization module, and a spatial attention sigmoid function connected sequentially, wherein...

[0192] The first convolutional block of spatial attention is connected to the output of the channel attention adder;

[0193] The first and second convolutional blocks of spatial attention use the same kernel size;

[0194] When the spatial attention module uses residual connections, the spatial attention sigmoid function is connected to the input of the spatial attention adder, and the input of the first spatial attention convolutional block is also connected to the input of the spatial attention adder. The output of the spatial attention adder is connected to the attention linear layer.

[0195] Figure 4 In the diagram, C1 is the first spatial attention convolutional block, C2 is the second spatial attention convolutional block, BNLR is the normalized activation function module, BN is the normalization module, S2 is the spatial attention sigmoid function, and Ad2 is the spatial attention adder. F3 is the feature map output by the spatial attention module, which is also the feature map output by the spatial attention adder Ad2. In specific implementation, the size of the convolutional kernels used in the first and second spatial attention convolutional blocks can both be 7*7. The normalized activation function module BNLR can include a normalization layer and a LeakyReLU activation function. Feature map F3 is passed through the attention linear layer to obtain feature map x3. It should be noted that the first and second spatial attention convolutional blocks use only one convolutional kernel. By using a 7*7 kernel, the receptive field can be increased to perceive a wider range of features.

[0196] When the spatial attention module adopts the above form, it first performs a convolution operation on the feature map using the first spatial attention convolution block, and outputs a single-channel feature map after the convolution operation to effectively capture large-scale spatial dependencies. At the same time, reducing the number of channels to a single channel can generate spatial weights. The normalized activation function module can eliminate the deep input offset problem of the network and improve the effectiveness of backpropagation. Then, it performs a convolution operation through the second spatial attention convolution block. Finally, the spatial attention sigmoid function normalizes the convolution result into a 0-1 spatial attention weight map to identify the importance of different spatial locations, so as to realize dynamic feature selection in spatial dimension and make the image segmentation network of this invention pay more attention to the semantic regions related to the task.

[0197] As can be seen from the above description, setting up a first convolutional block and a second convolutional block within the spatial attention module can effectively enhance the ability to extract spatial features. This is because a larger receptive field helps to capture information more comprehensively, thereby improving the ability to extract spatial features.

[0198] The image segmentation network described above can be constructed using the following methods:

[0199] Construct a basic image segmentation model and a segmentation model training dataset for training the basic image segmentation model.

[0200] Configure the model training condition parameters, and train the model based on the configured model training condition parameters and the segmentation model training dataset. When the basic image segmentation model reaches the target state, configure the corresponding basic image segmentation model as the image segmentation network.

[0201] It is understandable that the basic image segmentation model is the model to be modeled, while the image segmentation network is the model trained on the basic image segmentation model to reach the target state. Therefore, the constructed basic image segmentation model should be consistent with the image segmentation network. For the case of the basic image segmentation model, please refer to the above description of the image segmentation network, which will not be repeated here.

[0202] To achieve image segmentation, a segmentation model training dataset should be constructed. The segmentation model training dataset should include several training samples. Each training sample should be created based on an image containing railway tracks taken in a coal mine. Specifically, when creating training samples, an image containing railway tracks should be acquired first. Then, tools such as VIA should be used to delineate a closed region containing railway tracks on the image, and a label for the railway track region should be added to the closed region to indicate that the current closed region is the railway track region.

[0203] As explained above, each training sample should be an image containing a closed region and a label. Each training sample should have a resolution of 1280×800, and the corresponding labels should be stored in JSON format. The number of training samples in the segmentation model training dataset can be selected as needed. Generally, training samples can be created using the methods described above. To improve the diversity and robustness of the model training dataset, after creating the training samples using the above methods, new training samples can be constructed using image transformation methods. Image training methods can include random scaling (ranging from 0.75 to 1.25), rotation (within a certain angle), and horizontal flipping, etc. The image transformation methods can be selected as needed and will not be elaborated here.

[0204] It should be noted that the configured model training condition parameters should generally include the loss function and necessary training parameters for model training. These may include the number of training samples per batch, an initial learning rate of 5e-3, and a planned decay rate of 0.004. Necessary training parameters can be selected as needed to meet training requirements; they will not be listed here individually. To address the challenges posed by imbalance, this invention employs a hybrid loss function, which is:

[0205]

[0206] Among them, L z Let H be the loss value of the loss function, H be the height of the training output segmentation image, W be the width of the training output segmentation image, and C be the number of segmented categories; (a, b) represent the coordinates of a pixel within the training output segmentation image; p j (a,b) represents the predicted probability that the pixel at coordinate (a,b) in the segmented image belongs to class j; y j (a,b) represents the coordinates of the pixel at position (a,b) in the segmented image of the training output. The pixel at position (a,b) represents the true value of class j. λ is the weighting parameter.

[0207] For any training sample, a corresponding training output segmentation image can be output through the basic image segmentation model. Based on the output training output segmentation image, the loss value of the aforementioned loss function can be applied. In specific implementation, the prediction probability p... j (a, b) can be directly obtained from the training output segmented image of the basic image segmentation model. Within the training output segmented image, when the pixel at coordinate position (a, b) belongs to category j, then y j The value of (a,b) should be 1; otherwise, y j The value of (a,b) should be 0. Furthermore, the weighting parameter λ can be 0.6. As can be seen from the above description of the image segmentation objective of this invention, the number of segmentation categories C should be 2.

[0208] It should be understood that the image size of the training output segmentation image is consistent with the size of the corresponding image within the training samples. When the image of the training sample is 1280×800, then the height H of the training output segmentation image is 1280, and the width W of the training output segmentation image is 800. In addition, the size of the underground track target image should also be 1280×800.

[0209] It should be noted that when training the basic image segmentation model using the segmentation model training dataset, when the value of the loss function tends to stabilize, it can be considered that the basic image segmentation training has reached the target state. At this time, the basic image segmentation model that has reached the target state can be configured as an image segmentation network.

Claims

1. An image segmentation method suitable for railway track images in underground coal mines, characterized in that, The image segmentation method includes: The process involves acquiring an image of the underground track target to be segmented, and then loading this image into a constructed image segmentation network. The network is used to segment the underground track target image, generating a segmented underground track image. This segmented image includes both railway track segmentation annotation regions and non-railway track segmentation annotation regions. The image segmentation network includes a segmentation network main unit based on the U-Net architecture and a segmentation network auxiliary unit based on SAM coding. The segmentation network main unit includes a main coding unit and a main decoding unit adapted and connected to the main coding unit, and the segmentation network auxiliary unit is adapted and connected to the main decoding unit. When performing image segmentation on the underground track target image, the main coding unit in the main unit of the segmentation network is used to perform the first coding process to generate the main coding processing feature map after the first coding process. At the same time, the auxiliary unit of the segmentation network is used to perform the second coding process on the underground track target image to generate the auxiliary coding embedding feature map after the second coding. The main encoding feature map and the auxiliary encoding embedded feature map are transmitted to the main decoding unit for decoding processing. After decoding, a subsurface track segmentation image is generated. When the main decoding unit performs decoding, it includes at least a first decoding process and several second decoding processes performed sequentially. During the first decoding process, the auxiliary encoding embedding feature map is concatenated with the main encoding deconvolution processing feature map and the main encoding attention processing feature map generated based on the main encoding processing feature map. This concatenation process generates the decoding baseline feature map. The main encoding feature map is deconvolved, and the deconvolutioned feature map of the main encoding is generated after the deconvolution process. The main encoding feature map is processed by an attention mechanism, and a main encoding attention-processed feature map is generated after the attention mechanism processing. The generated decoding reference feature map is subjected to a second decoding process, and the resulting underground track segmentation image is generated after the second decoding process. The main coding unit includes several main coding network layers connected in sequence. Within the main coding unit, the main coding network layer at the bottom of the U-Net architecture is configured as the main coding transition connection layer, and the remaining main coding network layers are respectively used as main coding processing layers. The main decoding unit includes several main decoding processing layers connected in sequence, wherein... Within the main unit of the segmentation network, the main encoding processing layer in the main encoding unit and the main decoding processing layer in the main decoding unit are in one-to-one correspondence, and the main encoding processing layer is connected to the corresponding main decoding processing layer by a channel-spatial attention module. The main encoding transition connection layer is adapted to the main decoding processing layer at the bottom of the U-Net architecture within the main decoding unit, and the generated main encoding processing feature map is transmitted to the corresponding connected main decoding processing layer through the main encoding transition connection layer. The main decoding processing layer at the bottom of the U-Net architecture also receives the auxiliary encoding embedded feature map. When the main decoding unit performs decoding, it uses the main decoding processing layer at the bottom of the U-Net architecture to perform the first decoding process, and configures the remaining main decoding processing layers to perform the second decoding process. The main coding network layer includes a main coding convolutional block and a main coding Ghost module connected in sequence, wherein, When the main coding unit performs the first coding process, it sequentially extracts coding features through the main coding network layers to perform the first coding subprocessing using each main coding network layer. Specifically, during coding feature extraction, the main coding convolution block performs coding convolution processing first, and after coding convolution processing, the main coding Ghost module performs feature extraction. After feature extraction by the main coding Ghost module, the corresponding network layer feature map is generated. Within the main coding unit, if the current main coding network layer is not used as a main coding transition connection layer, the generated network layer feature map should be transmitted to the adjacent main coding network layer. At the same time, the network layer feature map should be transmitted to the corresponding channel-spatial attention module. Thus, for two adjacent main coding network layers, along the direction from the opening of the U-Net architecture to the bottom, the network layer feature map generated by the upper main coding network layer is max-pooled to generate a max-pooled feature map, and the max-pooled feature map is loaded into the lower main coding network layer. When the first decoding process is performed using the main decoding processing layer at the bottom of the U-Net architecture, then: The main decoding processing layer receives the main encoding deconvolution processing feature map and loads the received main encoding sampled feature map into the corresponding connected channel-space attention module. The channel-space attention module also receives the network layer feature map output by the corresponding connected main encoding network layer. Based on the received main encoding deconvolution processing feature map and network layer feature map, the channel-space attention module performs attention mechanism processing to generate the main encoding attention processing feature map after the attention mechanism processing. The auxiliary encoding embedded feature map, the main encoding deconvolution processed feature map, and the main encoding attention processed feature map are concatenated. After feature concatenation, the feature map is processed by Ghost to generate the decoding baseline feature map.

2. The image segmentation method suitable for underground railway track images in coal mines according to claim 1, characterized in that, The main decoding processing layer, which performs the second decoding process, includes a decoding deconvolution block and a main decoding Ghost module connected in sequence. When performing the second decoding process, the basic decoding feature map to be decoded is received. Then, the basic decoding feature map is deconvolved using a decoding deconvolution block to generate a deconvolved decoding feature map. The deconvolutioned decoded feature map is loaded into the channel-spatial attention module of the current main decoding processing layer, and the channel-spatial attention module also receives the network layer feature map corresponding to the output of the main coding network layer; Based on the received deconvolution-decoded feature map and network layer feature map, the channel-space attention module performs attention mechanism processing to generate a decoded attention processing feature map after the attention mechanism processing. The main decoding Ghost module is used to extract features from the decoding attention processing feature map to generate a basic decoding feature map after feature extraction. Within the main decoding unit, for two adjacent main decoding processing layers, along the direction from the bottom of the U-Net architecture toward the opening, the basic decoding feature map is generated by the main decoding processing layer below and transmitted to the main decoding processing layer above.

3. The image segmentation method suitable for underground railway track images in coal mines according to claim 1, characterized in that, The channel-space attention module includes a first attention splicer, a channel attention module, a spatial attention module, and an attention linear layer, wherein... The attention-based first splicer is connected to the corresponding main encoding and decoding processing layers; The channel attention module is connected to the output of the first attention splicer, and the channel attention module uses a residual connection; The spatial attention module is connected to the output of the channel attention module, and the spatial attention module uses residual connection; The output of the spatial attention module is connected to the attention linear layer, and the attention linear layer is used as the output layer of the channel-spatial attention module.

4. The image segmentation method suitable for underground railway track images in coal mines according to claim 3, characterized in that, The channel attention module includes a first max pooling module for channel attention and a first average pooling module for channel attention, wherein... Both the first channel attention max pooling module and the first channel attention average pooling module are connected to the output of the first attention splicer. The outputs of the first maximum pooling module and the first average pooling module of channel attention are connected to the first linear module of channel attention. The first linear module of channel attention is connected to the second linear module of channel attention through the channel attention LeakyReLu activation function. The output of the second linear channel attention module is connected to the second maximum pooling module and the second average pooling module of channel attention, and both the second maximum pooling module and the second average pooling module of channel attention are connected to the channel attention splicer. The output of the channel attention splicer is connected to the channel attention adder, which is used to form the residual connection of the channel attention module, through the channel attention Sigmoid function, and is adapted to the spatial attention module through the channel attention adder.

5. The image segmentation method suitable for underground railway track images in coal mines according to claim 3, characterized in that, The spatial attention module includes a first spatial attention convolutional block, a normalized activation function module, a second spatial attention convolutional block, a normalization module, and a spatial attention sigmoid function connected sequentially. The first convolutional block of spatial attention is connected to the output of the channel attention adder; The first and second convolutional blocks of spatial attention use the same kernel size; When the spatial attention module uses residual connections, the spatial attention sigmoid function is connected to the input of the spatial attention adder, and the input of the first spatial attention convolutional block is also connected to the input of the spatial attention adder. The output of the spatial attention adder is connected to the attention linear layer.

6. The image segmentation method suitable for underground railway track images in coal mines according to any one of claims 1 to 3, characterized in that, The segmentation network auxiliary unit includes a SAM encoding unit and a size adjustment unit adapted and connected to the SAM encoding unit, wherein... The underground track target image is encoded using SAM coding units, and a basic coded embedding feature map is generated after the encoding process. The generated basic encoding embedding feature map is resized using a size adjustment unit to generate an auxiliary encoding embedding feature map. The feature map size of the auxiliary encoding embedding feature map is consistent with the feature map size of the main encoding deconvolution processing feature map and the main encoding attention processing feature map.

7. The image segmentation method suitable for underground railway track images in coal mines according to any one of claims 1 to 3, characterized in that, When acquiring images of underground orbital targets, the following steps are included: The source image of the underground track is acquired, and the acquired source image is preprocessed to generate the target image of the underground track. Preprocessing of the underground track source images includes bilateral filtering and / or contrast-limited adaptive histogram equalization enhancement. When performing contrast-limited adaptive histogram equalization enhancement processing, including Obtain the source image to be balanced and enhanced, and divide the source image into several source image sub-blocks. For any source image sub-block, then: Where (b1,b2) is the size of each source image sub-block, (H,W) is the size of the source image to be equalized and enhanced, K is the number of source image sub-blocks, and the source image for equalization and enhancement is the underground track source image or the underground track filtered image generated after bilateral filtering. Contrast limiting is applied to each source image sub-block, and the contrast limiting process includes: For a source image sub-block T, a grayscale histogram of source image sub-block T is generated, and the frequency of the i-th grayscale level is restricted based on the generated grayscale histogram, then: Where DB is the frequency limiting threshold, and H T (i) represents the frequency of the source image sub-block T at the i-th gray level, and L represents the number of gray levels of the source image to be equalized and enhanced. After frequency limiting and mirroring grayscale mapping, we have: in, Let i be the gray level mapping for the i-th gray level; Let be the cumulative distribution function of the i-th gray level; After the mirrored grayscale mapping process, a mapped grayscale histogram is generated based on the grayscale mapping information, and a corresponding contrast-limited sub-block is generated based on the mapped grayscale histogram. Any two contrast-limited sub-blocks are merged using linear interpolation to generate an image of the underground track target.

Images