Road extraction method fusing Schaar preprocessing and attention UNet

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a road extraction method fusing Schaar preprocessing and attention UNet, which comprises two main parts, namely an encoder and a decoder. The encoder and the decoder form an EEAU-Net network model; the encoder adopts two-channel input; the decoder introduces a convolution attention module in an original up-sampling process; according to the method, the Scharr operator is utilized to extract image edge features, the image edge features are fused with an original image, a learnable weighting parameter is introduced to flexibly control the influence of edge information, in addition, residual connection is adopted in the decoding stage to fuse the original and edge features, and a convolution attention module is introduced in a decoding module, so that the decoding efficiency is improved. Compared with an existing road extraction method, the EEAU-Net network model combining traditional edge detection and a convolution attention module has the advantages that a good effect is achieved in mining area road extraction, and the EEAU-Net network model is more suitable for being applied to mining area road extraction.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically is a road extraction method integrating Schaar preprocessing and attention UNet. Background Art

[0002] As global mining activities continue to grow, the demand for mineral resources is increasing, which not only promotes economic development, but also brings many environmental and social challenges. In this context, efficient mine management becomes particularly important, and one of the key factors is the road network in and around the mine. As an important part of the mine transportation system, non-structural roads (such as temporary mine car routes) play an indispensable role in ensuring material flow, personnel movement, and emergency response. Therefore, how to automatically, accurately and efficiently obtain road information is particularly critical for optimizing mine production.

[0003] Research on road extraction has a long history. Taking the success of Krizhevsky et al. in the ImageNet Challenge using deep learning methods as a time watershed, the development of road extraction methods can be divided into two stages: the first is the traditional method stage. In this stage, road extraction mainly uses three traditional methods, namely template matching, knowledge-driven, and object-oriented, and the time span can be extended from 1973 to 2012. Among them, the template matching method is widely used because of its simplicity and efficiency. Commercial software such as Erdaseasytrace and FeatureAnalyst have adopted this method. The object-oriented method has been applied in the eCognition software by decomposing the image into multiple meaningful areas for analysis. In contrast, although the knowledge-driven method can provide more accurate results, its commercialization degree is relatively low because it requires the pre-setting of multiple prior conditions.

[0004] With the development and application of deep learning technology, deep learning methods have made a series of breakthroughs in image classification, object recognition and other fields. Since 2012, some scholars have introduced deep learning technology into the research of road extraction. Compared with traditional template matching, knowledge-driven and object-oriented methods, deep learning methods have significantly improved the accuracy of road extraction. However, although these new methods have shown great potential, most of them are still in the research and experimental stage, and further research and development are needed before they can be widely used and popularized.

[0005] However, most of these studies are aimed at structured roads in cities or villages. In these road images, the road features are clear, the road boundaries are obvious, and the road shapes are standardized. In the mining environment, road-related information is often not clear enough, especially the road boundaries are not as obvious as the former. This results in the above methods being unable to identify and apply the roads in mining areas well.

[0006] Therefore, to address the above problems, a road extraction method that combines Schaar preprocessing and attention UNet is proposed. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] In view of the shortcomings of the prior art, the present invention provides a road extraction method that integrates Schaar preprocessing and attention UNet to solve the problems raised in the above background technology.

[0009] (II) Technical solution

[0010] To achieve the above object, the present invention provides the following technical solutions: a road extraction method integrating Schaar preprocessing and attention UNet, comprising two main parts, an encoder and a decoder;

[0011] The encoder has dual-channel input;

[0012] The decoder introduces a convolutional attention module during the original upsampling process.

[0013] Preferably, one input channel of the encoder is composed of a superposition of the original image and the image enhanced using the Scharr operator;

[0014] The second input channel of the encoder is the original road image.

[0015] Preferably, the dual-channel input contents of the encoder are connected in a residual manner, and the input information can be directly added to the output.

[0016] Preferably, the Schaar operator uses two sets of 3x3 convolution kernels, which are used to detect edges in the horizontal direction and the vertical direction respectively. The two sets of convolution kernels are optimized to improve the accuracy and rotation invariance of edge detection while maintaining simple calculations. The expressions are as follows:

[0017]

[0018] Where A represents the original image, Gx and Gy represent the grayscale values ​​of the image after horizontal and vertical edge detection, respectively, and * represents the basic convolution calculation between the input image and the two filters;

[0019] The grayscale value of each point is determined by combining the horizontal and vertical grayscale values ​​of each pixel in the image using the following formula. When the gradient G exceeds a certain threshold, the point (x, y) is classified as an edge point. The expression is as follows:

[0020]

[0021] Preferably, the application of the Scharr operator introduces a dynamically adjustable alpha coefficient to perform weighted fusion of the original image and the enhanced image, and can find the best balance between edge information and the original image under different input conditions. The expression is as follows:

[0022] I enhanced =(1-α)·I+α·S(I);

[0023] Where I represents the input image; S(I) represents the edge enhanced image obtained by the Scharr operator; α is a learnable coefficient that controls the fusion ratio of the original image and the edge enhanced image; Ienhanced represents the enhanced image after fusion.

[0024] Preferably, mathematically, the residual connection adds the input x and the output F(x) obtained through the network layer to obtain the final output y, which is expressed as follows:

[0025] y=F(x)+x;

[0026] Among them, F(x) is the tensor after a series of transformations, and x is the input tensor, which is directly added to the output to form the residual.

[0027] (III) Beneficial effects

[0028] Compared with the prior art, the present invention provides a road extraction method integrating Schaar preprocessing and attention UNet, which has the following beneficial effects:

[0029] The present invention proposes a road extraction method integrating Schaar preprocessing and attention UNet. The edge perception ability of the model is enhanced by introducing the Scharr operator. The edge features of the image are first extracted using the Scharr operator and fused with the original image. A learnable weighting parameter is introduced to flexibly control the influence of the edge information. In addition, the residual connection is used to fuse the original and edge features in the decoding stage, and a convolutional block attention module (CBAM) (Woo et al., 2018) is introduced in the decoding module to enhance the model's attention to key edge details. Compared with the existing road extraction methods, the EEAU-Net (Edge-EnhancedAttentionU-Net) network model combining traditional edge detection and convolutional attention module proposed in the present invention has achieved good results in mining road extraction and is more suitable for use in mining road extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0031] Figure 1 It is a structural diagram of the EEAU-Net network model of the present invention;

[0032] Figure 2 This is a schematic diagram of a mining area road image obtained by processing with the Scharr operator in the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Specific examples are given below.

[0035] See also Figure 1 and Figure 2 ,The present invention provides a road extraction method integrating Schaar preprocessing and attention UNet, including two main parts: an encoder and a decoder;

[0036] The encoder has dual-channel input;

[0037] The decoder introduces a convolutional attention module during the original upsampling process.

[0038] In the above solution, in the decoder part, a convolutional attention module is introduced in the original upsampling process to enhance the model’s attention to the details of road information in deep features, thereby improving the model’s extraction effect on road images.

[0039] Further Figure 1 As shown, one input channel of the encoder consists of the superposition of the original image and the image enhanced by the Scharr operator;

[0040] The second input channel of the encoder is the original road image.

[0041] The above scheme: the encoder part is improved from single-channel input to dual-channel input. One input consists of the superposition of the original image and the image enhanced by the Scharr operator, which can more effectively extract road edge features. However, due to the loss of some details in the superimposed data, the other input is the original road image to ensure the restoration of image details.

[0042] Compared with ordinary road images, mining road images have fewer boundary details and more blurred boundary contours, which brings challenges to image segmentation and target recognition. In order to alleviate the impact of this problem on the model effect, this study uses the Schaar operator to enhance the original image to strengthen the boundary information in the image. The Schaar operator is a discrete difference operator used for edge detection. Its core principle is to calculate the gradient (edge ​​strength and direction) of the image by convolution kernel convolution operation, so as to detect the edge in the image.

[0043] Further Figure 1 As shown, the dual-channel input content of the encoder is connected in a residual manner, and the input information can be directly added to the output.

[0044] The above solution: In order to avoid excessive modification of the original features that may be caused by convolution while ensuring the stability of gradient propagation, thereby improving training efficiency, the contents of the two paths are connected in a residual manner.

[0045] Further Figure 1 and Figure 2 As shown in the figure, the Schaar operator uses two sets of 3x3 convolution kernels to detect edges in the horizontal and vertical directions respectively. The two sets of convolution kernels are optimized to improve the accuracy and rotation invariance of edge detection while maintaining simple calculations. The expressions are as follows:

[0046]

[0047] Where A represents the original image, Gx and Gy represent the grayscale values ​​of the image after horizontal and vertical edge detection, respectively, and * represents the basic convolution calculation between the input image and the two filters;

[0048] The grayscale value of each point is determined by combining the horizontal and vertical grayscale values ​​of each pixel in the image using the following formula. When the gradient G exceeds a certain threshold, the point (x, y) is classified as an edge point. The expression is as follows:

[0049]

[0050] The application of the Scharr operator introduces a dynamically adjustable alpha coefficient to perform weighted fusion of the original image and the enhanced image, which can find the best balance between edge information and the original image under different input conditions. The expression is as follows:

[0051] I enhanced =(1-α)·I+α·S(I);

[0052] Where I represents the input image; S(I) represents the edge-enhanced image obtained by the Scharr operator; α is a learnable coefficient that controls the fusion ratio of the original image and the edge-enhanced image.

[0053] Ienhanced represents the enhanced image after fusion.

[0054] The above scheme: The application of the Scharr operator enhances the edge information in the image and improves the clarity of the road edge. However, the result obtained in this way requires a certain price. The original information in the image will be partially lost, which will lead to loss of contrast and details. In order to reduce the impact of this problem, the present invention introduces a dynamically adjustable alpha coefficient to perform weighted fusion of the original image and the enhanced image. Through this weighted fusion, the model can adaptively adjust alpha during the training process, so as to find the best balance between edge information and the original image under different input conditions, and improve the model's perception of edge features;

[0055] The mining area road image obtained by using the Scharr operator is shown in the attached manual. Figure 2 As shown, the overlay image in the figure shows that by multiplying the original data and the Scharr data by α (α is 0.5 at this time), the clarity of the road edge is enhanced while retaining other details. This processing effect produces good results.

[0056] Further Figure 1 As shown, mathematically, the residual connection adds the input x and the output F(x) obtained through the network layer to obtain the final output y, which is expressed as follows:

[0057] y=F(x)+x;

[0058] Among them, F(x) is the tensor after a series of transformations, and x is the input tensor, which is directly added to the output to form the residual.

[0059] The above solution: In the original UNet network, since image information is transmitted step by step through each layer, each layer will perform nonlinear transformation on the input. However, in deep networks, as the number of layers increases, the gradient information will gradually decrease, that is, the gradient disappears, making the model difficult to train. The residual connection can directly add the input information to the output to help alleviate this problem.

[0060] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A road extraction method integrating Schaar preprocessing and attention UNet, characterized in that: It consists of two main parts: encoder and decoder; The encoder has dual-channel input; The decoder introduces a convolutional attention module during the original upsampling process; One input channel of the encoder consists of the superposition of the original image and the image enhanced using the Scharr operator; The second input channel of the encoder is the original road image; The dual-channel inputs of the encoder are concatenated in a residual manner.

2. A road extraction method integrating Schaar preprocessing and attention UNet according to claim 1, characterized in that: The Schaar operator uses two sets of 3x3 convolution kernels to detect edges in the horizontal and vertical directions respectively. Its expression is as follows: Where A represents the original image, Gx and Gy represent the grayscale values ​​of the image after horizontal and vertical edge detection, respectively, and * represents the basic convolution calculation between the input image and the two filters; The grayscale value of each point is determined by combining the horizontal and vertical grayscale values ​​of each pixel in the image using the following formula. When the gradient G exceeds a certain threshold, the point (x, y) is classified as an edge point. The expression is as follows:

3. A road extraction method integrating Schaar preprocessing and attention UNet according to claim 1, characterized in that: The application of the Scharr operator introduces a dynamically adjustable alpha coefficient to perform weighted fusion of the original image and the enhanced image. The expression is as follows: I enhanced =(1-α)·I+α·S(I); Where I represents the input image; S(I) represents the edge enhanced image obtained by the Scharr operator; α is a learnable coefficient that controls the fusion ratio of the original image and the edge enhanced image; Ienhanced represents the enhanced image after fusion.

4. A road extraction method integrating Schaar preprocessing and attention UNet according to claim 1, characterized in that: Based on mathematical principles, the residual connection adds the input x and the output F(x) obtained through the network layer to obtain the final output y, which is expressed as follows: y=F(x)+x; Among them, F(x) is the tensor after a series of transformations, and x is the input tensor, which is directly added to the output to form the residual.

Images