A map line feature extraction method based on deep learning

Abstract

The application discloses a kind of based on deep learning map line feature extraction method, the based on deep learning map line feature extraction method includes: including collection color electronic map and its corresponding vector data image;Vector data file is preprocessed as label data;Label data is divided into line feature data and background data;Improved convolutional neural network is constructed, including based on U_Net network, join convolution attention mechanism module, the module can increase the weight of map line feature, join hollow convolution module, the module has different receptive field size, obtains different scale feature information;Improved network is carried out map line feature extraction training;Map line feature extraction model is generated;The map of line feature to be extracted is extracted using the trained model;The application realizes map line feature automatic extraction, and provides basis for the vectorization of line feature.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology for map line feature extraction, and specifically relates to a map line feature extraction method based on deep learning. Background Technology

[0002] Maps are a graphic form of spatial information transmission and a tool for information dissemination. They possess the unique ability to represent the real world in multiple ways, serving as an important medium for facilitating spatial understanding. Maps play a vital role in national economic development, national defense, and daily life.

[0003] Maps are a crucial source of geographic data. To fully utilize map data, it needs to be scanned into raster maps and then converted into vector data that is easily queried and analyzed by computers. Map vectorization, the process of converting raster maps into vector maps, is an important method for data acquisition in Geographic Information Science (GIS). Traditional map vectorization methods are completed through human-computer interaction, requiring users to create new layers, draw points, lines, and polygons, and add attributes using a mouse while referring to the raster map. When the number of maps to be vectorized is large, this method requires significant manpower and resources. Improving automation and increasing vectorization speed are urgent problems to be solved for rapid acquisition of geospatial data. Line extraction is a key step in automatic map vectorization; after extraction, lines can be easily converted into lines and polygons using raster-to-vector conversion methods.

[0004] In terms of map line feature extraction, traditional algorithms are used, requiring manual setting of thresholds. Adjusting these thresholds may generate additional noise. Furthermore, manual refining, deletion, and addition are necessary to achieve satisfactory results. Traditional algorithms for line feature extraction suffer from drawbacks such as the need for manual modification and unsatisfactory performance.

[0005] In recent years, with the development of computer technology, artificial intelligence and deep learning have developed particularly rapidly. Fully convolutional neural networks can effectively extract image features. Therefore, this invention uses U_Net fully convolutional neural network combined with attention mechanism and dilated convolution module to extract map line features. Summary of the Invention

[0006] To address the shortcomings of current methods for extracting line features from color electronic maps, the purpose of this invention is to provide a method for extracting map line features based on convolutional neural networks.

[0007] A method for extracting map line features based on deep learning, characterized by the following steps:

[0008] S1: Data Collection and Vector File Preprocessing: Collect color electronic maps and their corresponding vector data images; preprocess the vector data files as label data;

[0009] S2: Creating sample and label datasets: cropping the color electronic map images and corresponding label files to a size suitable for training the network model; normalizing the image data matrix and dividing the dataset into training, validation, and test sets;

[0010] S3: Construct an improved convolutional neural network model: Based on the U_Net network, add a convolutional attention mechanism module, which can increase the weight of map line features and suppress noise such as background text, thus better extracting map line features; add a dilated convolution module, which has different receptive field sizes and can obtain feature information at different scales, which is more conducive to the extraction of line features.

[0011] S4: Based on the training and validation sets divided in S2, train the improved network to extract map line features;

[0012] S5: Map line feature extraction model: Select the optimal neural network model based on the changes in the accuracy of the loss function and the validation set;

[0013] S6: Line feature extraction: The map to be extracted for line features is cropped using the sample data generated in S1 and then predicted by the convolutional neural network model generated in S5. Finally, the cropped small map pieces are stitched together to obtain the complete map line feature extraction result.

[0014] Preferably, in S1, the vector file of the map should have the background layer, annotation layer, etc. removed, leaving only the line feature layer;

[0015] Preferably, when converting the vector file into a JPG or PNG image file in step S1, the image size should be consistent with the image size of the color electronic map;

[0016] Preferably, in step S1, the vector file converted into image data is binarized, the pixel value of the line features is set to 255, and the pixel value of the background is set to 0.

[0017] Preferably, in step S2, the color electronic map and the corresponding tag file are cropped using a sliding window of 256x256 pixels to obtain image blocks;

[0018] Preferably, in step S2, the image data is normalized by converting the range of the image matrix pixel values ​​of the color electronic map to [0-1]; converting the line feature value with a pixel value of 255 in the label data to 1, while keeping the background value of 0 unchanged;

[0019] Preferably, the convolutional neural network constructed in S3 is based on an encoder-decoder structure. The encoder extracts the features of line elements in the map, and the decoder restores the features extracted by the encoder. An attention mechanism module is added between the skip connections between the encoder and the decoder, which increases the weight of the line element features. A dilated convolution module is added to the last layer of the encoder, which uses different receptive fields to extract the features of line elements. The probability result map of each class is obtained in the last layer of the decoder through 1x1x1 convolution and Softmax activation function.

[0020] Preferably, the loss function in S5 is binary_crossentropy, and the accuracy evaluation index is the Dice coefficient;

[0021] Preferably, the encoder part of the U_Net network consists of three blocks, each block being composed of two 3x3 convolutional ReLU activation functions. Max pooling layers connect the two blocks. A dilated convolutional module is connected after the last block of the encoder, with the dilation rate set to 1, 2, 4, and 8 in sequence. Features extracted from different receptive fields are then superimposed to obtain the final features. The decoder part also consists of three blocks, each block being composed of a 2x2 deconvolutional module with a stride of 2 and two 3x3 convolutional modules. The encoder and decoder are then used to encode the features. The skip connections between decoders pass through an attention module, which performs convolution operations on the input features and gating signals with their corresponding weights, then superimposes the results, passes them through the ReLU activation function, performs a 1×1×1 convolution, and then passes them through the Sigmoid activation function to obtain attention coefficients. The attention coefficients are multiplied by the input features to obtain the final output features, which are then connected to the corresponding decoder layer features. At the end of the decoder, a 1×1×1 convolution is connected to convert the number of channels in the feature map into the number of categories. Finally, the feature map is converted into the model's prediction probability map through the Softmax activation function.

[0022] The beneficial effects of this invention are:

[0023] 1) This invention uses U_Net as a base, which, compared to other networks, achieves accurate segmentation results with a small amount of training data;

[0024] 2) This invention introduces a dilated convolution module, which uses different receptive fields to extract line features and can obtain line feature features at different scales, which is more conducive to the extraction of map line features.

[0025] 3) This invention introduces an attention mechanism module, which can increase the weight of map line features and improve the extraction accuracy of line features;

[0026] In summary, the advantages of this invention are as follows: traditional map line feature extraction methods require manual drawing and the implemented algorithms require setting certain thresholds, resulting in a low degree of automation; while the deep learning-based map line feature extraction method can automatically extract the features of line features, reducing manual intervention and meeting certain accuracy requirements. Attached Figure Description

[0027] Figure 1 This is a flowchart of map line feature extraction in an embodiment of the present invention;

[0028] Figure 2 This is the line feature extraction network structure based on the U_Net network fusion attention mechanism and dilated convolution module in this embodiment of the invention;

[0029] Figure 3 This refers to the dilated convolution module in this embodiment of the invention.

[0030] Figure 4 This refers to the convolutional attention mechanism in this embodiment of the invention. Detailed Implementation

[0031] To clearly and completely describe the technical solution and effects of the present invention, the following embodiments are provided in detail.

[0032] Example 1

[0033] The environment used in the embodiments of the present invention is as follows: the server's CPU is an Intel(R) Xeon(R) CPU@2.20GHz, the GPU is a Tesla P100, the operating system is Ubuntu 18.04.5, and the compilation environment is Python 3.7.10, TensorFlow 2.4.1, and CUDA 11.2;

[0034] S1: Data collection and vector file preprocessing: This includes collecting color electronic maps and their corresponding vector data images; preprocessing the vector data files as label data;

[0035] First, collect color map images and their corresponding vector images. Remove all other layers from the vector images, leaving only the line feature layer. Then, convert the vector file containing only the line feature layer into a JPG or PNG image. Next, binarize the image, setting the background to 0 pixels and the line feature to 255 pixels.

[0036] S2: Creating sample and label datasets: This includes cropping color electronic map images and corresponding label files to a size suitable for training the network model; normalizing the image data matrix; and dividing the dataset into training set, validation set, and test set.

[0037] First, the color electronic map and its corresponding label data are cropped to a size of 256×256 pixels. The cropping method is to randomly crop the image using a sliding window of size 256. Then, the cropped dataset is normalized using the formula (1). Finally, it is divided into a training set and a validation set in a ratio of 7:3.

[0038]

[0039] Where x i σ represents the pixel input value of the image, μ is the average value of all image pixel values, and σ is the standard deviation of all image pixel values.

[0040] S3: Constructing an improved convolutional neural network model: Based on the U_Net network, an attention mechanism module is added to increase the weight of line features in the image; a dilated convolution module is added to extract line features using different receptive fields, obtaining line feature features at different scales. Adding these two modules makes line feature extraction more efficient.

[0041] The deep learning-based map line feature extraction network used in this embodiment is shown in the attached figure. Figure 2 , 3 As shown in Figure 4; u-net is the encoder / decoder structure, encoding its structure ( Figure 2 The left side extracts line feature characteristics from the map, and the decoder structure ( Figure 2 The right side restores the features extracted by the encoder; an attention mechanism module is added to the skip connection between the encoder and the decoder, and a dilated convolution module is added to the last layer of the encoder; in the last layer of the decoder, the restored feature map is passed through the Softmax activation function to obtain the probability result map of each class, and finally the class with the highest probability is the class to which the current pixel belongs.

[0042] In this embodiment, the input consists of an RGB color map image and a binarized label image, both 256×256 pixels in size; the encoder uses the first three blocks of the UNet network, each block consisting of two 3×3 convolutional modules. Figure 2 (Blue arrow in the middle) Each convolutional module includes 3×3 convolution operations, batch normalization, and ReLU activation function operations; two blocks are connected by a max pooling layer. Figure 2(Orange arrow in the middle) The pooling window size is 2×2 with a stride of 2. Pooling can reduce the size of the feature map by half, which allows for deeper extraction of image features. The three blocks all use 3×3 kernels, and the feature maps of each block are concatenated on the channel using a skip connection and upsampling from the decoder, which can incorporate more contextual information.

[0043] To obtain features at different scales, a dilated convolutional module is added to the last layer of the encoder. Figure 3 This module adds holes to a regular convolution kernel. Different levels of dilated convolution kernels increase the receptive field without increasing computation. The calculation method of dilated convolution is shown in Equation 2. This structure uses different sampling layers in the coding layer as input. The feature map generated after calculation is summed with the output of the corresponding upsampling layer as the input of the next upsampling layer. The dilated convolution structure uses dilated convolution with sampling levels of 1, 2, 4, and 8 to perform convolution operations on the input feature map. The receptive field size of each layer is 3, 7, 15, and 31, respectively. By using feature maps with different sampling levels to input the model for calculation, different scale features are obtained, and finally, the features are fused together. The multi-scale spatial information of the feature map is fully extracted and utilized to adapt to the extraction of line features in the map.

[0044]

[0045] In the formula, y[i,j] is the output of the dilated convolution, x[i,j] is the input, ω[k] is the convolution kernel of size k, and R represents the different sampling levels of the convolution kernel;

[0046] Attention mechanisms (AGs) are added to the skip connections between the encoders and decoders at each layer. Figure 4 The idea behind AGS is to enhance the learning ability of convolutional neural network models for line features by increasing the weight of line features in color electronic maps; thereby suppressing noise such as text in the map background and improving the extraction effect of map line features; input features and strobe signal g i and their corresponding weight values and Perform convolution operations, then add them together, apply the ReLU activation function, and then perform a 1x1x1 convolution operation using the convolution function ψ to obtain... Again The final attention coefficients are obtained using the sigmoid activation function. Finally, the obtained attention coefficients and input features are... Multiplication yields the final output features. The formulas for calculating the attention coefficient in the attention mechanism of the AGs module are (Equation 3) and (Equation 4), respectively.

[0047]

[0048]

[0049] Where Ψ represents a convolution function of size 1x1x1; σ1 represents the ReLU activation function; W x Input features The corresponding weight value; For input features; W g For the gating signal g i The corresponding weight value; g i b is the strobe signal; g b is the bias value of the strobe signal; Ψ σ1 represents the bias value corresponding to the 1x1x1 convolution function; σ2 represents the Sigmoid activation function.

[0050] The decoder section also consists of three blocks symmetrical to the encoder. Each block comprises a 2×2 deconvolution with a stride of 2 and two 3×3 convolutional modules. Each convolutional layer is followed by batch normalization and ReLU operation. After passing through the deconvolutional layer, the feature map's size is doubled and the number of channels is halved, which is called upsampling. The upsampled feature map is then concatenated with the feature map of the corresponding block in the encoder by channel and input into the 3×3 convolutional layer. In the last block of the decoder, a 1×1 convolution is added to convert the number of channels in the feature map into the number of categories. Finally, the Softmax activation function is used to transform the feature map into the model's prediction probability map.

[0051] Step 4: Train the improved network for map line feature extraction: Set appropriate network training parameters, select a suitable optimizer and loss function, and feed the data processed in Step 2 into the convolutional neural network built in Step 3 for training; During training, set the Adam optimizer as the method for parameter gradient descent. This optimizer comprehensively considers the first and second moment estimates of the gradient and calculates the update step size; Set the learning rate, which is a hyperparameter of the network weights adjusted by the gradient of the loss function. The initial learning rate is set to 0.001. Set a loss plateau; when the loss rate does not change much, change the learning rate; When the training iterations are 10, if the loss rate does not change much, reduce the learning rate by learning rate x 0.1; Set the minimum value at which the learning rate can be reduced to 0.000001; Set an early stopping mechanism to avoid overfitting during network training.

[0052] S5: Generate map line feature extraction model: Based on the changes in loss value and accuracy during training, select the optimal network model for saving; the defined loss function is the binary cross entropy loss function as shown in equation (5); the accuracy evaluation method is to select the Dice coefficient (Dice Similarity Coefficient, DSC). The Dice coefficient is an index of set similarity measurement, used to calculate the similarity between two samples, and the value range is [0-1]; the closer the value is to 1, the better the accuracy and the higher the accuracy of the extracted line features;

[0053]

[0054] Where y represents the true label value, and y' represents the output of the activation function, whose value is between 0 and 1;

[0055] S6: Line Feature Extraction: The map image to be extracted for line features is preprocessed in S2 and cropped into 256×256 pixel image blocks. The model saved in S5 is then used for prediction. The prediction results are then stitched together to restore the large map image, resulting in the final extraction results of map line features.

[0056] In practice, the method can be automated using computer software technology, and the corresponding system devices for implementing the method are also within the scope of protection of this invention.

[0057] By applying this patent, a convolutional neural network is used to extract map line features. A convolutional attention mechanism is introduced to increase the weight of line features. A dilated convolution module is introduced to extract features at different scales, thereby improving the effectiveness of line feature extraction.

[0058] The embodiments described in this invention are merely illustrative examples of the principles and implementation steps of the invention. Furthermore, those skilled in the art can make corresponding modifications or additions to the described embodiments without departing from the spirit and scope of the invention. Therefore, the content of this specification should not be construed as limiting the invention.

Claims

1. A method for extracting map line features based on deep learning, characterized in that, Includes the following steps: S1: Data Collection and Vector File Preprocessing: Collect color electronic maps and their corresponding vector data images; preprocess the vector data files as label data; S2: Creating sample and label datasets: cropping the color electronic map images and corresponding label files to a size suitable for training the network model; normalizing the image data matrix and dividing the dataset into training, validation, and test sets; S3: Construct an improved convolutional neural network model: Based on the U_Net network, add a convolutional attention mechanism module, which can increase the weight of map line features and suppress background text noise, thus better extracting map line features; add a dilated convolution module, which has different receptive field sizes and can obtain feature information at different scales, which is more conducive to the extraction of line features. S4: Based on the training and validation sets divided in S2, train the improved network to extract map line features; S5: Map line feature extraction model: Select the optimal neural network model based on the changes in the accuracy of the loss function and the validation set; S6: Line Feature Extraction: The map for which line features need to be extracted is processed by the sample data generated in S1, cropped, and then predicted by the convolutional neural network model generated in S5. Finally, the cropped small map pieces are stitched together to obtain the complete map line feature extraction result.

2. The method for extracting map line features based on deep learning according to claim 1, characterized in that, The vector file of the map in S1 should have its background layer and annotation layer removed, leaving only the line feature layer.

3. The method for extracting map line features based on deep learning according to claim 1, characterized in that, In step S1, when converting the vector file into a JPG or PNG image file, the image size must be consistent with the image size of the color electronic map.

4. The method for extracting map line features based on deep learning according to claim 1, characterized in that, In step S1, the vector file converted into image data is binarized, the pixel value of the line features is set to 255, and the pixel value of the background is set to 0.

5. The method for extracting map line features based on deep learning according to claim 1, characterized in that, In step S2, the color electronic map and the corresponding tag file are cropped using a sliding window of 256x256 pixels to obtain image blocks.

6. The method for extracting map line features based on deep learning according to claim 1, characterized in that, In step S2, the image data is normalized. This requires converting the range of image matrix pixel values ​​of the color electronic map to [0, 1]; converting the line feature value with a pixel value of 255 in the label data to 1, while keeping the background value of 0 unchanged.

7. The method for extracting map line features based on deep learning according to claim 1, characterized in that, The convolutional neural network constructed in S3 is based on an encoder-decoder structure. The encoder extracts the features of line features from the map, and the decoder restores the features extracted by the encoder. An attention mechanism module is added between the skip connections between the encoder and the decoder, which increases the weight of the line feature features. A dilated convolution module is added to the last layer of the encoder, which uses different receptive fields to extract the features of line features. The probability result map of each class is obtained by a 1x1x1 convolution and a softmax activation function in the last layer of the decoder.

8. The method for extracting map line features based on deep learning according to claim 1, characterized in that, In S5, the loss function is binary_crossentropy, and the accuracy evaluation metric is the Dice coefficient.

9. The method for extracting map line features based on deep learning according to claim 1, characterized in that, The encoder part of the U_Net network consists of 3 blocks, each block is composed of two 3x3 convolutional ReLU activation functions, and the two blocks are connected by a max pooling layer. After the last block of the encoder, a dilated convolutional module is connected, and the dilation rate is set to 1, 2, 4 and 8 at a time. The features extracted from different receptive fields are then superimposed to obtain the final features. The decoder also consists of three blocks. Each block comprises a 2x2 deconvolution with a stride of 2 and two 3x3 convolutional modules. The skip connection between the encoder and the decoder passes through an attention module. This module performs convolution operations on the input features and gating signals with their corresponding weights, then superimposes the results, applies a ReLU activation function, performs a 1x1x1 convolution, and then applies a Sigmoid activation function to obtain attention coefficients. These attention coefficients are multiplied by the input features to obtain the output features, which are then connected to the corresponding decoder layer features. Finally, a 1x1x1 convolution is applied at the end of the decoder to convert the number of channels in the feature map into the number of categories. Finally, a Softmax activation function is applied to transform the feature map into the model's predicted probability map.

Images