Shielded road feature extraction method, system and device and storage medium

Abstract

The invention belongs to the field of road feature extraction, and particularly relates to a blocked road feature extraction method, system and device and a storage medium, and the method comprises the steps: carrying out the multi-direction strip convolution of a separation feature of a road image after convolution kernel, obtaining a strip convolution feature, carrying out the horizontal and vertical strip convolution of a first output feature, obtaining a main branch feature, and obtaining a blocked road feature; obtaining a second output feature by combining the sampling separation feature, performing step-by-step down-sampling on the second output feature, performing feature extraction to obtain a sampling extraction feature, performing up-sampling on the sampling extraction feature, splicing the sampling extraction feature with a last sampling result, and performing channel weighting operation on the obtained sampling splicing feature to obtain a weighted splicing feature; and splicing the (n-1) th weighted splicing feature with the second output feature to obtain a final splicing feature. And judging the connectivity of the road by learning the relationship between adjacent pixels. Compared with the prior art, the shielded road can be clearly recognized, and the accuracy of road recognition is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of road feature extraction, and particularly relates to a method, system and device for extracting occluded road features and a storage medium. BACKGROUND

[0002] As an important part of infrastructure, roads play a crucial role in urban planning, traffic management, autonomous driving, emergency rescue and other fields. With the acceleration of urbanization and the continuous expansion of the transportation network, the demand for road information acquisition and updating is increasingly urgent.

[0003] In related technologies, road extraction mainly relies on traditional image processing methods such as morphological operations, edge detection (such as Sobel operator), texture analysis, etc. These methods identify roads by extracting features such as texture and edge in images.

[0004] In view of the above related technologies, the performance of identifying roads by extracting features such as texture and edge in images is limited by factors such as contrast between roads and backgrounds, noise, etc., and has poor adaptability to complex scenes. Moreover, it is not good enough to distinguish the occluded part of the road, which may incorrectly segment the occluded part as background. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a method, system, device and storage medium for extracting occluded road features, which can clearly identify the occluded part of the road and improve the accuracy of road feature recognition.

[0006] A method for extracting occluded road features, comprising:

[0007] obtaining a road image, wherein the road image is partially occluded;

[0008] performing convolution kernel on the road image to obtain separated features, and performing multi-directional strip convolution on the separated features to obtain a plurality of strip convolution features in different directions;

[0009] splicing the plurality of strip convolution features in different directions to obtain spliced features;

[0010] adding the spliced features and the separated features to obtain first output features;

[0011] performing horizontal and vertical strip convolution on the first output features to obtain main branch features;

[0012] down-sampling the road image to obtain sampling features, then performing feature separation on the sampling features to obtain sampling separated features, splicing the sampling separated features and the main branch features to obtain second output features;

[0013] The second output feature is downsampled n times in a step-by-step manner to obtain a third feature, wherein the step-by-step sampling is to use the sampling result of the previous sampling as input for the next sampling, perform feature extraction on the last sampling result to obtain a sampling extraction feature, upsample the sampling extraction feature and splice it with the previous sampling result to obtain a sampling splicing feature, perform a channel weighting operation on the sampling splicing feature to obtain a weighted splicing feature, splice the n-1th weighted splicing feature and the second output feature to obtain a final splicing feature, perform a channel weighting operation on the final splicing feature and then perform multi-scale feature extraction to obtain a final output feature;

[0014] The channel weighting operation includes:

[0015] The sampled splicing features are split into h groups along the channel dimension, the first h-1 groups are multiplied by the main elements of the learning parameters to obtain h-1 global semantic features, the h-th group is subjected to depthwise separable convolution to obtain depthwise convolution features, the h-1 global semantic features and the depthwise convolution features are spliced ​​to obtain recombined features, the recombined features are subjected to strip convolution in different directions and then spliced ​​to obtain weighted splicing features.

[0016] Specifically, the final output features can be restored to a road image through a decoder, and the restored road image can be used for road network updating. The road image is a remote sensing image taken by a satellite.

[0017] Specifically, the blocked portion of the road image is smaller than the unblocked portion, and the length and width of the unblocked portion of the road are determined by the resolution of the remote sensing image.

[0018] Optionally, performing convolution kernel separation on the road image to obtain separation features, and performing multi-directional strip convolution on the separation features to obtain strip convolution features in several different directions includes:

[0019] Set the convolution kernel size;

[0020] The road image is subjected to convolution kernel separation according to the convolution kernel size to obtain separation features, which are expressed as:

[0021] Z=k [n×n] ·X;

[0022] Among them, Z is the separation feature, X is the road image, n×n is the convolution kernel size, and K is the convolution kernel separation operation;

[0023] The separation feature is subjected to strip convolution in different directions to obtain several strip convolution features in different directions, which are expressed as:

[0024] z i =t [h,w]· Z;

[0025] wherein z i represents the i-th bar convolution feature, t [h,w] represents a bar convolution, h and w determine the direction of the bar convolution, h and w represent the horizontal offset and the vertical offset, respectively.

[0026] Optionally, the horizontal and vertical bar convolutions on the first output feature to obtain the main branch feature include:

[0027] performing a maximum pooling operation on the first output feature to obtain a pooling feature;

[0028] performing horizontal and vertical bar convolutions on the pooling feature to obtain the main branch feature, represented as:

[0029] y b = Maxpool(F i )·t 1×n ·t n×1 ;

[0030] wherein y b is the main branch feature, Maxpool is a maximum pooling operation, t 1×n represents a horizontal bar convolution, t n×1 represents a vertical bar convolution, n represents a convolution kernel size, and F i is the pooling feature.

[0031] Optionally, the down-sampling of the road image to obtain a sampling feature, then performing feature separation on the sampling feature to obtain a sampling separated feature, and splicing the sampling separated feature and the main branch feature to obtain a second output feature include:

[0032] the down-sampling of the road image to obtain a sampling feature, then performing feature separation on the sampling feature to obtain a sampling separated feature, represented as:

[0033] y res = DS a (X)·k n ·k n + DS(X);

[0034] wherein y res is the separated feature, DS a (X) represents down-sampling, a represents a down-sampling multiple, X is a road image, n represents a convolution kernel size, DS a (X)·k n ·k n represents feature separation, and DS(X) represents that the down-sampling multiple is 1.

[0035] The sampling separation feature and the main branch feature are spliced to obtain a second output feature, denoted as:

[0036] F i+1 =Cat(y res ·y b )·k1;

[0037] wherein Cat represents splicing, and k1 represents feature classification using a convolution kernel with a size of 1.

[0038] Optionally, the second output feature is split into h groups along the channel dimension, and the first h-1 groups are multiplied by a main element of a learning parameter to obtain h-1 global semantic features, including:

[0039] The learning parameter is obtained.

[0040] The second output feature is split into h groups along the channel dimension, and the first h-1 groups are multiplied by a main element of a learning parameter to obtain j-1 global semantic features, denoted as:

[0041] s j =e j ⊙p j (1≤j≤h-1);

[0042] wherein e j is the jth group of the split second output feature, p j is the learning parameter, and represents Hadamard product.

[0043] Optionally, the h-1 global semantic features and the deep convolution feature are spliced to obtain a reorganized feature, denoted as:

[0044] L i =Cat(SE(s j ));

[0045] wherein L i represents the reorganized feature, Cat represents splicing, SE represents a channel attention operation, s j represents the global semantic feature or the deep convolution feature, when j takes values [1, h-1], s j represents the global semantic feature, and when j takes value h, s j represents the deep convolution feature.

[0046] Optionally, the channel attention operation is denoted as:

[0047] Y=X·Sigmoid(LN(LN(Gp(X))));

[0048] Wherein, X is a road image, LN represents a full connection layer, Gp is a global average pooling, Sigmoid is an activation function, and Y is an output.

[0049] An occluded road feature extraction method, comprising:

[0050] An acquisition module is configured to acquire a road image, wherein the road image is partially occluded.

[0051] A first convolution module is configured to perform kernel convolution on the road image to obtain separated features, and perform multi-directional strip convolution on the separated features to obtain strip convolution features in different directions.

[0052] A first splicing module is configured to splice the strip convolution features in different directions to obtain spliced features.

[0053] An addition module is configured to add the spliced features and the separated features to obtain first output features.

[0054] A second convolution module is configured to perform horizontal and vertical strip convolution on the first output features to obtain main branch features.

[0055] A second splicing module is configured to down-sample the road image to obtain sampling features, perform feature separation on the sampling features to obtain sampling separated features, splice the sampling separated features and the main branch features to obtain second output features.

[0056] A feature fusion module is configured to perform n times of step-by-step down-sampling on the second output features to obtain third features, wherein the step-by-step sampling is to take the sampling result of the last time as the input for the next time of sampling, perform feature extraction on the sampling result of the last time to obtain a sampling extraction feature, splice the sampling extraction feature after up-sampling and the sampling result of the last time to obtain a sampling spliced feature, perform channel weighting operation on the sampling spliced feature to obtain a weighted spliced feature, splice the weighted spliced feature of the n-1th time and the second output features to obtain a final spliced feature, and perform multi-scale feature extraction on the final spliced feature after performing channel weighting operation again to obtain final output features.

[0057] The channel weighting operation comprises:

[0058] The sampling spliced feature is split into h groups along the channel dimension, the first h-1 groups are multiplied by a main element of a learning parameter to obtain h-1 global semantic features, the hth group is subjected to depth separable convolution to obtain a depth convolution feature, the h-1 global semantic features and the depth convolution feature are spliced to obtain a reorganized feature, and the reorganized feature is subjected to strip convolution in different directions and then spliced to obtain a weighted spliced feature.

[0059] A terminal device comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, a blocked road feature extraction method is adopted.

[0060] A computer readable storage medium stores a computer program, and when the processor loads and executes the computer program, a blocked road feature extraction method is adopted.

[0061] The beneficial effects of the present application are:

[0062] By separating the road image through the convolution kernel, the separated features are obtained, the separated features are subjected to multi-directional strip convolution, a plurality of strip convolution features in different directions are obtained, multi-scale extraction of road information is realized, then the first output feature is subjected to horizontal and vertical strip convolution, the main branch feature is obtained, the second output feature is obtained by combining the sampled separated features, the third feature is obtained by performing n times of step-by-step downsampling on the second output feature, the step-by-step sampling is to take the sampling result of the last sampling as the input for the next sampling, the sampling result of the last sampling is subjected to feature extraction to obtain the sampling extraction feature, the sampling extraction feature is upsampled and spliced with the sampling result of the last time to obtain the sampling splicing feature, the sampling splicing feature is subjected to channel weighting operation to obtain the weighted splicing feature, the final splicing feature is obtained by splicing the n-1th weighted splicing feature with the second output feature, and multi-scale feature extraction is performed on the final splicing feature after channel weighting operation to obtain the final output feature, the relationship between adjacent pixels is learned, and the connectivity of the road is judged. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 It is a flowchart of a blocked road feature extraction method of the present application;

[0064] Figure 2 It is a feature splicing flowchart of a blocked road feature extraction method of the present application;

[0065] Figure 3 It is an experimental result diagram of the present application on the DeepGlobe dataset

[0066] Figure 4 It is a visualization experimental diagram of the experiment of the present application on the DeepGlobe dataset;

[0067] Figure 5 It is an experimental result diagram of the present application on the CHN6 dataset;

[0068] Figure 6The visualization experimental diagram for the experiment of the application on the CHN6 dataset;

[0069] Figure 7 The ablation experiment diagram for the module composed of the steps of the application;

[0070] Figure 8 The experimental result diagram for the convolution kernel in the MFE module of the application;

[0071] Figure 9 The experimental result diagram for the convolution kernel in the MOD module of the application;

[0072] Figure 10 The comparative diagram of the heat map of the attention mechanism in the MOD of the application. DETAILED DESCRIPTION

[0073] A method for extracting occluded road features, as shown in Figure 1 , comprising:

[0074] S1, acquiring a road image, the road image being partially occluded;

[0075] Specifically, in the present application, the occluded part of the road image is smaller than the non-occluded part, the length and width of the non-occluded part of the road are determined by the resolution of the remote sensing image, and the method is suitable for roads that are partially occluded. According to the different resolutions, the length of the occluded part is also different. In the present application, the spatial resolution is 1 meter, the length of the occluded part cannot exceed 10 meters, and the width of the entire road cannot be less than 2 meters.

[0076] S2, separating the road image by a convolution kernel to obtain separated features, and performing multi-directional strip convolution on the separated features to obtain a plurality of strip convolution features in different directions;

[0077] Specifically, in most current researches, the traditional square convolution operation is usually relied on for channel expansion and early-stage feature processing. Although this method has shown effectiveness in general computer vision tasks, due to the interesting mismatch between the isotropic nature of the square convolution kernel and the anisotropic features of the road network, it has shown great limitations when applied to road segmentation. The symmetric receptive field of the traditional square convolution kernel is particularly insufficient for capturing the elongated and curved geometric feature of the road network.

[0078] This architectural limitation often leads to an inefficient representation of linear features in the initial encoding stage, where critical topological information can be lost. The loss of critical information in the initial processing stage can lead to insufficient feature extraction, ultimately resulting in partial road omissions in the final segmentation result. The elongated shape of the strip convolution highly matches the shape of the road and can achieve the extraction of linear road features, significantly reducing the interference of irrelevant information. However, real-world road networks exhibit complex direction changes, requiring adaptive feature capture. To address this issue, the MFE module proposed in this patent includes strip convolutions in multiple directions to comprehensively represent road features. This design significantly enhances the network's ability to retain critical structural information during initial feature extraction, effectively alleviating the common problem of early information loss.

[0079] The road image is separated by the convolution kernel to obtain separated features, and the separated features are subjected to multi-directional strip convolution to obtain a plurality of strip convolution features in different directions, including:

[0080] The size of the convolution kernel is set.

[0081] The road image is separated by the convolution kernel according to the size of the convolution kernel to obtain separated features, represented as:

[0082] Z = k [n×n] · X;

[0083] where Z is the separated feature, X is the road image, n x n is the size of the convolution kernel, the size is n*n, and K is the convolution kernel separation operation; the separated features are subjected to strip convolution in different directions to obtain a plurality of strip convolution features in different directions, represented as:

[0084] z i = t [h,w] · Z;

[0085] where z i represents the i-th strip convolution feature, t [h,w] represents strip convolution, h and w determine the direction of strip convolution, and h and w represent the horizontal and vertical offsets, respectively.

[0086] Specifically, in this embodiment, the size of the convolution kernel is set to 3*3, and the convolution direction is set to 4, which are 0°, 45°, 90°, and 135° directions.

[0087] S3, a plurality of strip convolution features in different directions are spliced to obtain spliced features.

[0088] Specifically, the splicing is represented as follows:

[0089] y = Cat(z_1, z_2, z_3, z_4);

[0090] Wherein, z_1 to z_4 represent the bar convolution features in different direction convolutions.

[0091] S4, add the spliced feature and the separated feature to obtain a first output feature;

[0092] Specifically, the spliced feature and the separated feature are added, and represented as:

[0093] F=y+Z.

[0094] Wherein, F is the first output feature, y is the spliced feature, and Z is the separated feature.

[0095] S5, performing horizontal and vertical strip convolution on the first output feature to obtain a main branch feature;

[0096] Specifically, in the encoder-decoder structure, the encoder is mainly responsible for feature extraction, which has a crucial influence on the final result. With the increase of network depth, the extracted features tend to pay more attention to local details, and information fusion may cause the loss of key global features. To solve these problems, the following solutions are given: through two different branches, aiming to capture complementary features to improve the performance of road segmentation. The main branch focuses on extracting detailed spatial features from the intermediate output of the encoder. The input corresponds to the output of each block in the encoder. In the main branch, the input feature first passes through a max-pooling layer to reduce the resolution, and then horizontal and vertical strip convolutions t(1x3) and t(3x1) are performed to learn road features, obtaining the feature map of the main branch. At the same time, the auxiliary branch processes the original input image X to provide complementary global information. The image is first down-sampled to match the resolution of the corresponding main branch feature. Then, a feature separation operation is applied to align the channel dimension. Inspired by the residual learning paradigm, an improved residual block is used to extract hierarchical features. Subsequently, the feature maps of the two branches are concatenated in the channel dimension. Finally, the output feature is obtained through a 1x1 convolution layer.

[0097] Performing horizontal and vertical strip convolution on the first output feature to obtain the main branch feature includes:

[0098] Performing max-pooling operation on the first output feature to obtain a pooling feature;

[0099] Performing horizontal and vertical strip convolution on the pooling feature to obtain the main branch feature, and represented as:

[0100] y b =Maxpool(F i )·t 1×n ·t n×1 ;

[0101] Wherein, y bMaxpool is a max-pooling operation, t 1×n represents a horizontal strip convolution, t n×1 represents a vertical strip convolution, n represents a convolution kernel size, F i is a pooled feature.

[0102] S6, down-sampling the road image to obtain a sampling feature, then performing feature separation on the sampling feature to obtain a sampling separation feature, and splicing the sampling separation feature and the main branch feature to obtain a second output feature;

[0103] down-sampling the road image to obtain a sampling feature, then performing feature separation on the sampling feature to obtain a sampling separation feature, and splicing the sampling separation feature and the main branch feature to obtain a second output feature includes:

[0104] down-sampling the road image to obtain a sampling feature, then performing feature separation on the sampling feature to obtain a sampling separation feature, and is represented as:

[0105] y res = DS a (X)·k n ·k n + DS(X);

[0106] wherein y res is a separation feature, DS a (X) represents down-sampling, a represents a down-sampling multiple, X is a road image, DS a (X)·k n ·k n represents feature separation, n represents a convolution kernel size, DS(X) represents a down-sampling multiple of 1, for example, k3 represents a convolution kernel with a size of 3x3. Since the additional branch directly down-samples the original input image and integrates information into the corresponding feature extraction stage, the network can learn more features.

[0107] splicing the sampling separation feature and the main branch feature to obtain a second output feature, and is represented as:

[0108] F i+1 = Cat(y res ·y b )·k1;

[0109] wherein Cat represents splicing, and k1 represents using a convolution kernel with a size of 1 for feature classification.

[0110] S7, performing n times of step-by-step downsampling on the second output feature to obtain a third feature, the step-by-step sampling being that a sampling result of a last time of sampling is taken as an input for a next time of sampling, a sampling and extraction feature is obtained by performing feature extraction on a last time of sampling result, the sampling and extraction feature is up-sampled and spliced with a last time of sampling result to obtain a sampling splicing feature, the sampling splicing feature is subjected to a channel weighting operation to obtain a weighted splicing feature, the n-1th weighted splicing feature is spliced with the second output feature to obtain a final splicing feature, and the final splicing feature is subjected to a channel weighting operation and then multi-scale feature extraction to obtain a final output feature;

[0111] Specifically, in the embodiment, the downsampling operation is performed 4 times, as shown in the following table. Figure 2 As shown in the table, the second output feature is first subjected to downsampling to obtain a third feature after sampling, the third feature is composed of 4 sampling results, which are described as 1, 2, 3 and 4 respectively, 1 is subjected to downsampling to obtain 2, 2 is subjected to downsampling to obtain 3, 3 is subjected to downsampling to obtain 4, 4 is subjected to feature extraction to obtain a sampling and extraction feature, then the sampling and extraction feature is up-sampled and spliced with 3 to obtain a sampling splicing feature (5 in the figure), 5 is subjected to a channel weighting operation to obtain 6 in the figure, the above operation is repeated to obtain 7, 8, 9 and 10, then 10 is up-sampled and spliced with the second output feature to obtain 11, 11 is subjected to a channel weighting operation to obtain 12, that is, a final splicing feature, then the final splicing feature is subjected to a channel weighting operation and then multi-scale feature extraction to obtain a final output feature. It is worth noting that the size change of the downsampling operation and the up-sampling operation is the same, for example, 3 is subjected to downsampling to obtain 4, and the size of 4 after being subjected to feature extraction and then up-sampling is the same as that of 3.

[0112] The channel weighting operation includes:

[0113] The sampling splicing feature is split into h groups along the channel dimension, the first h-1 groups are multiplied by the main element of the learning parameter to obtain h-1 global semantic features, the hth group is subjected to depth separable convolution to obtain a depth convolution feature, the h-1 global semantic features and the depth convolution feature are spliced to obtain a reorganization feature, and the reorganization feature is subjected to bar convolution in different directions and then spliced to obtain a weighted splicing feature.

[0114] Specifically, the decoder recovers the original resolution of the spatial features by progressively upsampling them using the semantic features extracted by the encoder. However, remote sensing images are often affected by occlusion problems, with roadside trees, vehicles, and tall buildings blocking the road surface, leading to fragmentation of the road topology and inadequate feature representation. Traditional square convolution is difficult to effectively solve these occlusions. Considering the elongation and directionality of roads, strip convolution is proven to be more suitable for capturing their structural patterns. At the same time, attention mechanisms are widely adopted due to their ability to enhance feature discrimination. These mechanisms enable the network to focus on semantic key areas by adaptively weighting the feature maps, improving local feature extraction capabilities.

[0115] The second output feature is split into h groups along the channel dimension, and the first h-1 groups are multiplied by the main element of the learning parameter to obtain h-1 global semantic features, denoted as:

[0116] Obtain the learning parameter;

[0117] The second output feature is split into h groups along the channel dimension, and the first h-1 groups are multiplied by the main element of the learning parameter to obtain j-1 global semantic features, denoted as:

[0118] s j =e j ⊙p j (1≤j≤h-1);

[0119] where e j is the jth group of the split second output feature, p j is the learning parameter, and ⊙ represents the Hadamard product.

[0120] The h-1 global semantic features and the deep convolutional features are concatenated to obtain the reorganized feature denoted as:

[0121] L i =Cat(SE(s j ));

[0122] where L i represents the reorganized feature, Cat represents concatenation, SE represents channel attention operation, s j represents global semantic features or deep convolutional features, s j represents global semantic features when j takes values [1, h-1], and s j represents deep convolutional features when j takes value h.

[0123] The channel attention operation is denoted as:

[0124] Y=X·Sigmoid(LN(LN(Gp(X))));

[0125] Wherein, X is a road image, LN represents a full connection layer, Gp is a global average pooling, Sigmoid is an activation function, and Y is an output.

[0126] Specifically, the final spliced feature is represented as:

[0127] E i+1 = Cat(t [h,w] ·L i )(h,w∈0,1);

[0128] Wherein, Cat represents splicing, t [h,w] represents performing convolution kernel separation in different directions, L i represents reorganizing features.

[0129] In the multi-branch architecture, the feature map is grouped and adaptively weighted in the channel, so that the network can give priority to the road area and enhance the feature representation. The subsequent occlusion discrimination module dynamically adjusts the receptive field through a variable-size strip convolution kernel. This design allows context learning from peripheral pixels for connectivity inference, while the inclined strip convolution as shown extends spatial perception beyond the regular horizontal / vertical direction by capturing diagonal connection patterns. Through the concatenation and hierarchical fusion of these multi-directional features, the system effectively corrects the occluded road prediction.

[0130] In the MOD module (the module corresponding to the channel weighting operation), the SE module is weighted through the attention mechanism, and the judgment is made through 4 strip convolutions in different directions. The strip convolutions are superimposed together, and the middle pixel is the pixel to be judged. The part that is not superimposed together is to judge the relationship between the middle pixel and the surrounding pixels, so as to realize the identification of the occluded road image.

[0131] Experimental results:

[0132] 1. Compared with other methods, good results are obtained on widely used public datasets. The experimental results on the DeepGlobe dataset are as shown in the following table. Figure 3

[0133] 1. Experimental results on the DeepGlobe dataset

[0134] Compared with the current advanced method, the model proposed in the patent achieves Sota performance. The quantitative experimental results are shown in the table in the following. Figure 2 F1 is 78.69%, and IoU is 64.73%, which is +1.24% and +1.66% higher than the best model MSMDFF-Net before. These indicators reflect the balanced precision-recall characteristics and spatial overlap accuracy, respectively, which together verify the effectiveness of the innovation of our architecture.

[0135] ​Qualitative analysis: Visualized experimental results are shown in Figure 4 The occlusions in this dataset mainly come from vegetation. This dense vegetation often leads to many models failing to segment the road. However, the visualized results show that our proposed model successfully segments the road with high precision. Even in areas where the segmentation performance is generally poor, our model extracts the road to the greatest extent, resulting in more accurate and clearer segmentation results. For example, in the images in the 2nd and 4th rows, the dense vegetation covers almost all parts of the road, but among all the models, the model proposed in this paper still achieves the best segmentation result compared to other models. In addition, buildings similar in color to the road often cause models to be mistaken, identifying areas that should not be roads as roads. For example, in the third row, it can be seen from the displayed image that the color of the road is very similar to that of the building, and almost all models produce false segmentation on this image. MSMDFF-Net is one of the areas of the building that is mistaken for a road. In contrast, the network proposed in this paper produces the smallest false segmentation among all models. In addition, our model can also obtain clearer and more accurate results in the correctly segmented areas. These experimental results show that the method proposed in this paper can extract more comprehensive road information. In addition, the connectivity discrimination module enhances the road connectivity and reduces the road fragmentation, further improving the segmentation quality.

[0136] Experiments on the CHN6 dataset, which is more challenging than traditional benchmarks due to its complex urban scenes, heterogeneous road types, and diverse occlusion patterns. The experimental results are shown in Figure 5 The network proposed in this patent achieves an IoU of 63.584% and an F1 of 76.74%, which is an improvement of +1.84% IoU and +1.41% F1 over the previous state-of-the-art network (MSMDFF-Net). The CHN6 dataset comes from six representative cities in China and focuses mainly on urban scenes. From the images in the dataset, it can be observed that compared to the DeepGlobe dataset, the roads exhibit diverse shapes and are subject to more complex occlusions, especially from dense and tall buildings. This greatly increases the difficulty of road extraction. Experimental results show that even in such complex scenarios, the model proposed in this paper maintains strong performance.

[0137] Qualitative analysis: Figure 6Part of the visualization of the experimental results on the CHN6 dataset is shown. From the visualization results, it can be observed that the roads in the city are complex and varied, and dense urban groups appear in the image. Some high-rise buildings and their shadows block the roads, making it impossible for the model to obtain road information in the blocked area. This poses a great challenge to segmentation. Thanks to the MOD module, a weighted method is used to obtain the attention area of the endpoint, and a strip convolution is used to learn the relationship between the target pixel and the domain pixel. Even in more complex urban environments, the method proposed in this patent can achieve the best performance.

[0138] Ablation experiment:

[0139] To verify the rationality of the parameter settings of the proposed method, experiments are conducted on each part and parameter to ensure that each part contributes to the final result and determine the best combination of parameters.

[0140] The ablation experiment of the proposed module is shown in Figure 7 The experimental results of the convolution kernel in the MFE module (corresponding to the content of the second output feature) are shown in Figure 8 The experimental results of the convolution kernel in the MOD module are shown in Figure 9 Among them, the function of the MFE module is the module composed of the process of processing the road image into the first output feature, the function of the EFE module (step-by-step downsampling) is to process the first output feature into the second output feature, the function of the MOD module is to split the second output feature into four groups along the channel dimension and finally process it to obtain the final output feature, and the comparison of the attention mechanism in the MOD is shown in Figure 10 .

[0141] An occluded road feature extraction method, comprising:

[0142] An acquisition module is configured to acquire a road image, wherein the road image is partially occluded.

[0143] A first convolution module is configured to perform convolution kernel on the road image to obtain a separated feature, and perform multi-directional strip convolution on the separated feature to obtain a plurality of strip convolution features in different directions.

[0144] A first splicing module is configured to splice the plurality of strip convolution features in different directions to obtain a spliced feature.

[0145] An addition module is configured to add the spliced feature and the separated feature to obtain a first output feature.

[0146] A second convolution module is configured to perform horizontal and vertical strip convolution on the first output feature to obtain a main branch feature.

[0147] The second splicing module is configured to down-sample the road image to obtain a sampling feature, perform feature separation on the sampling feature to obtain a sampling separated feature, splice the sampling separated feature and the main branch feature to obtain a second output feature;

[0148] The feature fusion module is configured to perform n times of step-by-step down-sampling on the second output feature to obtain a third feature, perform feature extraction on a last sampling result to obtain a sampling extraction feature, splice the sampling extraction feature after up-sampling and a last sampling result to obtain a sampling splicing feature, perform a channel weighting operation on the sampling splicing feature to obtain a weighted splicing feature, splice the (n-1)th weighted splicing feature and the second output feature to obtain a final splicing feature, perform a channel weighting operation on the final splicing feature, and perform multi-scale feature extraction on the final splicing feature to obtain a final output feature;

[0149] The channel weighting operation includes:

[0150] The sampling splicing feature is split into h groups along a channel dimension, the first h-1 groups are multiplied by a main element of a learning parameter to obtain h-1 global semantic features, the hth group is subjected to a depth separable convolution to obtain a depth convolution feature, the h-1 global semantic features and the depth convolution feature are spliced to obtain a reorganization feature, and the reorganization feature is subjected to bar convolution in different directions and spliced to obtain the weighted splicing feature.

[0151] The embodiment of the application further discloses a terminal device including a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor loads and executes the computer program, and adopts the occluded road feature extraction method.

[0152] The terminal device can be a computer device such as a desktop computer, a notebook computer or a cloud server, and the terminal device includes but is not limited to a processor and a memory, for example, the terminal device can further include an input / output device, a network access device and a bus, etc.

[0153] The processor can be a central processing unit (CPU), of course, according to the actual use, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), ready-to-program gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. can also be used, the general-purpose processor can be a microprocessor or any conventional processor, etc., and the application does not limit this.

[0154] The memory can be an internal storage unit of the terminal device, for example, a hard disk or a memory of the terminal device, or an external storage device of the terminal device, for example, a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD), or a flash card (FC) equipped on the terminal device, or a combination of the internal storage unit and the external storage device of the terminal device. The memory is used to store a computer program and other programs and data required by the terminal device, and can also be used to temporarily store data that has been output or will be output. The present application does not limit this.

[0155] The terminal device stores one of the occluded road feature extraction methods in the above embodiments in the memory of the terminal device, and loads and executes the method on the processor of the terminal device, for convenience.

[0156] The present application also discloses a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, one of the occluded road feature extraction methods in the above embodiments is used.

[0157] The computer program can be stored in a computer readable medium, and the computer program includes computer program code. The computer program code can be in the form of source code, object code, an executable file, or some intermediate form of the above. The computer readable medium includes any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the computer readable medium includes but is not limited to the above components.

[0158] The computer readable storage medium stores one of the occluded road feature extraction methods in the above embodiments in the computer readable storage medium, and loads and executes the method on the processor, for the storage and application of the method.

[0159] Those skilled in the art should understand that the above discussion of any embodiment is only exemplary and is not intended to limit the scope of protection of the present application to these examples; under the idea of the present application, the above embodiments or technical features in different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of one or more embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0160] It is intended that the embodiments of the application herein disclosed meet all the written requirements of the patent statutes and come within the judicial doctrines of equivalents and will not be construed to be limited to the embodiments shown and described and by the keeping within the spirit and scope of the embodiments of the application.

Claims

1. A method for extracting features of an obscured road, characterized in that: include: Acquire a road image, wherein a portion of the road image is obscured; Performing convolution kernel separation on the road image to obtain separation features, and performing multi-directional strip convolution on the separation features to obtain strip convolution features in several different directions; Several strip convolution features in different directions are spliced ​​together to obtain spliced ​​features; Adding the concatenated feature and the separated feature to obtain a first output feature; Performing horizontal and vertical strip convolution on the first output feature to obtain the main branch feature; Downsampling the road image to obtain sampling features, then performing feature separation on the sampling features to obtain sampling separation features, and concatenating the sampling separation features with the main branch features to obtain a second output feature; The second output feature is downsampled n times in a step-by-step manner to obtain a third feature, wherein the step-by-step sampling is to use the sampling result of the previous sampling as input for the next sampling, perform feature extraction on the last sampling result to obtain a sampling extraction feature, upsample the sampling extraction feature and splice it with the previous sampling result to obtain a sampling splicing feature, perform a channel weighting operation on the sampling splicing feature to obtain a weighted splicing feature, splice the n-1th weighted splicing feature and the second output feature to obtain a final splicing feature, perform a channel weighting operation on the final splicing feature and then perform multi-scale feature extraction to obtain a final output feature; The channel weighting operation includes: The sampled splicing features are split into h groups along the channel dimension, the first h-1 groups are multiplied by the main elements of the learning parameters to obtain h-1 global semantic features, the h-th group is subjected to depthwise separable convolution to obtain depthwise convolution features, the h-1 global semantic features and the depthwise convolution features are spliced ​​to obtain recombined features, the recombined features are subjected to strip convolution in different directions and then spliced ​​to obtain weighted splicing features.

2. The method for extracting features of an obscured road according to claim 1, wherein: The convolution kernel separation of the road image is performed to obtain separation features, and the separation features are subjected to multi-directional strip convolution to obtain strip convolution features in several different directions. Set the convolution kernel size; The road image is subjected to convolution kernel separation according to the convolution kernel size to obtain separation features, which are expressed as: Z=k [n×n] ·X; Among them, Z is the separation feature, X is the road image, n×n is the convolution kernel size, and K is the convolution kernel separation operation; The separation feature is subjected to strip convolution in different directions to obtain several strip convolution features in different directions, which are expressed as: z i =t [h,w] ·Z; Among them, z i represents the i-th strip convolution feature, t [h,w] Represents strip convolution, h and w determine the direction of the strip convolution, and h and w represent the horizontal offset and vertical offset respectively.

3. The method for extracting features of an obscured road according to claim 1, wherein: The horizontal and vertical strip convolutions are performed on the first output feature to obtain the main branch features, including: Performing a maximum pooling operation on the first output feature to obtain a pooled feature; The pooled features are convolved horizontally and vertically to obtain the main branch features, which are expressed as: y b =Maxpool(F i )·t 1×n ·t n×1 ; Among them, y b is the main branch feature, Maxpool is the maximum pooling operation, t 1×n represents horizontal strip convolution, tn×1 represents vertical strip convolution, n represents the convolution kernel size, F i is the pooling feature.

4. The method for extracting features of an obscured road according to claim 1, wherein: The road image is downsampled to obtain sampling features, and then the sampling features are subjected to feature separation. The sampling separation feature is obtained, and the sampling separation feature and the main branch feature are spliced ​​to obtain the second output feature including: The road image is downsampled to obtain sampling features, and then the sampling features are feature separated to obtain sampling separation features, which are expressed as: y res =DS a (X)·k n ·k n +DS(X); Among them, y res To separate features, DS a (X) represents downsampling, a represents the downsampling multiple, X is the road image, n represents the convolution kernel size, DS a (X)·k n ·k n Indicates feature separation, DS(X) indicates that the downsampling multiple is 1; The sampling separation feature and the main branch feature are concatenated to obtain the second output feature, which is expressed as: F i+1 =Cat(y res ·y b )·k1; Among them, Cat means splicing, and k1 means using a convolution kernel of size 1 for feature classification.

5. The method for extracting features of an obscured road according to claim 1, wherein: The second output feature is split into h groups along the channel dimension, and the first h-1 groups are multiplied by the main elements of the learning parameters to obtain h-1 global semantic features including: Get learning parameters; The second output feature is split into h groups along the channel dimension, and the first h-1 groups are multiplied by the main elements of the learning parameters to obtain j-1 global semantic features, which are expressed as: s j =e j ⊙p j (1≤j≤h-1); Among them, e j is the second output feature of the jth group of splits, p j is the learning parameter, and ⊙ represents the Hadamard product.

6. The method for extracting features of an obscured road according to claim 1, wherein: The h-1 global semantic features and the deep convolutional features are concatenated to obtain a recombined feature representation: THE i =Cat(SE(s j )); Among them, L i represents the recombinant feature, Cat represents concatenation, SE represents the channel attention operation, s j Represents the global semantic features or the deep convolution features. When j is [1, h-1], s j Represents the global semantic feature. When j is h, s j Represents deep convolutional features.

7. The method for extracting features of an obscured road according to claim 6, wherein: The channel attention operation is expressed as: Y=X·Sigmoid(LN(LN(Gp(X)))); Among them, X is the road image, LN represents the fully connected layer, Gp is ​​the global average pooling, Sigmoid is the activation function, and Y is the output.

8. A method for extracting features of an obscured road, characterized in that: include: An acquisition module, configured to acquire a road image, wherein the road image is partially obscured; A first convolution module is configured to perform convolution on the road image to obtain separation features, and perform multi-directional strip convolution on the separation features to obtain strip convolution features in multiple directions; The first splicing module is used to splice several strip convolution features in different directions to obtain a spliced ​​feature; an adding module, configured to add the splicing feature and the separation feature to obtain a first output feature; A second convolution module is used to perform horizontal and vertical strip convolution on the first output feature to obtain a main branch feature; A second splicing module is configured to downsample the road image to obtain sampling features, then perform feature separation on the sampling features to obtain sampling separation features, and splice the sampling separation features with the main branch features to obtain a second output feature; A feature fusion module is used to perform n-level downsampling on the second output feature to obtain a third feature, wherein the level-by-level sampling is to use the sampling result of the previous sampling as input for the next sampling, perform feature extraction on the last sampling result to obtain a sampling extraction feature, upsample the sampling extraction feature and splice it with the previous sampling result to obtain a sampling splicing feature, perform a channel weighting operation on the sampling splicing feature to obtain a weighted splicing feature, splice the n-1th weighted splicing feature and the second output feature to obtain a final splicing feature, perform a channel weighting operation on the final splicing feature and then perform multi-scale feature extraction to obtain a final output feature; The channel weighting operation includes: The sampled splicing features are split into h groups along the channel dimension, the first h-1 groups are multiplied by the main elements of the learning parameters to obtain h-1 global semantic features, the h-th group is subjected to depthwise separable convolution to obtain depthwise convolution features, the h-1 global semantic features and the depthwise convolution features are spliced ​​to obtain recombined features, the recombined features are subjected to strip convolution in different directions and then spliced ​​to obtain weighted splicing features.

9. A terminal device comprising a memory and a processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, the method according to any one of claims 1 to 7 is adopted.

10. A computer-readable storage medium storing a computer program, wherein: When the computer program is loaded and executed by a processor, the method according to any one of claims 1 to 7 is adopted.

Images