Lane line segmentation method and device based on spatial context, and storage medium

Abstract

The application discloses a lane line segmentation method and device based on spatial context and a storage medium, relates to the technical field of deep learning remote sensing image segmentation, and extracts high-resolution features and channel correlation of an input remote sensing image by using an improved high-resolution feature extraction network HRNet as a backbone network. Secondly, a spatial context attention module is designed to enhance the position and shape features of lane lines. Finally, in terms of data processing, the algorithm combines online enhancement and offline enhancement operations, increases sample diversity, and enhances the generalization ability of the model. The application of the method to lane line segmentation tasks on remote sensing images can significantly improve the segmentation accuracy and classification accuracy of various lane lines, and the segmentation effect is better than that of existing algorithms, which has important significance for remote sensing image information analysis.

Description

Technical Field

[0001] This invention belongs to the field of deep learning remote sensing image segmentation technology, specifically relating to lane line segmentation methods, devices, and storage media based on spatial context. Background Technology

[0002] With the continuous development of technology, in the field of remote sensing image segmentation, common network models (such as U-Net proposed by Olaf Ronneberger in 2015 and PSPNet proposed by Zhao Hengshuang in 2017) first use the backbone network to extract semantic features of the segmented targets on the input image, usually including color and texture; then add methods such as pyramid pooling, skip connections, and conditional random fields to optimize the front-end output and explicitly enhance the target information on the feature map; finally, based on the enhanced feature information, predict the semantic category of each pixel and output the segmented image. Through the above process, it can be seen that the performance of the segmentation network largely depends on the output of the backbone network. At present, commonly used VGG and ResNet can improve the feature extraction capability by stacking the number of convolutional layers, but frequent downsampling will lead to a continuous decrease in the resolution of the input image. The semantic features of small targets will inevitably become blurred or even lost during the convolution process, which is more obvious in remote sensing images. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a lane line segmentation method, device, and storage medium based on spatial context, thereby solving the technical problem that feature information is easily lost during the convolution process of input images in existing technologies.

[0004] The objective of this invention can be achieved through the following technical solution: a lane line segmentation method based on spatial context, the method comprising the following steps:

[0005] A remote sensing image dataset containing multiple lane line categories was created and labeled one by one. Using Python code scripts to process the remote sensing image dataset, all samples were divided into training set, validation set and test set.

[0006] Image enhancement includes online enhancement and offline enhancement. Online enhancement refers to randomly rotating, flipping, scaling, and changing the contrast of the image and its corresponding segmentation labels. Offline enhancement includes aliasing and random erasing operations.

[0007] A segmentation network model is established, including an improved high-resolution feature extraction network (also known as improved HRNet), a target context enhancement module, a spatial context attention module, and an upsampling output module. An enhanced 1536×1536 remote sensing image is input. The improved HRNet comprises four feature extraction stages with a parallel structure containing four convolutional branches. A parallel branch fusion module is introduced for multi-scale feature fusion. An ECA module is added after each extraction stage to monitor the correlation between channel information during feature extraction. Following the improved HRNet is the target context enhancement module, which calculates the correlation between a single pixel and its surrounding pixels. The system relies on relationships to help determine the semantic category of pixels. After the target context enhancement module, there is a spatial context attention module, which contains two parallel convolutional branches: the upper branch is based on the coordinate attention module and the lower branch is based on the stripe pooling module, which respectively enhance the position and shape features of lane lines in the feature map. After the spatial context attention module, there is an upsampling output module, which first merges the output of the spatial context attention module with the output of the improved HRNet, then uses interpolation to upsample the merged feature map to restore the feature map size to 1536×1536, and finally outputs the segmented image after passing through a 1×1 convolution kernel.

[0008] The divided training and validation sets are image-enhanced and then input into the segmentation network model. After training, the model is iterated for several rounds until it converges, that is, the loss function value of the segmentation network tends to stabilize.

[0009] The spatial context attention module is used to segment and classify the test set, and the visualization results of the segmentation network model output are obtained.

[0010] Furthermore, the ratio of the training set, validation set, and test set is 7:2:1.

[0011] Furthermore, the online enhancement includes random rotation from 1° to 360°, random horizontal flipping, random vertical flipping, random scaling from 0.5 to 1.0x, and random contrast variation. Before each training sample is fed into the neural network, any of the above online enhancement methods will be randomly applied.

[0012] The aliasing refers to the mixing of two sample images and their corresponding label images to increase sample complexity and expand the dataset size. Its mathematical description is as follows:

[0013]

[0014]

[0015] In the above formula, the subscripts i and j represent the index of the sample, and x ix j The tensor representing the original two images, y i y j This represents the tensor of the two corresponding labeled images, (x i y i ) and (x j y j ) Randomly selected from the original dataset; λ ranges from 0 to 1, representing the proportion of a single image in the aliasing process; These represent the tensor of the image after aliasing and the tensor of the label, respectively.

[0016] Random erasure involves randomly removing pixels from a rectangular region based on the size of the input image with a certain probability. Then, RGB noise is filled into the rectangular region. The goal is for the trained model to be able to determine the category based on the overall features of the lane lines, even if the lane lines are partially occluded, so as not to affect the classification result.

[0017] Furthermore, the calculation process of the ECA module includes:

[0018] Given an input feature map, the feature map is first subjected to global average pooling to generate a global representation of the features; then, one-dimensional convolution is used to obtain cross-channel interaction information; next, sigmoid non-linear processing is applied to obtain channel feature weights; finally, matrix multiplication is performed with the input feature map to obtain a feature map with enhanced channel information.

[0019] Furthermore, the specific calculation steps of the spatial context attention module are as follows:

[0020] The target context representation is first transformed into two one-dimensional feature vectors through one-dimensional horizontal adaptive pooling and one-dimensional vertical adaptive pooling. Features are then aggregated in both the length and width directions. This approach obtains the dependencies between channels in one spatial direction while preserving positional information in the other. The mathematical transformation is shown below:

[0021]

[0022]

[0023] In the above formula, the subscripts X-Pool and Y-Pool represent horizontal pooling and vertical pooling, respectively, and H and W represent the height and width of the input feature map, respectively; x c (h,i) represents the pixel in the h-th row and i-th column of the c-th channel, x c (j,w) represents the pixel in the j-th row and w-th column of the c-th channel; z c h (h) represents the feature vector of the c-th channel after horizontal pooling, z cw (w) represents the feature vector of the c-th channel after vertical pooling;

[0024] Then, for the two eigenvectors z c h (h), z c w (w) The processing is done in parallel on two branches. The left branch is based on coordinate attention to enhance the position information of the lane lines. The right branch is based on the idea of ​​striped pooling to capture long-distance dependencies in long and narrow regions along the horizontal and vertical dimensions, thereby enhancing the shape information of the lane lines.

[0025] The left branch first concatenates two one-dimensional feature vectors, and then performs convolution, batch normalization, and Swish nonlinearization. The mathematical transformation is shown below:

[0026] z′=δ(BN(Conv[z h ,z w ]))

[0027] in,

[0028]

[0029] δ(x) = x·sigmoid(βx)

[0030] In the above formula, [·] represents the Concat operation along the spatial dimension; Conv(·) represents the convolution operation with a 1×1 kernel; BN(·) represents the batch normalization operation; and mean(x) represents the mean of the batch samples. δ(·) represents the variance of the batch sample; β represents the nonlinearization operation, and the value of β ranges from 1 to 10, which can be finely adjusted according to the actual training situation.

[0031] Next, the feature vector z' is split to obtain a pair of orientation and position-sensitive attention weight vectors; after convolution and sigmoid non-linear processing on the two weight vectors respectively, the position-enhanced output is obtained.

[0032] The right branch then undergoes one-dimensional convolution and upsampling, and then merges at the corresponding positions to obtain an H×W feature map. After that, it undergoes 1×1 convolution and non-linear processing to obtain the output with enhanced stripe features.

[0033] Each position in the output feature map establishes a long-distance relationship with the corresponding position in the input feature map in the row and column. Through repeated aggregation, long-distance dependencies are built throughout the scene.

[0034] Finally, the outputs of the two side roads are combined with the original input feature maps, and global spatial context information is fused to obtain pixel representations of lane line position enhancement and stripe feature enhancement.

[0035] Furthermore, the segmentation network model was trained with 300 iterations before training, with a batch size of 2 training samples per iteration. The optimizer used was SGD, with an initial learning rate of 0.0025 that decayed exponentially during training. The loss function used was CrossEntropy Loss, optimized for multi-class classification problems, to measure the closeness between the network's output and the true values. The mathematical description is as follows:

[0036]

[0037] In the above formula, class represents the lane line category index, K represents the number of lane line categories; weight[class] represents the weight of the lane line of the class 1; p[class] represents the probability of the lane line of the class 2 in the network output; exp(p i ) represents the probability distribution of the network output after Softmax, where i ranges from 1 to K, and represents the summation of all probability values ​​of the network output; Loss(p,class) represents the loss value of the lane line of the class 1. The smaller the loss value, the better the network fits the lane line of that class.

[0038] Furthermore, the performance of the segmentation network model is evaluated using three metrics: Accuracy, mIoU, and Kappa. Accuracy represents the proportion of correctly labeled pixels, and its mathematical description is as follows:

[0039]

[0040] In the above formula, TP, TN, FP, and FN represent the number of pixels of true positives, true negatives, false positives, and false negatives, respectively.

[0041] mIoU represents the average intersection-union ratio, where IoU represents the ratio of the intersection to the union of the predicted and actual values. The IoU for each category is calculated separately, and then the average is taken to obtain the mIoU for the entire test set. The mathematical description of mIoU is as follows:

[0042]

[0043] In the above formula, k represents the number of lane line types, and the denominator k+1 includes the background class; i represents the actual value, j represents the predicted value; p ij This means predicting i as j, p ii p ji They represent TP and FP respectively;

[0044] Kappa is commonly used to check the accuracy of remote sensing image classification. The higher the Kappa coefficient, the higher the classification accuracy of the algorithm. Its mathematical description is as follows:

[0045]

[0046] in,

[0047]

[0048]

[0049] In the above formula, p o p represents the proportion of observational accuracy. c The proportion representing the consistency of chance; x ii The diagonal elements in the confusion matrix output by the model are represented by x. i+ x represents the sum of all elements in the i-th row. +i Let N represent the sum of all elements in the i-th column, and let N represent the sum of all elements.

[0050] Furthermore, experiments were conducted using a predefined test set, and the results showed that the accuracy of lane line segmentation reached 0.988, the mIoU of lane line segmentation reached 0.600, and the Kappa of lane line segmentation reached 0.755.

[0051] Furthermore, an apparatus includes:

[0052] One or more processors;

[0053] Memory, used to store one or more programs;

[0054] When the one or more programs are executed by the one or more processors, the one or more processors implement the spatial context-based lane segmentation method as described above.

[0055] Furthermore, a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the spatial context-based lane segmentation method as described above.

[0056] The beneficial effects of this invention are:

[0057] In its application, this invention uses an improved high-resolution feature extraction network (HRNet) as the backbone to extract high-resolution features and channel correlations from the input remote sensing image. Secondly, a spatial context attention module is designed to enhance the recognition of lane line position and shape features. Finally, in terms of data processing, the proposed algorithm combines online and offline enhancement operations, increasing sample diversity and enhancing the model's generalization ability. Applying the proposed method to lane line segmentation tasks on remote sensing images can significantly improve the segmentation accuracy and classification accuracy of various lane lines, achieving better segmentation results than existing algorithms. This is of great significance for remote sensing image information analysis. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of the method of the present invention;

[0060] Figure 2 This is a schematic diagram of the structure of a lane line segmentation network based on spatial context features in remote sensing images according to the present invention.

[0061] Figure 3 This is a schematic diagram of the improved HRNet structure of the present invention;

[0062] Figure 4 This is a schematic diagram of the spatial context attention module of the present invention;

[0063] Figure 5 This is a flowchart of the calculation process of the ECA module of the present invention;

[0064] Figure 6 This is a schematic diagram illustrating the aliasing effect in an embodiment of the present invention;

[0065] Figure 7 This is a schematic diagram illustrating the effect of random erasure in an embodiment of the present invention;

[0066] Figure 8 This is a schematic diagram illustrating the lane line segmentation effect of a lane line segmentation algorithm based on spatial context features in a remote sensing image according to an embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] like Figure 1-8 As shown, the lane segmentation method based on spatial context includes the following steps:

[0069] Step 1: Creating a dataset

[0070] A remote sensing image dataset containing multiple lane line categories was created and labeled one by one, with the resolution of a single sample set to 1536×1536.

[0071] Step 2: Dataset Partitioning

[0072] The dataset processing script randomly divides all samples into training, validation, and test sets according to a certain ratio. In this embodiment, the algorithm divides the training, validation, and test sets in a 7:2:1 ratio.

[0073] Step 3: Data Augmentation

[0074] The input image is augmented, including both online and offline augmentation, with the aim of preventing overfitting during model training and increasing the complexity of the dataset.

[0075] In this embodiment, the online enhancement strategies employed by the algorithm include random rotation from 1° to 360°, random horizontal flipping, random vertical flipping, random scaling from 0.5 to 1.0 times, and random contrast changes. Before each training sample is fed into the neural network, any of the above online enhancement methods will be randomly applied.

[0076] The offline enhancement strategies employed in this algorithm include aliasing and random erasure, with the actual effects as follows: Figure 6 , Figure 7 As shown. Aliasing refers to combining two sample images and their corresponding label images to increase sample complexity and expand the dataset size. Its mathematical description is as follows:

[0077]

[0078]

[0079] In the above formula, the subscripts i and j represent the index of the sample, and x i x j The tensor representing the original two images, y i y jThis represents the tensor of the two corresponding labeled images, (x i y i ) and (x j y j ) Randomly selected from the original dataset; λ ranges from 0 to 1, representing the proportion of a single image in the aliasing process; These represent the tensor of the image after aliasing and the tensor of the label, respectively.

[0080] Random erasure involves randomly removing pixels from a rectangular area based on the size of the input image with a certain probability, and then filling that area with RGB noise. The goal is for the trained model to be able to determine the category of the lane line based on its overall features, so that even if the lane line is partially occluded, it will not affect the classification result.

[0081] Step 4: Model Design

[0082] Design a convolutional neural network architecture for lane line segmentation in remote sensing imagery, such as Figure 2 As shown, it includes an improved high-resolution feature extraction network HRNet, a target context enhancement module, a spatial context attention module, and an upsampled output module.

[0083] The input is an enhanced remote sensing image of 1536×1536.

[0084] Improved HRNet, such as Figure 3 As shown, the structure is divided into four feature extraction stages from left to right, with an overall parallel structure. It contains four convolutional branches from top to bottom, capable of simultaneously collecting lane line features at different resolutions. A parallel branch fusion module is introduced for multi-scale feature fusion to enhance feature representation at the current resolution. An ECA module is added after each extraction stage to monitor the correlation between channel information during feature extraction. The specific calculation steps of the ECA module are as follows... Figure 5 As shown: Given an input feature map, firstly, global average pooling is applied to the feature map to generate a global representation of the features; then, one-dimensional convolution is used to obtain cross-channel interaction information; next, sigmoid non-linear processing is applied to obtain channel feature weights; finally, matrix multiplication is performed with the input feature map to obtain a feature map with enhanced channel information.

[0085] Following the backbone network is a target context enhancement module, which helps determine the semantic category of a pixel by calculating the dependency relationship between a single pixel and its surrounding pixels.

[0086] Following the target context enhancement, a spatial context attention module is then implemented, such as... Figure 4As shown, the spatial context attention module contains two parallel convolutional branches: the upper branch is based on the coordinate attention module, and the lower branch is based on the stripe pooling module, which respectively enhance the position and shape features of lane lines in the feature map. The specific calculation steps of the spatial context attention module are as follows: The target context representation is first transformed into two one-dimensional feature vectors through one-dimensional horizontal adaptive pooling and one-dimensional vertical adaptive pooling, and the features are aggregated in the length and width directions respectively. This can obtain the dependencies between channels in one spatial direction while preserving the positional information in the other spatial direction. The mathematical transformation is shown below:

[0087]

[0088]

[0089] In the above formula, the subscripts X-Pool and Y-Pool represent horizontal pooling and vertical pooling, respectively, and H and W represent the height and width of the input feature map, respectively; x c (h,i) represents the pixel in the h-th row and i-th column of the c-th channel, x c (j,w) represents the pixel in the j-th row and w-th column of the c-th channel; z c h (h) represents the feature vector of the c-th channel after horizontal pooling, z c w (w) represents the feature vector of the c-th channel after vertical pooling.

[0090] Then, for the two eigenvectors z c h (h), z c w (w) The processing is done in parallel on two branches. The left branch is based on coordinate attention to enhance the positional information of the lane lines; the right branch is based on striped pooling to capture long-distance dependencies within long and narrow regions along both horizontal and vertical dimensions, thereby enhancing the shape information of the lane lines.

[0091] The left branch first concatenates two one-dimensional feature vectors, and then performs convolution, batch normalization, and Swish nonlinearization. The mathematical transformation is shown below:

[0092] z′=δ(BN(Conv[z h ,z w ]))

[0093] in,

[0094]

[0095] δ(x) = x·sigmoid(βx)

[0096] In the above formula, [·] represents the Concat operation along the spatial dimension; Conv(·) represents the convolution operation with a 1×1 kernel; BN(·) represents the batch normalization operation; and mean(x) represents the mean of the batch samples. δ(·) represents the variance of the batch sample; β represents the nonlinearization operation, and the value of β ranges from 1 to 10, which can be finely adjusted according to the actual training situation.

[0097] Next, the feature vector z' is split to obtain a pair of orientation and position-sensitive attention weight vectors; after convolution and sigmoid nonlinear processing of the two weight vectors, the position-enhanced output is obtained.

[0098] The right branch then undergoes one-dimensional convolution and upsampling, followed by merging at the corresponding positions to obtain an H×W feature map. After 1×1 convolution and non-linear processing, the output with enhanced stripe features is obtained.

[0099] In the above process, each position in the output feature map establishes a long-distance relationship with the corresponding position in the row and column of the input feature map. By repeating the above aggregation process, long-distance dependencies can be built throughout the entire scene.

[0100] Finally, the outputs of the two side roads are combined with the original input feature maps, and global spatial context information is fused to obtain pixel representations of lane line position enhancement and stripe feature enhancement.

[0101] Following the spatial context attention module, there is an upsampling output module. First, the output of the spatial context attention module is merged with the output of the backbone network. Then, the merged feature map is upsampled using interpolation to restore the feature map size to 1536×1536. Finally, the segmented image is output after a 1×1 convolution operation.

[0102] Step 5: Model Training

[0103] After the training and validation sets divided in step two are subjected to the data augmentation operation described in step three, they are input into the segmentation network model described in step four for training, and the network weights are updated using the backpropagation strategy.

[0104] Before model training, appropriate hyperparameters are initialized: In this embodiment, the number of iterations is set to 300, the batch size of training samples fed into the network each time is 2, the optimizer is SGD, the initial learning rate is set to 0.0025 and decays exponentially during training, and the loss function is CrossEntropy Loss, optimized for multi-class problems, to measure the closeness between the network output value and the true value. The smaller the loss value, the more convergent the model. Its mathematical description is as follows:

[0105]

[0106] In the above formula, class represents the lane line category index, K represents the number of lane line categories; weight[class] represents the weight of the lane line of the class 1; p[class] represents the probability of the lane line of the class 2 in the network output; exp(p i ) represents the probability distribution of the network output after Softmax, where i ranges from 1 to K, representing the summation of all probability values ​​of the network output; Loss(p,class) represents the loss value of the lane line of class 1. The smaller the loss value, the better the network fits that lane line.

[0107] Training ends when 300 iterations have been completed or the model has converged.

[0108] Step Six: Model Evaluation

[0109] The lane line segmentation model based on spatial context features of remote sensing imagery is used to segment and classify the test set divided in step one, obtaining the visualization results output by the model. (See attached image.) Figure 8 As shown, (a) is the input remote sensing image, and (b) is the output segmented image. The lines in (b) represent the lane lines in (a): 1 represents a double yellow solid line lane line, 2 represents a single white solid line lane line, and 3 represents a single white dashed line lane line.

[0110] In this embodiment, the present invention uses three metrics—Accuracy, mIoU, and Kappa—to evaluate model performance.

[0111] Accuracy represents the proportion of correctly labeled pixels, and its mathematical description is as follows:

[0112]

[0113] In the above formula, TP, TN, FP, and FN represent the number of pixels in true positives, true negatives, false positives, and false negatives, respectively. Experiments using the test set divided in step two show that the accuracy of lane line segmentation reaches 0.988.

[0114] mIoU stands for Mean Intersection over Union (IoU), used to measure segmentation accuracy in segmentation tasks. IoU represents the ratio of the intersection to the union of predicted and actual values. The mIoU for the entire test set is obtained by calculating the IoU for each class separately and then averaging them. The mathematical description of mIoU is as follows:

[0115]

[0116] In the above formula, k represents the number of lane line types, and the denominator k+1 includes the background class; i represents the actual value, j represents the predicted value; p ij This means predicting i as j (i.e., FN), p ii p ji Let TP and FP represent TP and FP respectively. Experiments were conducted using the test set partitioned in step two, and the results showed that the mIoU of lane line segmentation reached 0.600.

[0117] The Kappa coefficient is a metric for measuring classification accuracy, commonly used to check the correctness of remote sensing image classification. A higher Kappa coefficient indicates higher classification accuracy of the algorithm. Its mathematical description is as follows:

[0118]

[0119] in,

[0120]

[0121]

[0122] In the above formula, p o p represents the proportion of observational accuracy. c The proportion representing the consistency of chance; x ii The diagonal elements in the confusion matrix output by the model are represented by x. i+ x represents the sum of all elements in the i-th row. +i Let N represent the sum of all elements in the i-th column, and let N represent the sum of all elements. Experiments were conducted using the test set partitioned in step two, and the results showed that the Kappa for lane line segmentation reached 0.755.

[0123] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0124] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A lane line segmentation method based on spatial context, characterized in that, The method comprises the following steps: A remote sensing image dataset containing multiple lane line categories is made and labeled one by one, and all samples are divided into a training set, a validation set and a test set by a python code script for processing the remote sensing image dataset; The image is enhanced, including online enhancement and offline enhancement, the online enhancement refers to random rotation, flipping, scaling, changing contrast of the image and the corresponding segmentation label, and the offline enhancement includes aliasing and random erasing operation; A segmentation network model is established, including an improved high-resolution feature extraction network, also known as an improved HRNet, a target context enhancement module, a spatial context attention module and an up-sampling output module, and an enhanced remote sensing image of 1536*1536 is input, the improved HRNet includes four feature extraction stages, the overall structure is a parallel structure, containing four convolution branches, a parallel branch fusion module is introduced for multi-scale feature fusion, and an ECA module is added after each extraction stage to pay attention to the correlation between channel information while extracting features; After the improved HRNet, the target context enhancement module is connected, the dependence between a single pixel point and surrounding pixel points is calculated to assist in judging the semantic category of the pixel point; after the target context enhancement module, the spatial context attention module is connected, the spatial context attention module includes two parallel convolution branches: the upper branch is based on the coordinate attention module and the lower branch is based on the stripe pooling module, which respectively strengthens the position and shape features of the lane line in the feature map; After the spatial context attention module, the up-sampling output module is connected, the output of the spatial context attention module is merged with the output of the improved HRNet, and then the interpolated method is used for up-sampling the merged feature map to restore the feature map size to 1536*1536, and finally a convolution kernel of 1*1 is used for convolution to output the segmented image; The training set and the validation set samples are enhanced and then input into the segmentation network model, and the training is iterated for a number of rounds until the segmentation network model converges, and the model converges when the loss function value of the segmentation network model tends to be stable; The test set is segmented and classified by using the spatial context attention module to obtain the visualization result output by the segmentation network model. 2.The spatial context-based lane line segmentation method according to claim 1, wherein, The division ratio of the training set, the validation set and the test set is 7:2:

1. 3.The spatial context-based lane line segmentation method of claim 1, wherein, The online enhancement includes 1°-360° random rotation, random horizontal flipping, random vertical flipping, 0.5-1.0 times random scaling and random contrast change, and before each training sample is fed into the neural network, it will randomly use any online enhancement method; The aliasing refers to mixing two samples and their corresponding label pictures to improve the sample complexity and expand the data set size, and the mathematical description is as follows: In the above formula, the subscripts i and j represent the index of the sample, x i , x j represents the tensor of the original two images, y i , y j represents the tensor of the corresponding two labeled images, (x i , y i ) and (x j , y j ) are randomly selected from the original data set; λ is in the range of 0-1, representing the proportion of single image in the mixing process; respectively represent the tensor of the mixed image, the tensor of the label; Random erasing is to randomly remove pixel points in a rectangular region according to the size of the input image, and then fill the rectangular region with RGB noise, so that the trained model can determine the category of the lane line according to the overall features of the lane line, even if the lane line is partially blocked, the classification result will not be affected. 4.The method of claim 1, wherein, The calculation process of the ECA module comprises: Given an input feature map, first, the feature map is subjected to global average pooling to generate a global representation of the features; then, cross-channel interaction information is obtained using one-dimensional convolution; then, after Sigmoid nonlinear processing, the channel feature weight is obtained; finally, matrix multiplication is performed with the input feature map to obtain a feature map with enhanced channel information. 5.The spatial context-based lane line segmentation method of claim 1, wherein, The specific calculation steps of the spatial context attention module are as follows: The target context representation is first converted into two one-dimensional feature vectors through one-dimensional horizontal adaptive pooling and one-dimensional vertical adaptive pooling, which aggregates features from both the length and width directions. This way, the dependence between channels in one spatial direction can be obtained, while the position information in the other spatial direction is maintained. The mathematical transformation is as follows: In the above formula, the subscripts X-Pool and Y-Pool represent horizontal and vertical pooling respectively, and H and W represent the height and width of the input feature map respectively; x c (h,i) represents the pixel point representation of the hth row and ith column in the cth channel, x c (j,w) represents the pixel point representation of the jth row and wth column in the cth channel; z c h (h) represents the feature vector of the cth channel after horizontal pooling, z c w (w) represents the feature vector of the cth channel after vertical pooling; Then, two feature vectors z c h (h), z c w (w) is processed in two branches in parallel, the left branch based on coordinate attention, which is used to enhance the position information of lane lines; The right branch is based on the idea of stripe pooling, capturing long-range dependencies within long and narrow regions along the horizontal and vertical dimensions, thereby enhancing the shape information of lane lines. The left branch first concatenates the two one-dimensional feature vectors, then performs convolution, batch normalization, and Swish nonlinearization. The mathematical transformation is as follows: z' = δ(BN(Conv[z h , z w ])) where, δ(x) = x sigmoid(βx) In the above formula, [·] represents the Concat operation along the spatial dimension; Conv(·) represents the convolution operation with a 1x1 convolution kernel; BN(·) represents the batch normalization operation, mean(x) represents the mean value of the batch samples, represents the variance of the batch samples; δ(·) represents the nonlinearization operation, and β takes a value in the range of 1-10; Next, the feature vector z' is split to obtain a pair of direction and position sensitive attention weight vectors. After convolution and Sigmoid nonlinear processing of the two weight vectors, the position information enhanced output is obtained. The right branch then merges the outputs of the two branches at the corresponding positions to obtain a HxW feature map, followed by 1x1 convolution and nonlinear processing to obtain the stripe feature enhanced output. Each position in the output feature map establishes a long-range relationship with the row and column in the corresponding position of the input feature map. Through repeated aggregation, long-range dependencies are constructed in the entire scene. Finally, the outputs of the two branches are combined with the original input feature map to fuse the global spatial context information, obtaining the pixel representation of lane line position enhancement and stripe feature enhancement. 6.The method of claim 1, wherein, The segmentation network model is set to iterate for 300 rounds before training, with a batch size of 2 for each training sample fed into the network. The optimizer is SGD, the initial learning rate is set to 0.0025, and it is exponentially decayed during training. The loss function is CrossEntropy Loss, which is optimized for multi-classification problems to measure the closeness between network output values and true values. The mathematical description is as follows: In the above formula, class represents a lane line category index, K represents a lane line category number; weight[class] represents a weight of the class category lane line; p[class] represents a probability of the class category lane line output by the network; exp(p i ) represents a probability distribution after the network output is subjected to Softmax, i takes a value from 1 to K, and represents summing all probability values of the network output; and Loss(p, class) represents a loss value of the class category lane line.

7. The spatial context-based lane line segmentation method of claim 1, wherein, The performance of the segmentation network model is evaluated using Accuracy, mIoU, and Kappa. Accuracy represents the proportion of correctly labeled pixel points, and its mathematical description is as follows: In the above formula, TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively. mIoU represents the average intersection over union (IoU), where IoU represents the ratio of the intersection to the union of the predicted value and the true value. The IoU of each class is calculated, and then the average is taken to obtain the mIoU of the entire test set. The mathematical description of mIoU is as follows: In the above formula, k represents the number of lane line types, the denominator k + 1 accounts for the background class; i represents the true value, j represents the predicted value; p ij represents that i is predicted as j, p ii , p ji respectively represent TP and FP; Kappa is commonly used to check the correctness of remote sensing image classification. The higher the Kappa coefficient, the higher the classification accuracy of the algorithm. The mathematical description of Kappa is as follows: wherein, In the above formula, p o denotes the proportion of observed accuracy, p c denotes the proportion of coincidences by chance; x ii denotes the diagonal elements in the confusion matrix of the model output, x i+ denotes the sum of all elements in the i-th row, x +i denotes the sum of all elements in the i-th column, N denotes the sum of all elements.

8. The spatial context-based lane line segmentation method of claim 7, wherein, The experiment is conducted by using the divided test set, and results show that the Accuracy of lane line segmentation reaches 0.988, the mIoU of lane line segmentation reaches 0.600, and the Kappa of lane line segmentation reaches 0.

755.

9. An apparatus, comprising: Comprising: one or more processors; memory for storing one or more programs; when the one or more programs are executed by the one or more processors, so that the one or more processors implement the lane line segmentation method based on spatial context as claimed in any one of claims 1-8.

10. A storage medium containing computer-executable instructions, wherein: The computer executable instructions when executed by a computer processor are for performing the lane line segmentation method based on spatial context as claimed in any one of claims 1-8.

Images