A parking space and lane line segmentation method fusing dual cross attention mechanisms
By introducing a dual cross-attention mechanism into the parking space and lane line segmentation network, the problems of information redundancy and high computational complexity are solved, achieving more efficient parking space and lane line segmentation and improving recognition ability and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI UNIV OF TECH
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-05
AI Technical Summary
Existing parking space and lane line segmentation methods suffer from severe information redundancy during cross-layer feature fusion, insufficient linear structure recognition capability, and high computational complexity.
A parking space and lane line segmentation network with a dual cross-attention mechanism is adopted. By introducing a dual cross-attention module between the encoder and decoder, a hierarchical cross-attention structure is constructed to reduce redundant information interference, enhance the representation of linear structure features, and reduce computational complexity.
It effectively suppresses background interference, improves the segmentation accuracy and robustness of parking spaces and lane lines, reduces computational complexity, and is suitable for real-time in-vehicle applications.
Smart Images
Figure CN122157209A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of autonomous driving and computer vision technology, and in particular to a parking space and lane line segmentation method that integrates a dual cross-attention mechanism. Background Technology
[0002] With the development of autonomous driving technology, automated parking systems have gradually become an important component of intelligent vehicles, and environmental perception is one of the key aspects of realizing automated parking. Accurate detection and segmentation of parking spaces and lane lines are fundamental to path planning and vehicle control.
[0003] In existing technologies, semantic segmentation methods based on deep learning are commonly used to detect parking spaces and lane lines, such as network models based on encoder-decoder structures. While these methods can achieve pixel-level segmentation, they still have the following problems: (1) The encoder and decoder directly concatenate features through skip connections, lacking an effective feature filtering mechanism. This leads to redundant background information in low-level features being introduced into high-level semantic features, thus affecting the segmentation accuracy. (2) Under complex environmental conditions, such as strong light, shadow or occlusion, the model has a weak ability to identify slender structural targets such as parking lines; (3) Existing attention mechanisms mostly adopt global modeling, which has high computational complexity and is not conducive to real-time vehicle applications.
[0004] Therefore, it is necessary to propose a parking space and lane line segmentation method that can effectively suppress background interference, enhance the expression of linear structural features, and take into account computational efficiency. Summary of the Invention
[0005] The technical problem to be solved by the present invention is that the prior art suffers from serious information redundancy, insufficient linear structure recognition ability and high computational complexity in the cross-layer feature fusion process.
[0006] To achieve the objectives of this invention, the following technical solution is adopted: A parking space and lane line segmentation method incorporating a dual cross-attention mechanism is proposed. This method trains a parking space and lane line segmentation network with this dual cross-attention mechanism to obtain a network with optimal recognition performance. The network is then used to segment parking spaces and lane lines. Specifically, the method includes the following steps: Step 1, collect Zhang provides parking space surround view images in various scenarios, using... Pixel window The extraction operation is performed using pixels as the window movement step size to obtain... Zhang Parking space surround view image, and use annotation tool to annotate the parking space lines and lane lines in the image with different colors to generate corresponding labels; The training set consists of parking space surround view images with color labels; Step 2: Construct a parking space and lane line segmentation network integrating a dual cross-attention mechanism, denoted as the initial network. This initial network includes two loops: the first loop consists of an encoder, a bottleneck layer, and a decoder connected in sequence; the second loop consists of an encoder, a skip connection structure, and a decoder connected in sequence. Dual cross-attention modules are inserted into both the bottleneck layer and the skip connection structure. The input to this initial network is... The feature image of the pixel is output as a single-channel indexed image; Step 3: Input the color-labeled parking space surround view images from the training set into the initial network for training to obtain a parking space and lane line segmentation network with optimal segmentation performance. Step 4: Use the parking space and lane line segmentation network to achieve parking space and lane line segmentation.
[0007] Preferably, the structure of the parking space and lane line segmentation network that integrates the dual cross-attention mechanism described in step 2 is as follows: Step 2.1, Build the encoder The encoder includes the following structure connected in series along the input-output direction: encoding module Encoding module Encoding module Encoding module ; The encoding module The following structure is included, which is connected in series along the encoder input-output direction: a convolutional module consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 64 channels. ,one Activation function, a pooling window size of Max pooling layer with a step size of 2; Encoding module The following structure is included in series along the encoder input-output direction: a convolutional module α2 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 128 channels, a ReLU activation function, and a max pooling layer with a pooling window size of 2×2 and a stride of 2. The encoding module Ω3 includes the following structure connected in series along the encoder input-output direction: a convolutional module α3 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 256 channels, a ReLU activation function, and a max pooling layer with a pooling window size of 2×2 and a stride of 2. The encoding module Ω4 includes the following structure connected in series along the encoder input-output direction: a convolutional module α4 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and a channel count of 512; a ReLU activation function; and a max pooling layer with a pooling window size of 2×2 and a stride of 2. Step 2.2, Build the bottleneck layer The bottleneck layer comprises the following structures connected in series along the input-output direction: A convolutional module consists of two convolutional layers with a kernel size of 3×3, a stride of 1, and 1024 channels. ; A dual cross-attention module; A convolutional kernel size is A transposed convolutional layer with a stride of 2; Step 2.3, Build the decoder The decoder includes the following structure connected in series along the input-output direction: Decoding module Decoding module Decoding module Decoding module ; The decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included along the input-output direction of the decoder: A convolutional module consisting of convolutional layers with a stride of 1 and 512 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 256 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 128 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 64 channels. ,one Activation function, a convolution kernel size of A convolutional layer with 1 channel; Step 2.4, Construct the jump connection structure The skip connection structure includes four skip connection layers, denoted as skip connection layers respectively. Jump connection layer Jump connection layer and jump connection layer Each skip connection layer includes a dual cross attention module; Let the number of channels of the convolution before inserting the double cross-attention module be denoted as... , It equals 64, 128, 256, 512, or 1024; The input to the initial network is the encoding module. The output is the output of decoding module A1.
[0008] Preferably, the connection relationships between the various structures in the initial network are as follows: Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is , ; Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is denoted as , ; Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is , ; Encoding module Medium convolution module The output after the skip connection layer is denoted as The output of the bottleneck layer is denoted as Decoding module The input is , .
[0009] Preferably, the feature map changes as follows during the initial network input to output process: The pixel size of the input feature map is The number of channels is 1; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 64; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 128; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 256; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 512; The feature map pixel size remains unchanged after the bottleneck layer, but the number of channels is first expanded from 512 to 1024, and then reduced from 1024 to 512 after one convolution; after the decoding module... Feature map pixel size upsampled to The number of channels was reduced to 256; after decoding module Feature map pixel size upsampled to The number of channels was reduced to 128; after the decoding module Feature map pixel size upsampled to The number of channels was reduced to 64; after the decoding module Feature map pixel size upsampled to The number of channels is reduced to 1; the final output is a single-channel indexed image, where each pixel value represents a mask identifier of the category to which that location belongs.
[0010] Preferably, the training process described in step 3 is as follows: The color-labeled parking space surround view images in the training set are recorded as samples. These samples are fed into the initial network for training, with the number of training rounds set to [number missing]. The training steps for any one round are as follows: Step 3.1: Feed the samples into the initial network; the unnormalized score of the model is... , Where B is the number of samples input into the network in one operation. CLet H be the number of categories, H be the feature map height, and W be the feature map width; let the segmentation target categories be... c The model input is processed by the Softmax function to obtain pixel-level probabilities. ,in Indicates the first The first one in the picture The pixel belongs to the first The predicted probability value of the class. This represents the unnormalized score of the network output. Indicates the same pixel position Summation over all categories; Then calculate the category. c Number of positive samples N : ,in, Indicates the first The first one in the picture The pixel belongs to the _th Pixels of a class; N Indicates the first The sum of the number of all pixels belonging to class c in the image; Step 3.2: Combine the above data and calculate the network loss. loss The formula for its calculation is: ,in, Indicates the first The first one in the picture The pixel does not belong to the first pixel. The predicted probability value of the class; Step 3.3, complete network loss. loss After calculation, the parking space and lane line segmentation network is updated through backpropagation, and hyperparameter optimization is performed to complete this round. Training of Zhang's parking space surround view image; Step 3.4, perform the following judgment: If not completed After completing one round of training, return to step 3.1 to begin the next round; if completed... After one round of training, the network that has completed the training through the above steps is identified as the network with the best detection performance and is recorded as the parking space and lane line segmentation model.
[0011] Preferably, step 4 is implemented as follows: Step 4.1: Record the parking space surround view image to be segmented as the image to be segmented; perform normalization processing on the image to be segmented and adjust the pixels to... ; Step 4.2: Then, the image to be segmented processed in Step 4.1 is fed into the parking space and lane line segmentation network, and the output is a single-channel indexed image. This single-channel indexed image gives the segmented parking space and lane line, realizing the segmentation of parking space and lane line.
[0012] Preferably, the The function expression is as follows:
[0013] in, express The input of the function, express The output of the function.
[0014] Preferably, the dual cross-attention module comprises two cascaded cross-attention units, each cross-attention unit comprising the following structure: a convolutional kernel with a size of [missing information]. The number of channels is The convolutional layer Z1 has a kernel size of... The number of channels is The convolutional layer Z2 has a kernel size of 1. The number of channels is The convolutional layer Z3; The cross-attention unit provides three paths, assuming the input feature map is... In this process, the first path generates a query vector from the input feature map through a convolutional layer Z1. The second path generates a key vector from the input feature map through a convolutional layer Z2. The third path generates a value vector from the input feature map through a convolutional layer Z3. The formulas for calculating the three vectors are as follows: ;in: , and Let represent the learnable weight matrices, , This represents the transformed channel dimension; subsequently, the query vector will be... and key vector The transpose of the matrix is used to perform a dot product operation to calculate the attention weights in the horizontal direction. Attention weights in the vertical direction Subsequently, the value vector is adjusted based on the attention weights. We perform a weighted summation to obtain the enhanced features: , And enhance the features in the horizontal direction. Enhanced features in the vertical direction The features are fused to obtain the first cross-attention output features: ; The features output by the first cross-attention unit As input to the second cross-attention unit, the above operation is repeated to further calculate the attention weights and obtain the features of the double cross-attention output. .
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention introduces a dual cross-attention mechanism in skip connections to achieve effective screening of low-level features and reduce the interference of redundant information on the segmentation results; (2) By constructing a hierarchical cross-attention structure, it is possible to model local directional dependencies and global contextual information simultaneously, thereby improving feature representation capabilities; (3) By utilizing the characteristic of cross attention to establish dependencies in the horizontal and vertical directions, it can effectively enhance the ability to identify linear structural targets such as parking lines and lane lines. (4) Compared with the global self-attention mechanism, the present invention significantly reduces computational complexity and memory usage while ensuring segmentation performance, making it more suitable for real-time vehicle applications. (5) By optimizing the semantic mismatch problem in the cross-layer feature fusion process, the robustness of the model in complex environments is improved. Attached Figure Description
[0016] Figure 1 This is a flowchart of the parking space line and lane line segmentation method of the present invention.
[0017] Figure 2 This is an overall structural diagram of the parking space and lane line segmentation network constructed by the present invention, which integrates a dual cross-attention mechanism.
[0018] Figure 3 This is a schematic diagram of a single cross-attention unit.
[0019] Figure 4 This is a diagram showing the segmentation effect of the method of the present invention. Detailed Implementation
[0020] Figure 1 This is a flowchart of the parking space line and lane line segmentation method of the present invention. Figure 1 As can be seen, this invention provides a parking space and lane line segmentation method that integrates a dual cross-attention mechanism. The method includes training a parking space and lane line segmentation network that integrates a dual cross-attention mechanism to obtain a parking space and lane line segmentation network with optimal recognition performance, and then using this network to segment parking spaces and lane lines. Specifically, it includes the following steps: Step 1, collect Zhang provides parking space surround view images in various scenarios, using... Pixel window The extraction operation is performed using pixels as the window movement step size to obtain... Zhang Parking space surround view image, and use annotation tool to annotate the parking space lines and lane lines in the image with different colors to generate corresponding labels; The training set consists of 3 parking space surround view images with color labels.
[0021] Step 2: Construct a parking space and lane line segmentation network integrating a dual cross-attention mechanism, denoted as the initial network. This initial network includes two loops: the first loop consists of an encoder, a bottleneck layer, and a decoder connected in sequence; the second loop consists of an encoder, a skip connection structure, and a decoder connected in sequence. Dual cross-attention modules are inserted into both the bottleneck layer and the skip connection structure. The input to this initial network is... The feature image of the pixel is output as a single-channel indexed image; Figure 2 This is an overall structural diagram of the parking space and lane line segmentation network constructed by the present invention, which integrates a dual cross-attention mechanism.
[0022] In this embodiment, the structure of the parking space and lane line segmentation network that integrates the dual cross-attention mechanism described in step 2 is as follows: Step 2.1, Build the encoder The encoder includes the following structure connected in series along the input-output direction: encoding module Encoding module Encoding module Encoding module ; The encoding module The following structure is included, which is connected in series along the encoder input-output direction: a convolutional module consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 64 channels. ,one Activation function, a pooling window size of Max pooling layer with a step size of 2; Encoding module The following structure is connected in series along the encoder input-output direction: a convolutional module α2 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 128 channels; a ReLU activation function; and a max pooling layer with a pooling window size of 2×2 and a stride of 2.
[0023] The encoding module Ω3 includes the following structures connected in series along the encoder input-output direction: a convolutional module α3 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 256 channels; a ReLU activation function; and a max pooling layer with a pooling window size of 2×2 and a stride of 2.
[0024] The encoding module Ω4 includes the following structures connected in series along the encoder input-output direction: a convolutional module α4 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and a channel count of 512; a ReLU activation function; and a max pooling layer with a pooling window size of 2×2 and a stride of 2.
[0025] Step 2.2, Build the bottleneck layer
[0026] The bottleneck layer comprises the following structures connected in series along the input-output direction: A convolutional module consists of two convolutional layers with a kernel size of 3×3, a stride of 1, and 1024 channels. ; A dual cross-attention module; A convolutional kernel size is A transposed convolutional layer with a stride of 2; Step 2.3, Build the decoder The decoder includes the following structure connected in series along the input-output direction: Decoding module Decoding module Decoding module Decoding module ; The decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included along the input-output direction of the decoder: A convolutional module consisting of convolutional layers with a stride of 1 and 512 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 256 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 128 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 64 channels. ,one Activation function, a convolution kernel size of A convolutional layer with 1 channel; Step 2.4, Construct the jump connection structure The skip connection structure includes four skip connection layers, denoted as skip connection layers respectively. Jump connection layer Jump connection layer and jump connection layer Each skip connection layer includes a dual cross attention module; Let the number of channels of the convolution before inserting the double cross-attention module be denoted as... , It could be equal to 64, 128, 256, 512, or 1024.
[0027] The input to the initial network is the encoding module. The output is the output of the decoding module A1.
[0028] In this embodiment, the connection relationships between the various structures in the initial network are as follows: Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is , ; Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is denoted as , ; Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is , ; Encoding module Medium convolution module The output after the skip connection layer is denoted as The output of the bottleneck layer is denoted as Decoding module The input is after , .
[0029] In this embodiment, the feature map changes as follows during the initial network input to output process: The pixel size of the input feature map is The number of channels is 1; after the encoding module Input feature map pixel size downsampled to The number of channels has been expanded to 64; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 128; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 256; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 512; The feature map pixel size remains unchanged after the bottleneck layer, but the number of channels is first expanded from 512 to 1024, and then reduced from 1024 to 512 after one convolution; after the decoding module... Feature map pixel size upsampled to The number of channels was reduced to 256; after decoding module Feature map pixel size upsampled to The number of channels was reduced to 128; after the decoding module Feature map pixel size upsampled to The number of channels was reduced to 64; after the decoding module Feature map pixel size upsampled to The number of channels is reduced to 1; the output is a single-channel indexed image, where each pixel value represents a mask identifier of the category to which that location belongs.
[0030] In this embodiment, the The function expression is as follows:
[0031] in, express The input of the function, express The output of the function.
[0032] In this embodiment, the dual cross-attention module includes two cascaded cross-attention units, each of which includes the following structure: a convolutional kernel with a size of [missing information]. The number of channels is The convolutional layer Z1 has a kernel size of... The number of channels is The convolutional layer Z2 has a kernel size of 1. The number of channels is The convolutional layer Z3; The cross-attention unit provides three paths, assuming the input feature map is... In this process, the first path generates a query vector from the input feature map through a convolutional layer Z1. The second path generates a key vector from the input feature map through a convolutional layer Z2. The third path generates a value vector from the input feature map through a convolutional layer Z3. The formulas for calculating the three vectors are as follows: ;in: , and Let represent the learnable weight matrices, , This represents the transformed channel dimension; subsequently, the query vector will be... and key vector The transpose of the matrix is used to perform a dot product operation to calculate the attention weights in the horizontal direction. Attention weights in the vertical direction Subsequently, the value vector is adjusted based on the attention weights. We perform a weighted summation to obtain the enhanced features: , And enhance the features in the horizontal direction. Enhanced features in the vertical direction The features are fused to obtain the first cross-attention output features: ; The features output by the first cross-attention unit As input to the second cross-attention unit, the above operation is repeated to further calculate the attention weights and obtain the features of the double cross-attention output. .
[0033] Figure 3 This is a schematic diagram of a cross-attention unit.
[0034] Step 3: Input the color-labeled parking space surround view images from the training set into the initial network for training to obtain a parking space and lane line segmentation network with optimal segmentation performance.
[0035] In this embodiment, the training process described in step 3 is as follows: The color-labeled parking space surround view images in the training set are recorded as samples. These samples are fed into the initial network for training, with the number of training rounds set to [number missing]. The training steps for any one round are as follows: Step 3.1: Feed the samples into the initial network; the unnormalized score of the model is... , Where B is the number of samples input into the network in one operation. C Let H be the number of categories, H be the feature map height, and W be the feature map width; let the segmentation target categories be... c The model input is processed by the Softmax function to obtain pixel-level probabilities. ,in Indicates the first The first one in the picture The pixel belongs to the first The predicted probability value, This represents the unnormalized score of the network output. Indicates the same pixel position Summation over all categories; Then calculate the category. c Number of positive samples N : ,in, Indicates the first The first one in the picture The pixel belongs to the _th Pixels of a class; N Indicates the first The sum of the number of all pixels belonging to class c in the image; Step 3.2: Combine the above data and calculate the network loss. loss The formula for its calculation is: ,in, Indicates the first The first one in the picture The pixel does not belong to the first pixel. The predicted probability value of the class; Step 3.3, complete network loss. loss After calculation, the parking space and lane line segmentation network is updated through backpropagation, and hyperparameter optimization is performed to complete this round. Training of Zhang's parking space surround view image; Step 3.4, perform the following judgment: If not completed After completing one round of training, return to step 3.1 to begin the next round; if completed... After one round of training, the network A that has been trained through the above steps is identified as the network with the best detection performance and is denoted as the parking space and lane line segmentation model.
[0036] Step 4: Use the parking space and lane line segmentation network to achieve parking space and lane line segmentation.
[0037] In this embodiment, step 4 is implemented as follows: Step 4.1: Record the parking space surround view image to be segmented as the image to be segmented; perform normalization processing on the image to be segmented and adjust the pixels to... ; Step 4.2: Then, the image to be segmented processed in Step 1 is fed into the parking space and lane line segmentation model, and the output is a single-channel indexed image. This single-channel indexed image gives the segmented parking space and lane line, realizing the segmentation of parking space and lane line.
[0038] Figure 4 This is a diagram showing the segmentation effect of the method of the present invention. The left side is a panoramic view of the parking space, and the right side is a single-channel index image showing the segmented parking space and lane lines.
Claims
1. A parking space and lane line segmentation method integrating a dual cross-attention mechanism, characterized in that, By training a parking space and lane line segmentation network that integrates a dual cross-attention mechanism, a parking space and lane line segmentation network with the best recognition performance is obtained, and this parking space and lane line segmentation network is used to segment parking spaces and lane lines. Specifically, the following steps are included: Step 1, collect Zhang provides parking space surround view images in various scenarios, using... Pixel window The extraction operation is performed using pixels as the window movement step size to obtain... Zhang Parking space surround view image, and use annotation tool to annotate the parking space lines and lane lines in the image with different colors to generate corresponding labels; The training set consists of parking space surround view images with color labels; Step 2: Construct a parking space and lane line segmentation network integrating a dual cross-attention mechanism, denoted as the initial network. This initial network includes two loops: the first loop consists of an encoder, a bottleneck layer, and a decoder connected in sequence; the second loop consists of an encoder, a skip connection structure, and a decoder connected in sequence. Dual cross-attention modules are inserted into both the bottleneck layer and the skip connection structure. The input to this initial network is... The feature image of the pixel is output as a single-channel indexed image; Step 3: Input the color-labeled parking space surround view images from the training set into the initial network for training to obtain a parking space and lane line segmentation network with optimal segmentation performance. Step 4: Use the parking space and lane line segmentation network to achieve parking space and lane line segmentation.
2. The parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 1, characterized in that, The structure of the parking space and lane line segmentation network that integrates the dual cross-attention mechanism described in step 2 is as follows: Step 2.1, Build the encoder The encoder includes the following structure connected in series along the input-output direction: encoding module Encoding module Encoding module Encoding module ; The encoding module The following structure is included, which is connected in series along the encoder input-output direction: a convolutional module consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 64 channels. ,one Activation function, a pooling window size of Max pooling layer with a step size of 2; Encoding module The following structure is included in series along the encoder input-output direction: a convolutional module α2 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 128 channels, a ReLU activation function, and a max pooling layer with a pooling window size of 2×2 and a stride of 2. The encoding module Ω3 includes the following structure connected in series along the encoder input-output direction: a convolutional module α3 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and 256 channels, a ReLU activation function, and a max pooling layer with a pooling window size of 2×2 and a stride of 2. The encoding module Ω4 includes the following structure connected in series along the encoder input-output direction: a convolutional module α4 consisting of two convolutional layers with a kernel size of 3×3, a stride of 1, and a channel count of 512; a ReLU activation function; and a max pooling layer with a pooling window size of 2×2 and a stride of 2. Step 2.2, Build the bottleneck layer The bottleneck layer comprises the following structures connected in series along the input-output direction: A convolutional module consists of two convolutional layers with a kernel size of 3×3, a stride of 1, and 1024 channels. ; A dual cross-attention module; A convolutional kernel size is A transposed convolutional layer with a stride of 2; Step 2.3, Build the decoder The decoder includes the following structure connected in series along the input-output direction: Decoding module Decoding module Decoding module Decoding module ; The decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included along the input-output direction of the decoder: A convolutional module consisting of convolutional layers with a stride of 1 and 512 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 256 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 128 channels. ,one Activation function, a convolution kernel size of A transposed convolutional layer with a stride of 2; Decoding module The following structure, consisting of two concatenated kernels of size 1,000, is included in the encoder input-output direction: A convolutional module consisting of convolutional layers with a stride of 1 and 64 channels. ,one Activation function, a convolution kernel size of A convolutional layer with 1 channel; Step 2.4, Construct the jump connection structure The skip connection structure includes four skip connection layers, denoted as skip connection layers respectively. Jump connection layer Jump connection layer and jump connection layer Each skip connection layer includes a dual cross attention module; Let the number of channels of the convolution before inserting the double cross-attention module be denoted as... , It equals 64, 128, 256, 512, or 1024; The input to the initial network is the encoding module. The output is the output of the decoding module A1.
3. The parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 2, characterized in that, The connection relationships between the various structures in the initial network are as follows: Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is , ; Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is denoted as , ; Encoding module Medium convolution module The output passes through a skip connection layer. The output is denoted as Decoding module The output is denoted as Decoding module The input is , ; Encoding module Medium convolution module The output after the skip connection layer is denoted as The output of the bottleneck layer is denoted as Decoding module The input is , .
4. The parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 3, characterized in that, The feature map changes as follows during the initial network input to output process: The pixel size of the input feature map is The number of channels is 1; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 64; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 128; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 256; after the encoding module Feature map pixel size downsampled to The number of channels has been expanded to 512; The feature map pixel size remains unchanged after the bottleneck layer, but the number of channels is first expanded from 512 to 1024, and then reduced from 1024 to 512 after one convolution; after the decoding module... Feature map pixel size upsampled to The number of channels was reduced to 256; after decoding module Feature map pixel size upsampled to The number of channels was reduced to 128; after the decoding module Feature map pixel size upsampled to The number of channels was reduced to 64; after the decoding module Feature map pixel size upsampled to The number of channels is reduced to 1; the final output is a single-channel indexed image, where each pixel value represents a mask identifier of the category to which that location belongs.
5. The parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 1, characterized in that, The training process described in step 3 is as follows: The color-labeled parking space surround view images in the training set are recorded as samples. These samples are fed into the initial network for training, with the number of training rounds set to [number missing]. The training steps for any one round are as follows: Step 3.1: Feed the samples into the initial network; the unnormalized score of the model is... , Where B is the number of samples input into the network in one operation. C Let H be the number of categories, H be the feature map height, and W be the feature map width; let the segmentation target categories be... c The model input is processed by the Softmax function to obtain pixel-level probabilities. ,in Indicates the first The first one in the picture The pixel belongs to the first The predicted probability value of the class. This represents the unnormalized score of the network output. Indicates the same pixel position Summation over all categories; Then calculate the category. c Number of positive samples N : ,in, Indicates the first The first one in the picture The pixel belongs to the _th Pixels of a class; N Indicates the first The sum of the number of all pixels belonging to class c in the image; Step 3.2: Combine the above data and calculate the network loss. loss The formula for its calculation is: ,in, Indicates the first The first one in the picture The pixel does not belong to the first pixel. The predicted probability value of the class; Step 3.3, complete network loss. loss After calculation, the parking space and lane line segmentation network is updated through backpropagation, and hyperparameter optimization is performed to complete this round. Training of Zhang's parking space surround view image; Step 3.4, perform the following judgment: If not completed After completing one round of training, return to step 3.1 to begin the next round; if completed... After one round of training, the network that has completed the training through the above steps is identified as the network with the best detection performance and is recorded as the parking space and lane line segmentation model.
6. The parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 1, characterized in that, The implementation process of step 4 is as follows: Step 4.1: Record the parking space surround view image to be segmented as the image to be segmented; perform normalization processing on the image to be segmented and adjust the pixels to... ; Step 4.2: Then, the image to be segmented processed in step 4.1 is fed into the parking space and lane line segmentation network, and the output is a single-channel indexed image. This single-channel indexed image gives the segmented parking space and lane line, realizing the segmentation of parking space and lane line.
7. The parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 2, characterized in that, The The function expression is as follows: in, express The input of the function, express The output of the function.
8. A parking space and lane line segmentation method integrating a dual cross-attention mechanism according to claim 2, characterized in that, The dual cross-attention module comprises two cascaded cross-attention units, each of which includes the following structure: a convolutional kernel with a size of [missing information]. The number of channels is The convolutional layer Z1 has a kernel size of... The number of channels is The convolutional layer Z2 has a kernel size of 1. The number of channels is The convolutional layer Z3; The cross-attention unit provides three paths, assuming the input feature map is... In this process, the first path generates a query vector from the input feature map through a convolutional layer Z1. The second path generates a key vector from the input feature map through a convolutional layer Z2. The third path generates a value vector from the input feature map through a convolutional layer Z3. The formulas for calculating the three vectors are as follows: ;in: , and Let represent the learnable weight matrices, , This represents the transformed channel dimension; subsequently, the query vector will be... and key vector The transpose of the matrix is used to perform a dot product operation to calculate the attention weights in the horizontal direction. Attention weights in the vertical direction Subsequently, the value vector is adjusted based on the attention weights. We perform a weighted summation to obtain the enhanced features: , And enhance the features in the horizontal direction. Enhanced features in the vertical direction The features are fused to obtain the first cross-attention output features: ; The features output by the first cross-attention unit As input to the second cross-attention unit, the above operation is repeated to further calculate the attention weights and obtain the features of the double cross-attention output. .