Street scene semantic segmentation method and device based on deep spatial structure information

Abstract

The present disclosure provides a street scene semantic segmentation method and device based on deep spatial structure information. The method comprises: collecting a plurality of street scene images to form a data set, and preprocessing the street scene images in the data set; inputting the preprocessed street scene images into an encoder of a preset network model for feature extraction and compression; calculating spatial structure information for the extracted features; completing up-sampling using a decoder; outputting a prediction result and optimizing the model. The accuracy of the model can be increased without increasing the amount of calculation.

Description

Technical Field

[0001] This disclosure belongs to the field of artificial intelligence technology, specifically relating to a method and apparatus for semantic segmentation of street scenes based on deep spatial structure information. Background Technology

[0002] As artificial intelligence (AI) technology matures, people's lives are becoming more convenient and diverse. In the automotive field, autonomous driving technology is a key manifestation of AI. Autonomous driving, also known as driverless, computer-controlled, or wheeled mobile robot driving, is a cutting-edge technology that relies on computers and AI to complete safe and efficient driving without human intervention. Vision-based street scene semantic segmentation utilizes computer vision technology to enable vehicles to analyze the current road environment, thereby completing the task of autonomous driving.

[0003] Semantic segmentation is a typical computer vision problem. Its goal is to take raw data as input and transform it into masks with highlighted regions of interest. Each pixel in the image is assigned a category ID based on the object it belongs to. Semantic segmentation can be simply understood as pixel-level image classification. Street scene semantic segmentation applies semantic segmentation techniques to street scenes. Compared to other semantic segmentation tasks, street scenes are characterized by complexity and real-time requirements. In recent years, with the continuous development of deep learning technology, it has achieved remarkable results in semantic segmentation tasks. Deep learning methods for semantic segmentation typically employ end-to-end network structures, mainly consisting of an encoder and a decoder. The encoder extracts or compresses the input features; the decoder restores the compressed features to the original input size. The process involves first compressing the high-dimensional input image to a low-dimensional format, and then restoring the low-dimensional features to the original input size, thus completing pixel-based classification. Initially, scholars proposed Fully Convolutional Neural Networks (FCNs) for semantic segmentation. These networks removed fully connected layers and softmax computation, using only convolutional and pooling layers for feature extraction, followed by deconvolution for upsampling. To obtain feature maps with different resolutions, multi-scale feature fusion was added. Later, cross-scale feature fusion was proposed, fusing features from different encoding layers with features from corresponding decoding layers, thus solving finer-grained semantic segmentation problems. However, these methods all have limitations in handling semantic segmentation of street scenes. First, traditional semantic segmentation does not guarantee real-time performance and speed; second, it does not model the structural relationships of the input.

[0004] Current semantic segmentation models are mostly deep convolutional neural networks, with feature extractors often employing network structures such as VGG16 and Rsenet101. However, semantic segmentation of street scenes in the field of autonomous driving requires models to be real-time and fast. Therefore, while ensuring accuracy, the models should be more streamlined. Lightweight neural networks are an improvement on deep convolutional neural networks. Their main idea is to use multiple two-dimensional convolutions instead of three-dimensional convolutions to reduce the computational cost of parameters. Furthermore, road scenes are complex and diverse. To fully utilize the positional information between images, a spatial structure information module was added to the original semantic segmentation model, thus completing the semantic segmentation task for street scenes. Summary of the Invention

[0005] This disclosure aims to at least solve one of the technical problems existing in the prior art, and to provide a method and apparatus for semantic segmentation of street scenes based on depth spatial structure information.

[0006] One aspect of this disclosure provides a semantic segmentation method for street scenes based on deep spatial structure information, the method comprising: A dataset is formed by collecting several street scene images, and the street scene images in the dataset are preprocessed. The preprocessed street scene image is input into the encoder of a preset network model for feature extraction and compression; Calculate spatial structure information from the extracted features; Upsampling is performed using a decoder; Output the prediction results and optimize the model.

[0007] In some implementations, the preprocessing of the street scene images in the dataset includes: The street scene image is subjected to resizing, flipping, or compression to achieve data augmentation.

[0008] In some implementations, inputting the preprocessed street scene image into the encoder of a preset network model for feature extraction and compression includes: DeepLabv3 was chosen as the baseline model. During feature extraction, the model still adopted the strategy of DeepLabv3, but the ResNet101 in DeepLabv3 was replaced with the lightweight neural network MobileNetV2. The basic unit of MobileNetV2 is a depthwise separable convolution, which is decomposed into two smaller convolution operations: depthwise convolution and pointwise convolution. The depthwise convolution uses a different kernel for each input channel, while the pointwise convolution uses a 1x1 kernel. The specific steps of depthwise separable convolution are as follows: First, depthwise convolution is used to convolve different input channels separately; then, pointwise convolution is used to combine the outputs. Assume the input is The output is The kernel size is The parameters of depthwise separable convolution and ordinary convolution are compared as shown in the following formula (1): (1) Where DSC stands for Depthwise Separable Convolution, and C stands for Ordinary Convolution; MobilenetV2 proposed the linear bottleneck theory, which pointed out that ReLU would corrupt data in low-dimensional space, while having less impact on high-dimensional space. In low-dimensional space, linear activation layers are used instead of ReLU. Finally, MobileNetV2 introduced the concept of inverse residuals. The feature map is compressed by first passing through a 1x1 convolutional layer, then through a 3x3 convolutional layer, and finally through a 1x1 convolutional layer to expand the feature map channels back. The inverse residual expands first and then compresses. Assuming the expansion factor is 6, the process is as shown in the following formula (2): (2).

[0009] In some implementations, the step of calculating spatial structure information from the extracted features includes: Let the extracted features be The spatial structure information module functions as follows: Specifically: right Perform 1×1 convolution followed by ReLU to obtain a new mapping. It describes the spatial information of different feature elements; Each cell (at each i, j position) in the image I represents the semantic information of a region; by establishing spatial connections between different regions, the structural relationships between different parts of the object are modeled. Polar coordinates are chosen to model this structural relationship; the N×N plane is used as the polar coordinate plane, with the origin at the location of the maximum eigenvalue. ,area The polar coordinates can be written as This serves as a label for self-supervision; among which and The calculation formulas are shown in formulas (3) and (4) below: (3) (4) The formula for atan2 is shown in formula (5) below: (5) For each cell and the origin By cascading along the channel dimension and passing through fully connected layers and the ReLU activation function, the predicted polar coordinates are obtained. ; Will After adjusting the channels using a 1×1 convolution, a single-channel output is obtained. ,at this time This represents the overall structure of the target by combining the features; The spatial information structure module completes the modeling of the structural relationship between the overall and local features by minimizing the loss function of formula (6) as follows;

[0010]

[0011] (6) .

[0012] In some implementations, the upsampling using a decoder includes: The extracted features are upsampled using a decoder to restore them to their original size, and each pixel is labeled to facilitate the final result calculation.

[0013] In some implementations, the output prediction results and model optimization include: The decoder module outputs a labeled image of the same size as the original image, with each pixel assigned a predicted label. The predicted labels are then compared to the true labels, and the cross-entropy loss function is minimized. Optimize the model; the total loss of the model is now... .

[0014] Another aspect of this disclosure provides a semantic segmentation device for street scenes based on depth spatial structure information, the device comprising: The acquisition module is used to acquire several street scene images to form a dataset, and to preprocess the street scene images in the dataset; The processing module is used to input the preprocessed street scene image into the encoder of a preset network model for feature extraction and compression; The calculation module is used to calculate the spatial structure information of the extracted features; The sampling module is used to perform upsampling using the decoder; The optimization module is used to output prediction results and optimize the model.

[0015] In some embodiments, the processing module is further configured to: DeepLabv3 was chosen as the baseline model. During feature extraction, the model still adopted the strategy of DeepLabv3, but the ResNet101 in DeepLabv3 was replaced with the lightweight neural network MobileNetV2. The basic unit of MobileNetV2 is a depthwise separable convolution, which is decomposed into two smaller convolution operations: depthwise convolution and pointwise convolution. The depthwise convolution uses a different kernel for each input channel, while the pointwise convolution uses a 1x1 kernel. The specific steps of depthwise separable convolution are as follows: First, depthwise convolution is used to convolve different input channels separately; then, pointwise convolution is used to combine the outputs. Assume the input is The output is The kernel size is The parameters of depthwise separable convolution and ordinary convolution are compared as shown in the following formula (1): (1) Where DSC stands for Depthwise Separable Convolution, and C stands for Ordinary Convolution; MobilenetV2 proposed the linear bottleneck theory, which pointed out that ReLU would corrupt data in low-dimensional space, while having less impact on high-dimensional space. In low-dimensional space, linear activation layers are used instead of ReLU. Finally, MobileNetV2 introduced the concept of inverse residuals. The feature map is compressed by first passing through a 1x1 convolutional layer, then through a 3x3 convolutional layer, and finally through a 1x1 convolutional layer to expand the feature map channels back. The inverse residual expands first and then compresses. Assuming the expansion factor is 6, the process is as shown in the following formula (2): (2).

[0016] In another aspect of this disclosure, an electronic device is provided, comprising: One or more processors; A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method described above.

[0017] In another aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables the implementation of the method described above.

[0018] This disclosure discloses a street scene semantic segmentation method and apparatus based on deep spatial structure information. By analyzing the characteristics of the raw data and preprocessing it before inputting it into the network model, the dataset is augmented to achieve better training results. To ensure real-time efficiency in autonomous driving, the lightweight network model Mobilenetv2 is used as a feature extractor, reducing parameter computation. A spatial information structure module is added, combining global and local information and modeling the spatial structure to learn the connections between regions while extracting features. Simultaneously, cross-entropy loss and spatial structure information loss are minimized to optimize the model. This allows for increased model accuracy without increasing computational cost. Attached Figure Description

[0019] Figure 1 This is a flowchart of a street scene semantic segmentation method based on depth spatial structure information according to an embodiment of the present disclosure; Figure 2 This is a flowchart of a street scene semantic segmentation method based on depth spatial structure information according to another embodiment of the present disclosure; Figure 3 This is a structural diagram of the deeplabv3 model according to another embodiment of this disclosure; Figure 4 This is a structural diagram of MobilenetV2 according to another embodiment of this disclosure; Figure 5 This is a flowchart illustrating the calculation process of a spatial information module according to another embodiment of this disclosure. Figure 6 This is a schematic diagram of a street scene semantic segmentation device based on depth spatial structure information, according to another embodiment of the present disclosure. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this disclosure, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] This disclosure relates to a semantic segmentation method for street scenes based on deep spatial structure information. The method includes: analyzing a street scene dataset and performing operations such as rotation, scaling, and compression on the data before inputting it into a neural network model to enhance the data. The preprocessed data is input into the model, using DeepLabv3 as the baseline model, which mainly consists of two steps: an encoder and a decoder. To ensure the model's real-time performance and efficiency, MobileNetv2 is used instead of ResNet101 as the feature extractor. The extracted features are input into the spatial information structure module and the decoder module, respectively. The spatial information structure module uses polar coordinates to represent spatial location information and models the overall location and the relationships between different regions, learning the relationships between regions and calculating the spatial structure loss L1. The decoder restores the compressed features from the encoder to the original input size and calculates the pixel-based loss L2 using predicted and true values. Finally, the total loss L = L1 + L2 is obtained, and the model is optimized using the backpropagation algorithm.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings.

[0023] like Figure 1 and Figure 2 As shown, a street scene semantic segmentation method S100 based on depth spatial structure information includes: S110. Collect several street scene images to form a dataset, and preprocess the street scene images in the dataset.

[0024] Specifically, in this step, the dataset used is CityScapes. Before inputting the data into the network model, preprocessing is performed, including data augmentation and data expansion. CityScapes is a large-scale dataset focused on urban street scene understanding, providing semantic-level, instance-level, and dense pixel annotations (including flat surfaces, people, vehicles, buildings, objects, nature, sky, and empty spaces) across 8 categories and 30 subcategories. The dataset includes approximately 5000 finely annotated images and 20000 coarsely annotated images. The data was collected over several months from 50 cities, covering different times and weather conditions. Initially stored as video, the dataset manually selected frames based on the following characteristics: numerous dynamic objects, changing scene layouts, and changing backgrounds. We used a program to perform resizing, flipping, and compression operations on the data to increase its diversity.

[0025] S120. Input the preprocessed street scene image into the encoder of the preset network model for feature extraction and compression.

[0026] Specifically, in this step, the preprocessed image is input into the encoder of the model for further feature extraction and compression. The baseline model is deeplabv3, and its model structure diagram is shown below. Figure 3 As shown. During feature extraction, the model still employs the strategy of DeepLabv3, but replaces the ResNet101 in DeepLabv3 with the lightweight neural network MobileNetV2, whose network model is as follows. Figure 4 As shown.

[0027] The basic unit of MobileNetV2 is depthwise separable convolution, which is actually a decomposable convolution operation. It can be broken down into two smaller operations: depthwise convolution and pointwise convolution. Depthwise convolution differs from standard convolution. Standard convolution uses kernels on all input channels, while depthwise convolution uses a different kernel for each input channel; that is, one kernel corresponds to one input channel. Pointwise convolution, on the other hand, is a regular convolution that uses a 1x1 kernel.

[0028] For depthwise separable convolution, firstly, depthwise convolution is used to convolve different input channels separately, and then pointwise convolution is used to combine the outputs. The effect is similar to a standard convolution, but it significantly reduces computation and the number of model parameters. Assume the input is The output is The kernel size is The parameters of depthwise separable convolution and ordinary convolution are compared as shown in the following formula (1): (1) Wherein, DSC stands for Depthwise Separable Convolution, and C stands for Ordinary Convolution. When using a 3×3 convolution kernel, Depthwise Separable Convolution reduces the number of parameters by 9 times compared to standard convolution. At the same time, MobilenetV2 proposed the linear bottleneck theory, which pointed out that ReLU will destroy data in low-dimensional space, while the impact on high-dimensional space is relatively small. Linear activation layers are used to replace ReLU in low-dimensional space. Finally, MobilenetV2 introduced the concept of inverse residuals. The esiduals block first passes through a 1x1 convolutional layer to compress the number of channels in the feature map, then through a 3x3 convolutional layer, and finally through a 1x1 convolutional layer to expand the number of channels in the feature map back. The inverse residual first expands and then compresses. Let the expansion factor be 6, and the process is shown in the following formula (2): (2).

[0029] S130. Calculate the spatial structure information of the extracted features.

[0030] Specifically, in this step, let the extracted features be... The spatial structure information module functions as follows: Its calculation process is as follows: Figure 5 As shown. The purpose of this module is to learn the structural relationships between regions. First, for Perform 1×1 convolution followed by ReLU to obtain a new mapping. It describes the spatial information of different feature elements; Each cell (at each i, j position) in the image I represents the semantic information of a region; by establishing spatial connections between different regions, the structural relationships between different parts of the object are modeled.

[0031] Polar coordinates are chosen to model this structural relationship; the N×N plane is used as the polar coordinate plane, with the origin at the location of the maximum eigenvalue. ,area The polar coordinates can be written as This serves as a label for self-supervision; among which and The calculation formulas are shown in formulas (3) and (4) below: (3) (4) The formula for atan2 is shown in formula (5) below: (5) For each cell and the origin By cascading along the channel dimension and passing through fully connected layers and the ReLU activation function, the predicted polar coordinates are obtained. ; Will After adjusting the channels using a 1×1 convolution, a single-channel output is obtained. ,at this time This represents the overall structure of the target by combining the features; The spatial information structure module completes the modeling of the structural relationship between the overall and local features by minimizing the loss function of formula (6) as follows;

[0032]

[0033] (6) .

[0034] S140. Upsampling is performed using the decoder.

[0035] Specifically, in this step, upsampling is performed using a decoder. The decoder's role is to upsample the extracted features to restore them to their original size and label each pixel to facilitate the final result calculation. This step is accomplished using bilinear interpolation in DeepLabv3.

[0036] S150, Output the prediction results and optimize the model.

[0037] Specifically, in this step, the decoder module in step S140 outputs a labeled image of the same size as the original image, and each pixel in the output image is assigned a predicted label. The predicted labels are then compared with the true labels to minimize the cross-entropy loss function. Optimize the model; the total loss of the model is now... .

[0038] This embodiment of the street scene semantic segmentation method based on deep spatial structure information analyzes the characteristics of the original data and preprocesses the data before inputting it into the network model, thus expanding the dataset and improving training performance. To ensure real-time efficiency in autonomous driving, the lightweight network model MobileNetv2 is used as the feature extractor, reducing parameter computation. A spatial information structure module is added to combine global and local information and model the spatial structure, extracting features while learning the connections between regions. Simultaneously, cross-entropy loss and spatial structure information loss are minimized to optimize the model. This allows for increased model accuracy without increasing computational cost.

[0039] Another aspect of this disclosure, such as Figure 6 As shown, a street scene semantic segmentation device 100 based on depth spatial structure information is provided, which can be applied to the method described above. The device 100 includes: The acquisition module 110 is used to acquire several street scene images to form a dataset, and to preprocess the street scene images in the dataset.

[0040] Processing module 120 is used to input the preprocessed street scene image into the encoder of a preset network model for feature extraction and compression; Calculation module 130 is used to calculate spatial structure information of the extracted features; Sampling module 140 is used to perform upsampling using the decoder; The optimization module 150 is used to output prediction results and optimize the model.

[0041] This embodiment of the street scene semantic segmentation device based on deep spatial structure information analyzes the characteristics of the raw data and preprocesses the data before inputting it into the network model, thus expanding the dataset and improving training performance. To ensure real-time efficiency in autonomous driving, the lightweight network model Mobilenetv2 is used as a feature extractor, reducing parameter computation. A spatial information structure module is added to combine global and local information and model the spatial structure, extracting features while learning the connections between regions. Simultaneously, cross-entropy loss and spatial structure information loss are minimized to optimize the model. This allows for increased model accuracy without increasing computational cost.

[0042] In some embodiments, the processing module 120 is further configured to: DeepLabv3 was chosen as the baseline model. During feature extraction, the model still adopted the strategy of DeepLabv3, but the ResNet101 in DeepLabv3 was replaced with the lightweight neural network MobileNetV2. The basic unit of MobileNetV2 is a depthwise separable convolution, which is decomposed into two smaller convolution operations: depthwise convolution and pointwise convolution. The depthwise convolution uses a different kernel for each input channel, while the pointwise convolution uses a 1x1 kernel. The specific steps of depthwise separable convolution are as follows: First, depthwise convolution is used to convolve different input channels separately; then, pointwise convolution is used to combine the outputs. Assume the input is The output is The kernel size is The parameters of depthwise separable convolution and ordinary convolution are compared as shown in the following formula (1): (1) Where DSC stands for Depthwise Separable Convolution, and C stands for Ordinary Convolution; MobilenetV2 proposed the linear bottleneck theory, which pointed out that ReLU would corrupt data in low-dimensional space, while having less impact on high-dimensional space. In low-dimensional space, linear activation layers are used instead of ReLU. Finally, MobileNetV2 introduced the concept of inverse residuals. The feature map is compressed by first passing through a 1x1 convolutional layer, then through a 3x3 convolutional layer, and finally through a 1x1 convolutional layer to expand the feature map channels back. The inverse residual expands first and then compresses. Assuming the expansion factor is 6, the process is as shown in the following formula (2): (2).

[0043] In another aspect of this disclosure, an electronic device is provided, comprising: One or more processors; A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method described above.

[0044] In another aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables the implementation of the method described above.

[0045] The computer-readable medium may be included in the apparatus, device, or system disclosed herein, or it may exist independently.

[0046] The computer-readable storage medium may be any tangible medium that contains or stores a program, and may be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, optical fibers, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0047] The computer-readable storage medium may also include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code, specific examples of which include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination thereof.

[0048] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A semantic segmentation method for street scenes based on deep spatial structure information, characterized in that, The method includes: A dataset is formed by collecting several street scene images, and the street scene images in the dataset are preprocessed. The preprocessed street scene image is input into the encoder of a preset network model for feature extraction and compression; Calculate spatial structure information from the extracted features; Upsampling is performed using a decoder; Output the prediction results and optimize the model; The calculation of spatial structure information from the extracted features includes: Let the extracted features be The spatial structure information module functions as follows: Specifically: right Perform 1×1 convolution followed by ReLU to obtain a new mapping. It describes the spatial information of different feature elements; Each i, j position in each cell represents the semantic information of a region in image I; by establishing spatial connections between different regions, the structural relationships between different parts of the object are modeled. Polar coordinates are chosen to model this structural relationship; the N×N plane is used as the polar coordinate plane, with the origin at the location of the maximum eigenvalue. ,area The polar coordinates can be written as This serves as a label for self-supervision; among which and The calculation formulas are shown in formulas (3) and (4) below: (3) (4) The formula for atan2 is shown in formula (5) below: （5） For each cell and the origin By cascading along the channel dimension and passing through fully connected layers and the ReLU activation function, the predicted polar coordinates are obtained. ; Will After adjusting the channels using a 1×1 convolution, a single-channel output is obtained. ,at this time This represents the overall structure of the target by combining the features; The spatial information structure module completes the modeling of the structural relationship between the overall and local features by minimizing the loss function of formula (6) as follows; (6) 。 2. The method according to claim 1, characterized in that, The preprocessing of the street scene images in the dataset includes: The street scene image is subjected to resizing, flipping, or compression to achieve data augmentation.

3. The method according to claim 1, characterized in that, The step of inputting the preprocessed street scene image into the encoder of a preset network model for feature extraction and compression includes: DeepLabv3 was chosen as the baseline model. During feature extraction, the model still adopted the strategy of DeepLabv3, but the ResNet101 in DeepLabv3 was replaced with the lightweight neural network MobileNetV2. The basic unit of MobileNetV2 is a depthwise separable convolution, which is decomposed into two smaller convolution operations: depthwise convolution and pointwise convolution. The depthwise convolution uses a different kernel for each input channel, while the pointwise convolution uses a 1x1 kernel. The specific steps of depthwise separable convolution are as follows: First, depthwise convolution is used to convolve different input channels separately; then, pointwise convolution is used to combine the outputs. Assume the input is The output is The kernel size is The parameters of depthwise separable convolution and ordinary convolution are compared as shown in the following formula (1): (1) Where DSC stands for Depthwise Separable Convolution, and C stands for Ordinary Convolution; MobilenetV2 proposed the linear bottleneck theory, which pointed out that ReLU would corrupt data in low-dimensional space, while having less impact on high-dimensional space. In low-dimensional space, linear activation layers are used instead of ReLU. Finally, MobileNetV2 introduced the concept of inverse residuals. The feature map is compressed by a 1x1 convolutional layer, then expanded by a 3x3 convolutional layer, and finally expanded by a 1x1 convolutional layer. The inverse residual expands first and then compresses. Assuming the expansion factor is 6, the process is as shown in the following formula (2): （2）。 4. The method according to any one of claims 1 to 3, characterized in that, The upsampling using the decoder includes: The extracted features are upsampled using a decoder to restore them to their original size, and each pixel is labeled to facilitate the final result calculation.

5. The method according to any one of claims 1 to 3, characterized in that, The output prediction results and model optimization include: The decoder module outputs a labeled image of the same size as the original image, with each pixel assigned a predicted label. The predicted labels are then compared to the true labels, and the cross-entropy loss function is minimized. Optimize the model; the total loss of the model is now... .

6. A semantic segmentation device for street scenes based on depth spatial structure information, characterized in that, The device includes: The acquisition module is used to acquire several street scene images to form a dataset, and to preprocess the street scene images in the dataset; The processing module is used to input the preprocessed street scene image into the encoder of a preset network model for feature extraction and compression; The calculation module is used to calculate the spatial structure information of the extracted features; The sampling module is used to perform upsampling using the decoder; The optimization module is used to output prediction results and optimize the model; The computing module is specifically used for: Let the extracted features be The spatial structure information module functions as follows: Specifically: right Perform 1×1 convolution followed by ReLU to obtain a new mapping. It describes the spatial information of different feature elements; Each i, j position in each cell represents the semantic information of a region in image I; by establishing spatial connections between different regions, the structural relationships between different parts of the object are modeled. Polar coordinates are chosen to model this structural relationship; the N×N plane is used as the polar coordinate plane, with the origin at the location of the maximum eigenvalue. ,area The polar coordinates can be written as This serves as a label for self-supervision; among which and The calculation formulas are shown in formulas (3) and (4) below: (3) (4) The formula for atan2 is shown in formula (5) below: （5） For each cell and the origin By cascading along the channel dimension and passing through fully connected layers and the ReLU activation function, the predicted polar coordinates are obtained. ; Will After adjusting the channels using a 1×1 convolution, a single-channel output is obtained. ,at this time This represents the overall structure of the target by combining the features; The spatial information structure module completes the modeling of the structural relationship between the overall and local features by minimizing the loss function of formula (6) as follows; (6) 。 7. The apparatus according to claim 6, characterized in that, The processing module is further configured to: DeepLabv3 was chosen as the baseline model. During feature extraction, the model still adopted the strategy of DeepLabv3, but the ResNet101 in DeepLabv3 was replaced with the lightweight neural network MobileNetV2. The basic unit of MobileNetV2 is a depthwise separable convolution, which is decomposed into two smaller convolution operations: depthwise convolution and pointwise convolution. The depthwise convolution uses a different kernel for each input channel, while the pointwise convolution uses a 1x1 kernel. The specific steps of depthwise separable convolution are as follows: First, depthwise convolution is used to convolve different input channels separately; then, pointwise convolution is used to combine the outputs. Assume the input is The output is The kernel size is The parameters of depthwise separable convolution and ordinary convolution are compared as shown in the following formula (1): (1) Where DSC stands for Depthwise Separable Convolution, and C stands for Ordinary Convolution; MobilenetV2 proposed the linear bottleneck theory, which pointed out that ReLU would corrupt data in low-dimensional space, while having less impact on high-dimensional space. In low-dimensional space, linear activation layers are used instead of ReLU. Finally, MobileNetV2 introduced the concept of inverse residuals. The feature map is compressed by a 1x1 convolutional layer, then expanded by a 3x3 convolutional layer, and finally expanded by a 1x1 convolutional layer. The inverse residual expands first and then compresses. Assuming the expansion factor is 6, the process is as shown in the following formula (2): （2）。 8. An electronic device, characterized in that, include: One or more processors; A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it can implement the method according to any one of claims 1 to 5.

Images