Cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios

Abstract

The application belongs to the technical field of computer vision and edge computing, and relates to a cloud-edge collaborative lightweight semantic segmentation method for an automatic driving scene, which comprises the following steps: inputting a target image containing road scene elements into a semantic segmentation model to output a pixel-level class probability map consistent with the size of the target image, and obtaining a semantic segmentation map through Softmax processing; the construction steps of the semantic segmentation model are as follows: firstly, constructing and initializing a cloud-edge collaborative segmentation system, then generating a candidate encoder of a student model and combining the candidate encoder with a decoder of the student model to obtain a candidate student model, performing short training and evaluation on the cloud, and selecting a target student model in combination with edge device constraints and filtering, and finally deploying the teacher model and the target student model to the cloud for full training and knowledge distillation optimization, and then deploying the target student model to an edge device. The application combines neural architecture search and knowledge distillation, can balance the performance of the edge device and the limitation of the computing resources, is high in accuracy and efficiency, and is suitable for the automatic driving environment perception demand.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and edge computing technology, and relates to a cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios. Background Technology

[0002] Cloud-edge collaborative semantic segmentation technology is a fusion technology of computer vision and edge computing. In autonomous driving scenarios, it plays a core role in pixel-level perception of the road environment. The edge devices on the vehicle need to rely on this technology to complete the semantic parsing of elements such as roads, vehicles, and pedestrians in milliseconds, while the high computing power of the cloud provides support for model optimization. This collaborative mode has become an important technical direction for autonomous driving environmental perception.

[0003] In the development of deep learning-driven semantic segmentation technology, mainstream solutions all adopt an encoder-decoder architecture, and most of them use classic image classification networks such as VGG (Visual Geometric Group Network) and ResNet (Residual Network) series as the encoder backbone to extract deep features. Typical examples include the VGG16-based two-stream frame semantic segmentation method and the ResNet101 (101-layer deep residual network) multi-scale semantic guidance method. These methods have achieved basic road scene semantic segmentation by leveraging the mature feature extraction capabilities of classification networks. However, the design goal of classification networks is only image-level category judgment, which does not match the pixel-level prediction requirements of semantic segmentation. They cannot simultaneously take into account multi-scale context modeling and high-resolution spatial detail restoration. In autonomous driving scenarios, the segmentation effect of fine targets such as lane lines and vehicle boundaries is not good, and there are obvious shortcomings in scene adaptability.

[0004] To address the poor adaptability of manually designed encoder architectures, the industry has gradually combined NAS (Neural Architecture Search) technology with lightweight networks such as MobileNet (Mobile Lightweight Network) to automatically design semantic segmentation encoder structures. This aims to replace manual parameter tuning with automated search, creating encoder architectures that are more suitable for lightweight requirements. However, these solutions are based on a single computational domain for architecture evaluation and constraint, using only proxy constraints or single metrics as optimization targets. They fail to incorporate core constraints such as the actual inference latency and FLOPs (Floating Point Operations) of the target edge device into the entire architecture design and search process, and they also fail to conduct collaborative evaluation of the encoder backbone and decoder. As a result, although the searched architecture meets the lightweight requirements, it cannot be directly adapted to the resource limitations of edge devices, making it difficult to deploy.

[0005] To address the deployment challenges of lightweight encoders, cloud-edge collaborative semantic segmentation solutions have emerged. These solutions employ a collaborative model where model training is completed in the cloud and inference is performed on edge devices. By leveraging the computing advantages of the cloud to optimize the model, it is then migrated to the edge, attempting to balance model accuracy with the resource limitations of edge devices. However, this type of collaboration only focuses on the allocation and acceleration of computing power during the inference stage, without incorporating constraints such as the computing power, memory / bandwidth budget of edge devices into the entire process of model structure design, search, and training. It lacks a closed-loop design centered on the target edge device, making it difficult to achieve a balance between segmentation accuracy and real-time inference efficiency after the model trained and optimized in the cloud is migrated to the edge device. This fails to meet the millisecond-level inference requirements of autonomous driving scenarios.

[0006] In the development of lightweight semantic segmentation through cloud-edge collaboration, in order to further improve the segmentation accuracy of lightweight models at the edge, the industry generally adopts the teacher-student knowledge distillation paradigm. This paradigm guides the training of lightweight student models through the output of high-performance teacher models, attempting to improve model performance without increasing inference costs. However, the knowledge transfer in this type of distillation scheme is only based on pixel-level soft targets or simple features L2 (L2 norm) constraints, and only constrains feature similarity through a single loss function. This results in the student model being unable to effectively learn the high-level semantic expression and boundary characterization capabilities of the teacher model. The effect of distillation optimization is limited, making it difficult to improve the segmentation accuracy of fine targets in autonomous driving scenarios.

[0007] It is necessary to provide a lightweight semantic segmentation method that is cloud-edge collaborative and takes into account both lightweight design and high accuracy for autonomous driving, in order to solve the above problems. Summary of the Invention

[0008] The purpose of this invention is to address the problems existing in the prior art and provide a lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A lightweight semantic segmentation method for cloud-edge collaboration for autonomous driving scenarios is proposed. After acquiring a target image containing road scene components (road, lane lines, vehicles, pedestrians, traffic signs, buildings, curbs, sky, vegetation, etc.), the target image is input into a semantic segmentation model, which outputs a pixel-level category probability map with the same size as the target image. Then, the pixel-level category probability map is processed by Softmax to obtain a semantic segmentation map corresponding to the road scene components.

[0011] The steps for building a semantic segmentation model are as follows:

[0012] (a) Construct and initialize the cloud-edge collaborative segmentation system, including computing environment preparation, teacher model preparation and student model structure initialization;

[0013] The computing environment adopts a cloud-edge collaborative architecture. The cloud consists of high-performance servers (configured with 2×AMD EPYC 7543 (32C / 64T, 2.8 GHz) CPUs, 256 GB of memory, 4×NVIDIA A100 80 GB GPUs, a 1 TB NVMe SSD system disk + a 2 TB NVMe SSD data disk, and 25 GbE networking; the software environment is Ubuntu 22.04, CUDA 12.x, cuDNN 9.x, Python 3.10, PyTorch 2.2.x, and NCCL), capable of training deep neural networks, performing neural architecture search, and performing distillation optimization. Edge devices are resource-constrained embedded inference devices, such as low-power automotive platforms (e.g., equipped with GPUs or AI...). The cloud performs high-computational operations (search, training, and filtering), while the edge device is responsible for real-time inference deployment. The cloud explicitly considers the constraints of the edge device during the search and training phases, including inference latency, floating-point operations (FLOPs), memory / bandwidth budget, etc., so that the final generated model can run directly on the edge device at near real-time speed.

[0014] This method incorporates the real-time performance and computational constraints of the target edge device into the entire process of model structure search, training, and evaluation during the design phase. It also combines latency measurements from real devices for deployability filtering and multi-objective trade-offs. This collaborative optimization approach ensures that the resulting network is aligned with hardware deployment from the initial design stage, making it easier to achieve higher pixel-level accuracy and more stable real-time performance under the same edge budget. Compared to existing technologies, it offers superior overall engineering performance. This method is implemented based on the SCTNet-NAS framework, which is specifically designed for cloud-edge collaborative perception. Through three core technologies—cloud-edge collaborative optimized neural network architecture search, feature-based knowledge distillation, and a dedicated decoder head—it provides a systematic solution for achieving efficient and accurate semantic segmentation for edge devices.

[0015] The student model adopts an encoder-decoder structure. Its design goal is to maintain lightweight while still having the ability to accurately parse semantics and clearly segment boundaries for autonomous driving road scenes. The encoder is used to extract multi-scale semantic features from the target image, and the decoder is used to fuse multi-scale semantic features and perform context fusion and attention-guided upsampling to restore spatial resolution and generate pixel-level class probability maps.

[0016] The student model is composed of publicly available basic neural network components, including convolution, batch normalization, activation, inverse residual and separable convolutional units, multi-head self-attention and feedforward networks, residual / skip connections, pooling (including adaptive average pooling), linear projection, sigmoid, bilinear upsampling, multi-scale pooling (SPP) and conventional feature fusion (CF), etc. All of these components are publicly available in the prior art. This invention conducts a collaborative design for edge deployment in terms of component selection, connection order and training / search constraints to obtain a target student model that meets the trade-off between real-time performance and accuracy.

[0017] (b) Generate a candidate encoder for the student model and combine it with the decoder of the student model to obtain a candidate student model;

[0018] (c) Perform short-round training and evaluation on candidate student models in the cloud, and perform deployability filtering and Pareto selection based on segmentation accuracy (mIoU), inference latency, computational complexity and edge device constraints (Latency, FLOPs, etc.) to obtain the target student model;

[0019] Compared with existing semantic segmentation methods using NAS, this method aligns the real / equivalent latency of the target edge device and implements deployability filtering during the search phase. At the same time, it constructs a fine-grained two-level search space at the block and stage levels, and coordinates the evaluation of the backbone and boundary-oriented refinement decoder. Through multi-objective trade-offs and Pareto selection, it jointly optimizes the three dimensions of "accuracy-latency-computation". The resulting structure can meet the actual deployment constraints from the beginning of the design. Under the same edge budget, it is easier to obtain higher pixel-level accuracy and more stable real-time performance. The overall engineering effect is better than the existing NAS schemes that only use proxy constraints or single index optimization.

[0020] (d) Deploy the teacher model and the target student model in the cloud for full training and knowledge distillation optimization to further improve the segmentation accuracy and boundary quality of the student model without significantly increasing the inference cost;

[0021] Unlike existing distillation schemes that mainly rely on pixel-level soft targets or simple feature L2 constraints, this method uses a high-capacity teacher model and sets up aligned distillation. First, it uses adaptive average pooling and two-level linear projection to align the student's intermediate features to the comparable space. Then, it uses distribution consistency and orientation consistency for joint constraints. Under the same edge budget, it is easier to achieve higher mIoU and more stable real-time performance. The overall effect is better than the traditional teacher-student distillation approach.

[0022] (e) Deploy the target student model that has completed full training and knowledge distillation optimization to edge devices to perform real-time semantic segmentation reasoning on targets such as roads, vehicles, pedestrians, and traffic signs in autonomous driving scenarios, thus obtaining a semantic segmentation model.

[0023] As a preferred technical solution:

[0024] As described above, in the cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios, in step (a), the teacher model adopts a pre-trained Vision Transformer (ViT) class model, which has strong global context modeling capabilities and can extract high-level semantic relationships from the entire target image and output high-quality pixel-level category distribution and intermediate layer features. This teacher model can be obtained through the following public methods: using pre-trained / fine-tuned weights released by the community; initializing with public classification weights and reproducibly fine-tuning on the semantic segmentation dataset used in this invention until a preset verification performance threshold is met; the intermediate features and prediction results of the teacher model will be used as the target representation learned by the target student model in subsequent training.

[0025] As described above, in the cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios, in step (a), the encoder of the student model is divided into sequentially stacked... Stages Each stage contains multiple basic blocks, which are lightweight inverse residual convolutional blocks (MBConv-like structures, which are lightweight and meet the low computational requirements of edge devices, while possessing powerful feature extraction capabilities). The shallow stages maintain high spatial resolution to extract local details (such as lane lines, road edges, pedestrian outlines, vehicle boundary lines, etc.), while the deep stages gradually reduce spatial resolution and increase channel depth to extract scene-level context (such as road areas, relationships between objects, and spatial layout of traffic participants). The encoder outputs a set of hierarchical feature representations stage by stage. ,in, Let i represent the output characteristics of the i-th stage, where i = 1, 2, ..., L.

[0026] As described above, the cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios includes a student model decoder in step (a) consisting of an SCTHead (Standard Convolutional Transformer Head) module and an AU_SCTHead (Attention Upsample SCTHead) module. This allows for the simultaneous maintenance of global understanding and fine boundary recovery capabilities. The decoder employs a multi-scale decoding strategy that combines convolutional feature extraction with self-attention global modeling, enabling the aggregation and alignment of features at different scales and improving semantic expression capabilities in complex scenarios. Simultaneously, a multi-scale feature fusion module can be introduced between the encoder and decoder to efficiently aggregate and model image features at different scales, achieving collaborative expression of global context and local details. The multi-scale feature fusion module can adopt a pyramid structure to enhance adaptability to changes in target size and differences in feature hierarchy.

[0027] The SCTHead module achieves the integration of local structure and global context through a layered design. Its core process is as follows:

[0028] First, a combination of 3×3 convolutions, batch normalization (BN), and ReLU activation is used to focus on extracting low-level spatial details such as road boundaries and pedestrian contours, enhancing the local perception capability of the feature map. Then, a Transformer encoding module based on a self-attention mechanism is connected, using multi-head self-attention units (MHSA) to capture long-distance dependencies within the feature map, and combining this with a feedforward network (FFN) to enhance global context modeling, compensating for the locality defects of convolution operations. To stabilize the training process, residual connections are introduced between the Transformer input and output to prevent gradient degradation caused by deep networks. Finally, another set of 3×3 convolutions, BN, and ReLU is used to spatially compress and adjust the channels of the fused features, outputting refined features as input to the upsampling module. This design balances detail preservation and global correlation, making it suitable for high-precision image segmentation tasks.

[0029] The AU_SCTHead module introduces an attention-guided upsampling mechanism based on the SCTHead module to achieve spatial resolution restoration and critical boundary refinement. Its core process is as follows:

[0030] First, a spatial attention weight map is generated through 1×1 convolution, and then normalized to the [0,1] interval using the Sigmoid function. Next, the weight map is multiplied element-wise with the feature map to enhance the feature responses of key driving safety areas such as vehicle outlines, pedestrian boundaries, and lane lines, while suppressing background noise. Then, bilinear interpolation is used to progressively upsample the weighted features, gradually restoring spatial details. Finally, a second refinement of the amplified features is performed using 3×3 convolution + batch normalization + ReLU activation to eliminate the ambiguity introduced by interpolation and optimize boundary representation. This design achieves spatial adaptive enhancement of features through an attention mechanism, significantly improving the accuracy of segmentation results in key regions. It adaptively adjusts the upsampling weights based on feature importance, highlighting key edges and small targets while restoring spatial resolution, further improving the boundary clarity and overall accuracy of the segmentation results.

[0031] The SCTHead module is used in the early stages of decoding to achieve multi-scale semantic aggregation and global context understanding, while the AU_SCTHead module is used in the later stages of decoding to enhance boundary quality while gradually upsampling. Through the above decoder structure, the student model can still produce clear and stable pixel-level segmentation results on key targets such as road areas, vehicle boundaries, and pedestrians while maintaining lightweight design, making it suitable for real-time inference scenarios deployed at the edge.

[0032] As described above, in the cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios, in step (b), the encoder of the student model is defined by a set of parameters. Changing these parameters generates candidate encoders for the student model. These parameters include the convolutional kernel size. Channel expansion ratio Jump Link Mark ,coefficient ,parameter and stage depth factor , =1,…, , , and Used to control the shape of basic blocks. k is used to control the receptive field scale to balance the ability to capture local details and aggregate global context; e is used to control the amplification of the number of channels in the intermediate layer of the inverted residual structure, which determines the expressive power and computational cost; , =1 indicates that skip connection is enabled, which directly connects or adds the block input features to the block output, enhancing gradient flow and feature preservation. =0 indicates that skipping connections are not enabled; The first for the overall scaling encoder The channel width of all basic blocks in the stage is adjusted to accommodate different computing power budgets and storage bandwidth limitations. , For the encoder's first The number of basic blocks contained in a stage. , The first for the encoder The sequence modeling structure within each stage is deeply expanded to enhance the feature modeling and contextual representation capabilities of that stage. .

[0033] By combining different basic block structures and different and The configuration can generate multiple expressive power-computational cost trade-offs for each stage; the parameter combinations across all stages form the backbone structure of the entire encoder, thereby the set of all feasible structures of the encoder constitutes the search space for neural architecture search.

[0034] The lightweight semantic segmentation method for cloud-edge collaboration for autonomous driving scenarios, as described above, includes the following specific steps (c):

[0035] (c1) Initialize the population;

[0036] Each individual in the population is a single candidate student model;

[0037] Each individual can be represented by a set of discrete structure codes, which indicate the parameters at each stage. The parameters include kernel size k, channel expansion ratio e, jump flag s, and the corresponding coefficients for that stage. With stage depth factor When initializing the population, various different encoder structures are randomly sampled from the search space as the initial population. At the same time, a weight sharing mechanism can be introduced to accelerate the population evaluation process, reduce the time and computational cost in the search process, and improve the efficiency of architecture search.

[0038] (c2) Cloud-based short-wheel training and evaluation;

[0039] Each individual instance is trained briefly in the cloud, with the training objective being a pixel-by-pixel supervised semantic segmentation loss function. (e.g., cross-entropy loss function) reaches its minimum value, and after training, the segmentation accuracy, inference latency, and computational complexity of each individual are evaluated;

[0040] (c3) Edge device constraint filtering;

[0041] Individuals that do not meet the edge device constraints are removed, and the remaining individuals constitute a deployable candidate set;

[0042] This step uses the measured / equivalent latency and FLOPs of the target edge device as the core constraints. Candidate architectures that do not meet the deployment requirements of the edge device are directly eliminated. For example, if the inference latency exceeds the real-time threshold of the edge or the FLOPs exceed the preset computing power budget, the structure is directly discarded, and only the candidate architectures that meet the resource constraints are retained. This allows the candidate structures to be aligned with the hardware from the source, avoiding the loss of accuracy caused by later pruning.

[0043] (c4) Select the individual that has the best trade-off between "segmentation accuracy - inference latency - computational complexity" from the deployable candidate set and use it as the parent individual;

[0044] (c5) Perform crossover and mutation operations on the parent individuals to generate offspring individuals;

[0045] Crossover operation involves swapping the number of encoder stages in two parent individuals. Cross-operations can exchange the stage configurations of two parent architectures at the stage level, using stage-granular exchange to ensure architectural diversity without compromising deployability.

[0046] The mutation operation involves randomly modifying the kernel size of the encoder in each parent individual. Channel expansion ratio Jump Link Mark ,coefficient ,parameter and stage depth factor This allows for the exploration of new combinations; the mutation operation employs bounded mutation, making random modifications only within a preset candidate set of parameters to avoid exceeding the constraints of the edge device after mutation.

[0047] (c6) Population renewal;

[0048] The parent individuals are merged with their offspring individuals to form a new generation of population;

[0049] (c7) Determine whether the number of iterations has reached the preset maximum number of iterations. If so, terminate and select one offspring from the offspring of the last iteration as the target student model; otherwise, return to step (c2). This step uses a neural architecture search algorithm based on an evolutionary strategy. Through multiple generations of iteration, the overall population gradually approaches the optimal solution among segmentation quality, inference latency, and computational complexity. After the evolutionary process converges, a set of final candidate architectures is obtained. These architectures achieve an optimal trade-off between segmentation accuracy, computational complexity, and inference latency, and meet the real-time deployment requirements of the target edge device. Finally, the target student model is selected from this candidate set.

[0050] The lightweight semantic segmentation method for cloud-edge collaboration for autonomous driving scenarios, as described above, has the following specific steps (d):

[0051] (d1) Deploy the teacher model and the target student model in the cloud. The intermediate features of the teacher model are denoted as... The intermediate features of the target student model are denoted as Where n represents the batch size, which is the number of samples processed simultaneously in one forward propagation;

[0052] (d2) Adjustment Space dimensions , so that it is with Space dimensions Consistency allows for direct comparison of the two at pixel-by-pixel positions;

[0053] Spatial size adjustment can be achieved through adaptive average pooling, ensuring that the adjusted features are aligned pixel-by-pixel with the teacher features in the spatial dimension;

[0054] (d3) Perform a linear transformation and introduce a nonlinear activation (preferably the ReLU activation function) to reduce the number of channels from It becomes an intermediate value, then undergoes another linear transformation to change its channel count from the intermediate value to... (Right now (Number of channels), to obtain , and Alignment has been achieved in terms of channel count and semantic subspace;

[0055] The linear transformation and nonlinear activation in this step constitute a differentiable feature alignment operator. Channel mapping is achieved through two-level linear projection. The first linear projection layer projects the number of channels of the student feature to an intermediate dimension and introduces nonlinear activation to enhance expressive power. The second linear projection layer further maps this intermediate representation to the number of channels of the teacher feature, completing the alignment of the channel dimension.

[0056] (d4) Perform full training on the target student model in the cloud, with the training objective being the joint loss function. Reaching the minimum value, The calculation formula is as follows:

[0057] ;

[0058] ;

[0059] In the formula, For pixel-wise supervised semantic segmentation loss function, used to constrain the pixel-level class probability map output by the target student model to be consistent with the real pixel-wise labels; The feature alignment loss in knowledge distillation is used to constrain the intermediate features of the target student model to gradually approximate the intermediate features of the teacher model. The Jensen-Shannon divergence measures the consistency between the distributions of teacher characteristics and student characteristics. Minimizing this divergence can guide the distribution of student characteristics to approximate the distribution of teacher characteristics. , which is a weighting coefficient used to balance the relative importance of the distribution consistency term and the feature similarity term in the overall optimization objective. Its value can be set according to the specific task requirements, for example, it can be a positive real number. Cosine similarity is used to measure the similarity between the intermediate features of the student model and the intermediate features of the teacher model in the high-level semantic space after processing by the feature alignment module. and They respectively represent based on and The pixel-wise category probability distribution obtained by Softmax normalization in the channel dimension is used to characterize the difference between the two at the prediction distribution level. This step introduces the collaborative evaluation of deployability filtering and backbone-decoder-distillation in the search and training stages, and improves the global semantics and boundary discrimination of the lightweight student model with aligned distillation. Under the same edge budget, it is easier to obtain a model that can be directly deployed, has higher accuracy and more stable real-time performance.

[0060] By minimizing This invention does not significantly increase the number of parameters in the target student model. Under the premise of considering inference latency, the target student model gradually acquires the high-level semantic expression capabilities and boundary characterization capabilities of the teacher model. After this stage of training, the target student model is capable of performing real-time semantic segmentation and inference in autonomous driving scenarios on edge devices.

[0061] The cloud-edge collaborative lightweight semantic segmentation method for autonomous driving scenarios described above, after obtaining the semantic segmentation map corresponding to the constituent elements of the road scene, further performs thresholding, morphological filtering, boundary refinement and small region removal processing on the semantic segmentation map to obtain the target region mask and contour for autonomous driving perception and planning.

[0062] Beneficial effects:

[0063] (1) The present invention constructs a cloud-edge collaborative segmentation system and puts the constraints of edge device inference latency, floating-point computation, memory / bandwidth budget, etc. into the entire process of student model design, search and training. When the cloud performs high computational operations, it explicitly considers the edge device constraints. The target student model obtained by deployability filtering and Pareto selection can be directly deployed on the edge device to realize real-time semantic segmentation inference in autonomous driving scenarios.

[0064] (2) The present invention generates candidate encoders for student models within a preset parameter range through an evolutionary algorithm, combines them with decoders to form candidate student models, and obtains the target student model through multiple rounds of iterative evolution. The automated architecture search replaces the traditional manual encoder design method, making the encoder architecture more suitable for semantic segmentation tasks in autonomous driving scenarios and improving the effectiveness of feature extraction.

[0065] (3) The present invention deploys the teacher model and the target student model in the cloud for full training and knowledge distillation optimization. Through feature alignment operation, the intermediate features of the student model and the teacher model are aligned in terms of spatial size, number of channels and semantic subspace. Combined with the joint loss function to constrain model training, the target student model can effectively learn the high-level semantic expression ability and boundary characterization ability of the teacher model, thereby improving the segmentation accuracy without significantly increasing the inference cost.

[0066] (4) The present invention performs post-processing operations such as thresholding and morphological filtering on the semantic segmentation graph to obtain a target region mask and contour suitable for autonomous driving perception and planning, so that the semantic segmentation results can be directly connected to the subsequent decision-making process of autonomous driving, thereby improving the practical application value of the method in autonomous driving scenarios. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall architecture of cloud-edge collaborative semantic segmentation in the SCTNet-NAS framework of this invention;

[0068] Figure 2 This is a schematic diagram of the MobileNetV2 architecture;

[0069] Figure 3 This is a schematic diagram of the multi-branch search space structure of the search module in the neural architecture search of this invention;

[0070] Figure 4 This is a schematic diagram of the feature alignment module structure for knowledge distillation in this invention;

[0071] Figure 5 This is a schematic diagram of the structure of the SCTHead module and the AU_SCTHead module of the present invention;

[0072] Figure 6 This is a comparison chart of the segmentation performance of the model of this invention and mainstream semantic segmentation methods in a typical urban road scene. Detailed Implementation

[0073] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0074] A lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios is proposed, with the following specific steps:

[0075] (1) Obtain a target image containing elements of a road scene;

[0076] The elements that make up a road scene include roads, lane lines, vehicles, pedestrians, traffic signs, sky, vegetation, etc.

[0077] The target images can come from public datasets such as Cityscapes, CVRG-Pano, and CARLA, or from real-time acquisition results from the onboard vision acquisition unit of autonomous vehicles.

[0078] (2) Construct a semantic segmentation model;

[0079] The overall structure of this step is as follows: Figure 1 As shown, the details are as follows:

[0080] (a) Construct and initialize the cloud-edge collaborative segmentation system, including computing environment preparation, teacher model preparation and student model structure initialization;

[0081] The computing environment adopts a cloud-edge collaborative architecture;

[0082] The cloud platform consists of high-performance servers (configured with 2×AMD EPYC 7543 (32C / 64T, 2.8 GHz) CPUs, 256 GB of memory, 4×NVIDIA A100 80 GB GPUs, a 1 TB NVMe SSD system disk + a 2 TB NVMe SSD data disk, 25 GbE networking, and a software environment of Ubuntu 22.04, CUDA 12.x, cuDNN 9.x, Python 3.10, PyTorch 2.2.x, and NCCL), capable of training deep neural networks, performing neural architecture search, and performing distillation optimization.

[0083] The edge device uses an automotive low-power platform equipped with NVIDIA Jetson Orin Nano, which has a GPU AI acceleration unit and is a resource-constrained embedded inference device. This platform has low power consumption and high real-time performance, and is specifically adapted to autonomous driving vehicle deployment scenarios.

[0084] The cloud is responsible for performing computationally intensive operations (neural architecture search, short-round training and evaluation of candidate models, full training and distillation optimization of the target student model), while the edge devices are only responsible for the final real-time semantic segmentation inference deployment. The key innovation is that the cloud explicitly considers the core constraints of the edge devices (inference latency, floating-point computation FLOPs, memory / bandwidth budget) during the architecture search and model training stages, so that the model is aligned with the deployment hardware from the beginning of the design, avoiding the accuracy loss caused by the traditional "design first, then prune" approach, and making it easier to achieve higher pixel-level segmentation accuracy and more stable real-time performance under the same edge computing power budget.

[0085] The teacher model employs a pre-trained Vision Transformer (ViT) class model, possessing strong global context modeling capabilities. It can extract high-level semantic relationships from the entire target image and output high-quality pixel-level category distributions and intermediate layer features, providing a teacher benchmark for subsequent knowledge distillation. This teacher model is obtained through the following public methods: directly reusing pre-trained / fine-tuned weights already released in the community (such as publicly available weights pre-trained based on ImageNet and then fine-tuned using a semantic segmentation dataset); or initializing with publicly available classification weights on a semantic segmentation dataset (Cityscapes dataset: a real-world outdoor city street scene dataset containing 2975 training images, 500 validation images, and 500 test images). Collected from 50 different cities, it provides pixel-level annotations for 19 main semantic categories, covering various urban traffic elements such as roads, buildings, vehicles, and pedestrians; CVRG-Pano dataset: a real-world outdoor panoramic image dataset containing 600 panoramic images, of which 524 are used for training and 76 for testing, annotated with 7 semantic categories (images collected covering busy streets, residential areas, commercial areas, and natural landscapes). Reproducible fine-tuning training is performed on publicly available datasets, using the cross-entropy loss function, iterating until the average intersection-union ratio (mIoU) on the validation set meets a preset performance threshold (e.g., for the Cityscapes dataset, the preset validation set mIoU threshold is 80%).

[0086] The student model employs an encoder-decoder architecture, designed to maintain lightweight design while providing accurate semantic parsing and clear boundary segmentation capabilities for autonomous driving road scenes. The encoder extracts multi-scale semantic features from the target image, while the decoder fuses these features and performs context-based fusion and attention-guided upsampling to restore spatial resolution and generate pixel-level class probability maps. The student model consists of publicly available basic neural network components, including convolution, batch normalization, activation functions, inverse residuals and separable convolutional units, multi-head self-attention and feedforward networks, residual / skip connections, pooling (including adaptive average pooling), linear projection, sigmoid, bilinear upsampling, multi-scale pooling (SPP), and conventional feature fusion (CF).

[0087] The encoder is divided into L stacked stages, where L=5. Each stage contains multiple basic blocks, which are MobileNetV2, and their structure is as follows: Figure 2 As shown; the shallow stage maintains high spatial resolution and extracts local details such as lane lines and road edges; the deep stage reduces spatial resolution and increases channel depth to extract scene-level context such as road regions and relationships between targets; the encoder outputs a set of hierarchical feature representations stage by stage: ,in, This represents the output feature of the i-th stage, where i = 1, 2, ..., L;

[0088] The decoder includes the SCTHead module and the AU_SCTHead module, which can simultaneously maintain global understanding and fine boundary recovery capabilities. The module structure is as follows: Figure 5 As shown; among them, the SCTHead module is used in the early stage of decoding to achieve multi-scale semantic aggregation and global context understanding, and the AU_SCTHead module is used in the later stage of decoding to enhance boundary quality while gradually upsampling;

[0089] The SCTHead module is primarily responsible for the extraction and decoding of deep global semantic features, combined with... Figure 5 As shown in the structure on the right, this module adopts a single data stream input design, only receiving deep features with a scale of H / 8×W / 8 and 2C channels. First, the Transform_inputs module performs preliminary dimensional alignment and preprocessing on the deep features. Then, the features sequentially enter two sets of pre-activated convolutional units: first, they undergo batch normalization (BN), ReLU activation, and regular convolutional layers (Conv) for channel compression and deep extraction of contextual information; then, they undergo batch normalization (BN), ReLU activation, and segmentation convolutional layers (Conv_seg) to complete the final pixel-level classification mapping. The design purpose of this module is to fully utilize the high-level semantic information at the end of the backbone network to generate basic segmentation results with a macroscopic receptive field.

[0090] Unlike the SCTHead module, the AU_SCTHead module focuses on the recovery of multi-scale spatial information and the refinement of local boundary details, combining... Figure 5 As can be seen from the structure on the left, this module adopts a dual-branch input architecture: in addition to receiving deep backbone features with a scale of H / 8×W / 8, a high-resolution feature branch from the shallow layer (From stage2) of the encoder is also introduced. In this module, Transform_inputs plays a core role in cross-stage feature fusion, responsible for concatenating and integrating shallow features rich in spatial details with deep features rich in semantic information. The fused multi-level features are then input to the same two sets of pre-activated convolutional units as SCTHead (i.e., sequentially processed by batch normalization (BN) + rectified linear activation function (ReLU) + regular convolutional layer (Conv) and batch normalization (BN) + rectified linear activation function (ReLU) + segmentation convolutional layer (Conv_seg)) for secondary refinement and mapping. By directly supplementing the shallow features, this module effectively compensates for the edge details lost by the deep network due to downsampling, significantly improving the boundary segmentation accuracy of key targets such as vehicles and pedestrians.

[0091] A multi-scale feature fusion module is also introduced between the encoder and decoder. This module adopts a pyramid structure, which integrates the outputs of each stage of the encoder. Features are scale-aligned and channel-fused to achieve a collaborative expression of global context and local details, enhancing the model's adaptability to targets of different sizes (such as large vehicles and small pedestrians) and differences in feature hierarchy.

[0092] (b) Generate candidate encoders for the student model and combine them with the decoder of the student model to obtain candidate student models. The complete search space structure of the encoder is as follows: Figure 3 As shown;

[0093] The encoder of the student model is defined by a set of parameters. Changing these parameters generates candidate encoders for the student model. The parameters defining the encoder are the kernel size. Channel expansion ratio Jump Link Mark ,coefficient ,parameter and stage depth factor , =1,…, , , and Used to control the shape of basic blocks. It is used to control the receptive field scale to balance the ability to capture local details and aggregate global context; It is used to control the amplification of the number of channels in the intermediate layer of the inverted residual structure, and determines the expressive power and computational cost; , =1 indicates that skip connection is enabled, which directly connects or adds the block input features to the block output, enhancing gradient flow and feature preservation; =0 indicates that skipping connections are not enabled; The first for the overall scaling encoder The channel width of all basic blocks in the stage is adjusted to accommodate different computing power budgets and storage bandwidth limitations. , For the encoder's first The number of basic blocks contained in a stage. , The first for the encoder The sequence modeling structure within each stage is deeply expanded to enhance the feature modeling and contextual representation capabilities of that stage. ;

[0094] All the feasible encoder parameter combinations mentioned above constitute a complete search space for neural architecture search, which ensures the diversity of architectures while avoiding the generation of structures that exceed the carrying capacity of edge devices through parameter constraints.

[0095] (c) Based on the complete search space for neural architecture search, candidate models are screened and optimized through evolutionary algorithms. Candidate student models are pre-trained and their performance evaluated in the cloud. Deployability filtering and Pareto selection are performed based on segmentation accuracy (mIoU), inference latency, computational complexity and edge device constraints (Latency, FLOPs) to finally determine the target student model architecture.

[0096] (c1) Initialize the population;

[0097] A predetermined number of encoder structures are randomly sampled from the search space as the initial population, with a population size of 20. Each individual corresponds to a candidate student model. Each individual is represented by a set of discrete structure codes, and the encoded content fully contains the number of basic blocks for each stage. The size of the convolution kernel in each block Expansion ratio Jump Link Mark And the corresponding channel width scaling factor for this stage. With stage depth factor Parameter configuration; To reduce search costs, a weight sharing mechanism is introduced, that is, different individuals (encoder structures) share some basic parameters (such as batch normalization layer parameters) during short training rounds. The Supernet is pre-trained on the Cityscapes dataset for 240k rounds to accelerate population evaluation efficiency.

[0098] (c2) Cloud-based short-wheel training and evaluation;

[0099] We used 2975 training images from the Cityscapes dataset and 524 training images from the CVRG-Pano dataset as training data. Both datasets provided pixel-level semantic annotations (Cityscapes contains 19 classes, and CVRG-Pano contains 7 classes). We used a pixel-wise supervised semantic segmentation loss function as the training objective. In this stage, the learning rate was set to 0.0004, the batch size was set to 16, and the loss function was the cross-entropy loss function, which was used to constrain the pixel-level class probability map output by the model to be consistent with the real pixel-wise labels. After training, we evaluated the segmentation accuracy (mean intersection-union ratio mIoU), inference latency (measured on the target edge device), and computational complexity (FLOPs) of each individual on the validation set (500 validation images from the Cityscapes dataset and 76 test images from the CVRG-Pano dataset (used as the validation set)). At the same time, we recorded the number of model parameters as an auxiliary evaluation metric.

[0100] (c3) Edge device constraint filtering;

[0101] Individuals that do not meet the edge device constraints are removed, and the remaining individuals constitute a deployable candidate set. This step uses the measured / equivalent latency and FLOPs of the target edge device as the core constraints. Candidate architectures that do not meet the edge device deployment requirements are directly removed. That is, when the inference latency exceeds the real-time threshold of the edge or the FLOPs exceed the preset computing power budget, the structure is directly discarded, and only candidate architectures that meet the resource constraints are retained. This ensures that the candidate structures are aligned with the hardware from the source and avoids the loss of accuracy caused by later pruning.

[0102] (c4) Select the individual that has the best trade-off between “segmentation accuracy, inference latency and computational complexity” from the deployable candidate set and use it as the parent individual; this step adopts Pareto optimal selection and selects 20 individuals at the Pareto front as the parent individuals;

[0103] (c5) Perform crossover and mutation operations on the parent individuals to generate offspring individuals;

[0104] Crossover operation involves swapping the number of encoder stages in two parent individuals. A stage-level granularity exchange is adopted to ensure architectural diversity without compromising deployability, and the probability of cross-operation is set to 0.9.

[0105] The mutation operation involves randomly modifying the kernel size of the encoder in each parent individual. Channel expansion ratio Jump Link Mark ,coefficient ,parameter and stage depth factor This allows for the exploration of new combinations. The mutation operation employs polynomial mutation with a mutation factor of 1.0, and is a bounded mutation. It only performs random modifications within the preset candidate set of parameters to avoid exceeding the constraints of the edge device after mutation.

[0106] (c6) Population renewal;

[0107] The parent individuals are merged with their offspring individuals to form a new generation of population;

[0108] (c7) Determine whether the number of iterations has reached the preset maximum number of iterations (set to 60 generations). If yes, terminate and select one child from the child individuals of the last iteration as the target student model; otherwise, return to step (c2).

[0109] At this point, the architecture of the semantic segmentation model is complete, resulting in a target student model with a fixed network structure that needs to be trained and converged.

[0110] (d) Deploy the teacher model and the target student model in the cloud for full training and knowledge distillation optimization;

[0111] (d1) Deploy the teacher model and the target student model in the cloud. The intermediate features of the teacher model are denoted as... The intermediate features of the target student model are denoted as Where n represents the batch size, i.e., the number of samples processed simultaneously in one forward propagation; hn is the height of the teacher feature map, and wn is the width of the teacher feature map. h′n is the number of channels for the intermediate features in the teacher model; h′n is the height of the student feature map; and w′n is the width of the student feature map. The number of channels for intermediate features in the target student model;

[0112] Simultaneously complete the training data configuration, using labeled images from the Cityscapes dataset and the CVRG-Pano dataset, which are public semantic segmentation datasets. Use the pixel-level semantic annotations corresponding to the images as the ground truth labels to complete the division of the training set and validation set. Initialize the training hyperparameters such as optimizer, learning rate, and maximum number of iterations (using AdamW optimizer, initial learning rate set to 0.0004, weight decay coefficient set to 0.0125, and using a multinomial decay strategy to adjust the learning rate, batchSize set to 16).

[0113] (d2) Adjustment Space dimensions , so that it is with Space dimensions Consistency is achieved, allowing direct comparison between the two at pixel-by-pixel positions; this step uses adaptive average pooling to adjust the spatial size, ensuring that the adjusted features are aligned pixel-by-pixel with the teacher features in the spatial dimension.

[0114] (d3) Perform a linear transformation and introduce the ReLU activation function, so that the number of channels is reduced from... It becomes an intermediate value, then undergoes another linear transformation to change its channel count from the intermediate value to... (Right now (Number of channels), to obtain , and Alignment has been achieved in terms of channel count and semantic subspace;

[0115] In this step, feature transformation and nonlinear activation constitute a differentiable feature alignment operator. Channel mapping is achieved through two consecutive convolutional layers. The first convolutional layer maps the number of channels of the student features to an intermediate dimension and introduces nonlinear activation to enhance expressive power. The second convolutional layer further maps this intermediate representation to the number of channels of the teacher features, completing the alignment of the channel dimensions. Figure 4 As shown;

[0116] (d4) Perform full training on the target student model in the cloud, with the training objective being the joint loss function. Reaching the minimum value, The calculation formula is as follows:

[0117] ;

[0118] ;

[0119] In the formula, For pixel-wise supervised semantic segmentation loss function, used to constrain the pixel-level class probability map output by the target student model to be consistent with the real pixel-wise labels; The feature alignment loss in knowledge distillation is used to constrain the intermediate features of the target student model to gradually approximate the intermediate features of the teacher model. The Jensen-Shannon divergence measures the consistency between the distributions of teacher characteristics and student characteristics. Minimizing this divergence can guide the distribution of student characteristics to approximate the distribution of teacher characteristics. , where is the weighting coefficient (set to 0.5), used to balance the relative importance of the distribution consistency term and the feature similarity term in the overall optimization objective; Cosine similarity is used to measure the similarity between the intermediate features of the student model and the intermediate features of the teacher model in the high-level semantic space after processing by the feature alignment module. and They respectively represent based on and The pixel-wise category probability distribution obtained by Softmax normalization in the channel dimension is used to characterize the difference between the two at the prediction distribution level.

[0120] During training, the backpropagation algorithm is used to iteratively update the weight parameters of the target student model. After each training round, the segmentation accuracy and inference performance of the model are verified on the validation set. Training is terminated when the average intersection-union ratio (mIoU) on the validation set tends to stabilize, the joint loss function converges to the preset threshold, or the preset maximum number of training rounds is reached.

[0121] (e) Deploy the target student model, which has undergone full training and knowledge distillation optimization, to an edge device to obtain the semantic segmentation model;

[0122] Export the trained student model to a common format, including ONNX (Open Neural Network Exchange Format) or TensorRT (NVIDIA Tensor Inference Accelerator Engine) format. During the export process, the network structure, weight parameters, and parameters such as the mean and variance of the BatchNorm layer are fully preserved. If the target edge platform supports the TensorRT acceleration engine, further optimization operations such as structure quantization (e.g., INT8 quantization) and layer fusion (e.g., convolution + BN + activation fusion) are performed after export. While ensuring that the accuracy loss is controllable (mIoU decrease does not exceed 1%), inference efficiency is further improved and memory usage is reduced.

[0123] The converted and optimized model file is transmitted to the edge device (NVIDIA Jetson Orin Nano) via network transmission (such as 25GbE Ethernet) or local interface (such as USB 3.0); the model is loaded using the device's built-in TensorRT and cuDNN AI acceleration libraries. During the loading process, the input tensors are initialized (dimensions match the model input size), the model context, and GPU memory usage, completing the initialization preparation for the inference process; test images are input for inference verification to confirm that the model output is accurate and the inference latency meets real-time requirements;

[0124] (3) Input the target image into the semantic segmentation model, which outputs a pixel-level category probability map with the same size as the target image;

[0125] First, the target image is standardized preprocessed: the target image is scaled to the fixed input size preset by the model (including two specifications: 512×1024 and 1536×768), converted into tensor format (dimensions [C,H,W], where C=3 is the number of RGB channels), and the pixel values ​​are normalized to the [0,1] interval to obtain the preprocessed target image that conforms to the model input specification;

[0126] The preprocessed target image is then input into the semantic segmentation model trained on the edge device, and forward inference is performed: the encoder extracts multi-scale semantic features layer by layer through 5 stacked stages. The features output by the shallow stage retain the local spatial details of the target image, while the features output by the deep stage contain scene-level global context information of the target image. The decoder receives the multi-scale semantic features output by the encoder, first completes multi-scale semantic aggregation and global context fusion through the SCTHead module, and then performs attention-guided progressive upsampling through the AU_SCTHead module to gradually restore the spatial resolution of the feature map. Finally, it outputs a pixel-level category probability map with the same size as the input target image, and the number of channels of the probability map is consistent with the number of preset road scene semantic categories.

[0127] (4) The pixel-level category probability map is processed by the Softmax function in the channel dimension. The predicted values ​​of each category corresponding to each pixel position are normalized to the interval [0,1], and the sum of all predicted values ​​of each category at each pixel position is 1, so as to obtain the confidence distribution of each category corresponding to each pixel. Then, the category with the highest confidence is taken as the semantic category of each pixel, and a semantic segmentation map with the same size as the target image and each pixel is labeled with the corresponding road scene constituent element category is generated.

[0128] (5) The semantic segmentation map is successively thresholded, morphological filtered, boundary refined and small region removed to obtain the target region mask and contour for autonomous driving perception and planning.

[0129] To verify the effectiveness of this method, the Cityscapes dataset, CVRG-Pano dataset, and CARLA panoramic simulation dataset were used for testing. The core evaluation metrics included mean intersection-over-union ratio (mIoU), mean pixel accuracy (mAcc), number of parameters (ModelSize), and inference speed (FPS), as detailed below:

[0130] (1) Dataset performance testing;

[0131] CVRG-Pano dataset test: input size 512×1024, 7 semantic categories, 4.56M model parameters, mIoU of 76.50%, mAcc of 82.14%; with a parameter count comparable to mainstream lightweight backbones such as MobileNetV2 (4.51M) and EfficientNet-B0 (4.82M), mIoU is improved by 0.32% and 0.16% respectively, achieving a better balance between accuracy and efficiency;

[0132] Cityscapes dataset test: Supports two input sizes. When the input is 1024×512, the number of parameters is 4.34M, mIoU reaches 73.15%, mAcc reaches 82.18%, and the inference speed is 35FPS. When the input is 1536×768, the number of parameters is still 4.34M, mIoU reaches 76.76%, and mAcc reaches 84.77%. Compared with similar NAS solutions such as RLNAS and FasterSeg, it achieves leading accuracy indicators, and the search cost is only 2.5 GPU-days, with extremely high search and deployment efficiency.

[0133] CARLA panoramic simulation dataset test: input size 512×1024, semantic categories 27, model parameters 4.34M, mIoU reaches 68.11%, mAcc reaches 76.62%, inference speed 38FPS, verifying that the model has good cross-scene generalization ability;

[0134] (2) Edge deployment performance testing;

[0135] Deployment performance tests were conducted on the target edge device (NVIDIA Jetson Orin Nano), and the key metrics are as follows: inference latency (≤28ms for input 512×1024, ≤31ms for input 1536×768), memory usage (GPU memory ≤512MB, CPU memory ≤1GB), power consumption (device power consumption ≤15W during inference), and stability (continuous input of 1000 frames of test images, no crashes, no output anomalies, and inference latency fluctuation ≤±1ms). All of these meet the requirements of autonomous driving scenarios for real-time performance, low resource consumption, and stability.

[0136] (3) Visual comparison of segmentation results;

[0137] The semantic segmentation method of this invention is compared with the segmentation performance of mainstream semantic segmentation methods such as Inception V3, EfficientNet-B0, ResNet18, MobileNetV2, and SCTNet-S. Figure 6 As shown, the test scenarios cover complex autonomous driving scenarios such as normal lighting, low light, rain, and occlusion. The comparison results show that the semantic segmentation method of the present invention is clearer and more accurate in the boundary recognition of targets such as roads, pedestrians, vehicles, and buildings, without obvious blurring or breakage. It has better noise suppression effect in complex environments, and the misclassification and omission phenomena are significantly reduced. It can still accurately identify each semantic category and maintain a high degree of segmentation consistency in scenarios with dense targets and occlusion.

Claims

1. A lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios, characterized in that, After acquiring the target image containing road scene components, the target image is input into the semantic segmentation model, which outputs a pixel-level category probability map with the same size as the target image. Then, the pixel-level category probability map is processed by Softmax to obtain the semantic segmentation map corresponding to the road scene components. The steps for building a semantic segmentation model are as follows: (a) Construct and initialize the cloud-edge collaborative segmentation system, including computing environment preparation, teacher model preparation and student model structure initialization; The computing environment adopts a cloud-edge collaborative architecture, where the cloud consists of high-performance servers and the edge devices are resource-constrained embedded inference devices; The student model adopts an encoder-decoder structure, where the encoder is used to extract multi-scale semantic features from the target image, and the decoder is used to fuse the multi-scale semantic features and perform context fusion and attention-guided upsampling to restore spatial resolution and generate pixel-level class probability maps. (b) Generate a candidate encoder for the student model and combine it with the decoder of the student model to obtain a candidate student model; (c) Perform short-round training and evaluation on the candidate student model in the cloud, and perform deployability filtering and Pareto selection based on segmentation accuracy, inference latency, computational complexity and edge device constraints to obtain the target student model; (d) Deploy the teacher model and the target student model in the cloud for full training and knowledge distillation optimization; (e) Deploy the target student model, which has undergone full training and knowledge distillation optimization, to an edge device to obtain the semantic segmentation model.

2. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 1, characterized in that, In step (a), the teacher model uses a pre-trained Vision Transformer class model.

3. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 1, characterized in that, In step (a), the encoder of the student model is divided into sequentially stacked... Each stage Each stage contains multiple basic blocks, which are lightweight inverse residual convolutional blocks.

4. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 3, characterized in that, In step (a), the decoder for the student model includes the SCTHead module and the AU_SCTHead module.

5. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 3, characterized in that, In step (b), the encoder of the student model is defined by a set of parameters. Changing these parameters generates candidate encoders for the student model. These parameters include the kernel size. Channel expansion ratio Jump Link Mark ,coefficient ,parameter and stage depth factor , =1,…, , , and Used to control the shape of basic blocks. , , , The first for the overall scaling encoder The channel width of all basic blocks in the stage. , For the encoder's first The number of basic blocks contained in a stage. , The first for the encoder The sequence modeling structure within each stage is expanded in depth. .

6. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 5, characterized in that, Step (c) is as follows: (c1) Initialize the population; Each individual in the population is a single candidate student model; (c2) Cloud-based short-wheel training and evaluation; Each individual instance is trained briefly in the cloud, with the training objective being a pixel-by-pixel supervised semantic segmentation loss function. Once the minimum value is reached, the segmentation accuracy, inference latency, and computational complexity of each individual are evaluated after training. (c3) Edge device constraint filtering; Individuals that do not meet the edge device constraints are removed, and the remaining individuals constitute a deployable candidate set; (c4) Select the individual that has the best trade-off between "segmentation accuracy - inference latency - computational complexity" from the deployable candidate set and use it as the parent individual; (c5) Perform crossover and mutation operations on the parent individuals to generate offspring individuals; Crossover operation involves swapping the number of encoder stages in two parent individuals. ; The mutation operation involves randomly modifying the kernel size of the encoder in each parent individual. Channel expansion ratio Jump Link Mark ,coefficient ,parameter and stage depth factor In order to explore new combinations; (c6) Population renewal; The parent individuals are merged with their offspring individuals to form a new generation of population; (c7) Determine whether the number of iterations has reached the preset maximum number of iterations. If so, terminate and select a child individual from the child individuals of the last iteration as the target student model; otherwise, return to step (c2).

7. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 1, characterized in that, Step (d) is as follows: (d1) Deploy the teacher model and the target student model in the cloud. The intermediate features of the teacher model are denoted as... The intermediate features of the target student model are denoted as ; (d2) Adjustment Space dimensions , so that it is with Space dimensions Consistent; (d3) Perform a linear transformation and introduce a nonlinear activation, so that the number of channels is reduced from It becomes an intermediate value, then undergoes another linear transformation to change its channel count from the intermediate value to... ,get ; (d4) Perform full training on the target student model in the cloud, with the training objective being the joint loss function. Reaching the minimum value, The calculation formula is as follows: ； ； In the formula, For pixel-by-pixel supervised semantic segmentation loss function; Feature alignment loss in knowledge distillation; Jensen-Shannon divergence; These are the weighting coefficients; Cosine similarity; and They respectively represent based on and The pixel-wise category probability distribution is obtained by Softmax normalization in the channel dimension.

8. The lightweight semantic segmentation method for cloud-edge collaboration in autonomous driving scenarios according to claim 1, characterized in that, After obtaining the semantic segmentation map corresponding to the constituent elements of the road scene, the semantic segmentation map is further subjected to thresholding, morphological filtering, boundary refinement and small region removal processes in sequence to obtain the target region mask and contour for autonomous driving perception and planning.

Images