Intelligent automobile multi-task environment sensing system with cross-scene generalization and sustainable evolution

Abstract

The invention provides a cross-scene generalization and sustainable evolution intelligent automobile multi-task environment sensing system, which comprises a multi-mode sensing input processing module, an occupation prediction-target detection joint decoding module, a meta-learning adaptive training module and a continuous learning self-evolution module. Redundant calculation of multiple tasks is remarkably reduced, and the efficiency of multi-task cooperative processing is improved; a'meta-training-meta-testing 'double-circulation framework is adopted, and parameter priori of a new sample can be adapted through a small amount of gradient updating; in order to further support scene change and task evolution in the long-term operation process of the system, a continuous learning evolution framework integrating task self-detection, sample memory playback and a weight elastic updating mechanism is provided; a technical foundation is laid for constructing an environment sensing system with the characteristics of autonomous learning and self-adaptive evolution for an intelligent automobile by constructing a sample cache region, generating a pseudo tag by utilizing structural constraint and constructing a sub-model for local parameter updating.

Description

Technical Field

[0001] This invention relates to an intelligent vehicle perception system, and more particularly to an intelligent vehicle multi-task environmental perception system that supports cross-scenario generalization and sustainable evolution. Background Technology

[0002] As intelligent vehicles increasingly demand higher precision in understanding the 3D structure of the environment and resolving dynamic scene changes, environmental perception technology is evolving from simple target detection and semantic segmentation tasks towards multi-task collaborative modeling, spatially consistent representation, and online adaptive and evolutionary capabilities. Among these, 3D occupancy prediction environmental perception systems based on a bird's-eye view (BEV) have become one of the core paradigms in the field of intelligent vehicle environmental perception. This system continuously predicts the accessibility, obstacle status, and dynamic changes at various locations in 3D space using a gridded approach, providing a topologically consistent and semantically complete dynamic map for subsequent intelligent vehicle decision-making, planning, and control modules, or an integrated end-to-end architecture. However, current 3D occupancy prediction environmental perception technology still faces several major bottlenecks:

[0003] (1) The system output interface is fixed, limiting the ability of multi-task collaboration and expansion.

[0004] Existing 3D occupancy prediction networks typically output occupancy raster predictions only at a single resolution and with a single depth projection, making it impossible to share feature representations with various environmental perception tasks such as semantic segmentation and object detection. This single output paradigm necessitates the parallel deployment of independent network branches when current 3D occupancy prediction environmental perception systems expand their multi-task capabilities, significantly increasing computational costs and exacerbating error accumulation, thereby limiting the overall performance of 3D occupancy prediction environmental perception systems across multiple tasks.

[0005] (2) The system parameters are fixed, which limits the generalization ability for long-tail scenarios.

[0006] Current 3D occupancy prediction environment perception systems typically employ fixed weights, with system performance entirely dependent on offline supervised training. However, the real-world scenarios faced by intelligent vehicles are complex, varied, and inexhaustible. Manually labeled training samples cannot effectively cover them. In long-tail scenarios, including unknown road structures, unknown types of traffic participants, extreme weather conditions, or abnormal lighting conditions, 3D occupancy prediction environment perception systems trained only once offline may experience significant performance degradation. Currently, the main approach to improving performance in long-tail scenarios is costly, inefficient, and inflexible large-scale offline retraining, lacking a mechanism for rapid online parameter tuning based on a small number of long-tail samples. This severely limits the initial performance capabilities of intelligent vehicle perception systems in long-tail scenarios.

[0007] (3) The system's learning mechanism is rigid and lacks the ability to self-evolve and continuously learn.

[0008] Existing 3D occupancy prediction and perception systems generally lack the ability to continuously evolve in response to changes in perception tasks during long-term deployment. As intelligent vehicles continuously operate in dynamic environments, the system faces dynamic changes such as the addition of new target categories, continuous evolution of perception scenarios, and sudden changes in semantic structure. Existing systems struggle to implement mechanisms for recognizing new scenarios and updating them online. On the one hand, the system cannot effectively integrate new knowledge while maintaining existing task performance, making it prone to knowledge forgetting and performance degradation. On the other hand, the parameters of the backbone inference network are highly fixed, lacking flexible and controllable local fine-tuning mechanisms. This prevents the 3D occupancy prediction and environmental perception system from continuously and progressively improving its performance during actual operation, making it difficult to meet the requirements of real-world vehicle applications for system output stability and generalization capabilities.

[0009] Therefore, how to build an intelligent vehicle 3D occupancy prediction environment perception system with unified multi-task output capability, rapid scene adaptation capability, and continuous learning and self-evolution capability has become a core key technology that urgently needs to be broken through in the field of intelligent vehicle environment perception technology. Summary of the Invention

[0010] To address the aforementioned technical issues, this invention provides a cross-scenario generalization and sustainable evolution intelligent vehicle multi-task environment perception system, comprising a multimodal perception input processing module, an occupancy prediction-object detection joint decoding module, a meta-learning adaptive training module, and a continuous learning self-evolution module. The system operation includes the following steps:

[0011] The multimodal perception input processing module receives multi-view camera images and 3D LiDAR point cloud data collected from the intelligent vehicle, and constructs structure-aligned image BEV representations and point cloud BEV representations respectively, which are then used as inputs for subsequent modules.

[0012] The occupancy prediction-target detection joint decoding module includes a 3D occupancy prediction branch and a 3D target detection branch, which decouple the tasks based on shared image and point cloud BEV features; and outputs an occupancy probability grid and a set of 3D detected targets; the occupancy probability grid, the set of 3D detected targets, and their confidence and consistency indices serve as the basis for online optimization of the meta-learning adaptive training module and the continuous learning self-evolution module;

[0013] The meta-learning adaptive training module uses the network parameters of the occupancy prediction-object detection joint decoding module as the optimization object. In the offline training stage, it incorporates a dual-loop process of "meta-training-meta-testing" to split multiple scene data into task sets and repeatedly perform inner loop small-step updates and outer loop aggregation to obtain meta-initialization parameters with fast generalization ability with few samples. In the online inference stage after system deployment, when a change in scene distribution is detected, the meta-learning adaptive training module performs lightweight fine-tuning based on small-scale new scene samples and pseudo-labeling information constructed by the joint decoding module, and outputs adaptively updated model parameters to update the joint decoding module, achieving efficient adaptation.

[0014] The continuous learning self-evolution module uses the output parameters of the meta-learning adaptive training module as initial input. During the system deployment and operation phases, it combines new scenarios, new tasks, or knowledge drift phenomena encountered during long-term vehicle operation to construct a continuous learning mechanism with memory, transfer, and update capabilities. With online monitoring and phased incremental learning as the core, it dynamically identifies potential perception degradation or task imbalance through joint analysis of perception indicators, including input distribution, prediction confidence, and behavioral feedback. Without human intervention, it automatically initiates low-cost update strategies, including pseudo-label generation, self-supervised optimization, and backbone parameter easing, and outputs the evolved model parameters back to the joint decoding module to realize the self-evolution and stable optimization of the intelligent vehicle multi-task environment perception system in dynamic environments.

[0015] Furthermore, the multimodal perception input processing module includes three sub-modules: an image semantic structure extraction and temporal modeling module, a point cloud sparse coding and geometric mapping module, and an intermodal state modeling and fusion optimization module. It receives raw data from onboard multi-view cameras and LiDAR, and through intramodal coding, temporally consistent modeling, and cross-modal information fusion, generates a unified multimodal BEV feature representation that is structurally consistent, semantically rich, and spatiotemporally coordinated. This provides high-quality, multi-dimensional basic input for subsequent 3D occupancy prediction perception tasks, as detailed below:

[0016] The image semantic structure extraction and temporal modeling module extracts high-density semantic features from continuous image frames from a multi-view camera and constructs an inter-frame temporal consistency structure. The image sampling frequency of the multi-view camera is synchronized with the LiDAR, and the input image sequence is denoted as... ,in Represents a time frame. Indicates the camera number; the processing steps are as follows:

[0017] (1) Spatial semantic feature extraction: Each frame of image is first embedded by 7×7 convolution and max pooling and then enters a four-layer ResNet backbone network to generate four sets of spatial pyramid features. It possesses a multi-scale response structure that ranges from edge textures to global semantics;

[0018] (2) Semantic fusion using FPN: A lightweight feature pyramid fusion network (FPN) is introduced to perform channel alignment, bidirectional scale fusion, and spatial refinement on feature maps at different levels, forming a mid-level fused image semantic feature tensor with consistent resolution and unified channels. ;

[0019] (3) Mamba temporal state structure modeling: The state space model Mamba is used to perform bidirectional recursive modeling on the semantic feature tensor sequence of the mid-layer fused image. The temporal consistency expression is enhanced by the channel memory mechanism and the inter-frame difference mechanism, and the dynamic target trajectory and occlusion structure difference under continuous scene changes are explicitly modeled.

[0020] (4) Image-BEV mapping: The semantic features of each frame of image are mapped to a unified BEV space based on camera extrinsic parameters, pose estimation and depth guidance to form image BEV features. , where 𝐶 is the number of channels, and 𝐻 / 𝑊 are the height and width of the space.

[0021] The point cloud sparse coding and geometric mapping module transforms the sparse 3D point cloud data collected by the vehicle-mounted LiDAR into high-fidelity geometric structure features. The processing steps are as follows:

[0022] (1) Three-dimensional voxel division: the input point cloud is divided into three-dimensional voxel division. Mapped to a 3D voxel mesh, where, Indicates the input point cloud set. Represent the i-th point in the point cloud set; extract the geometric statistical features of the point set within each non-empty voxel to generate the initial sparse tensor. ;

[0023] (2) Point cloud sparse coding: A multi-layer sparse convolution stacked structure is adopted, and batch normalization and activation function are combined to construct a high-resolution structure-preserving sparse semantic tensor. Each layer possesses a residual structure and a cavity receptive field expansion mechanism;

[0024] (3) Point cloud BEV mapping: mapping the sparse semantic tensor Compressed projection is performed along the Z-axis to output the BEV features of the LiDAR point cloud. .

[0025] The intermodal state modeling and fusion optimization module describes the BEV features of the image. Features of point cloud BEV The following steps are involved in performing deep interaction and structure-level fusion:

[0026] (1) Mamba State Recursive Modeler: Constructs a shared state space structure for image and point cloud dual modalities, performs forward and backward recursion in the channel dimension and spatial dimension respectively, and mines the dynamic evolution trajectory and structural semantic coupling relationship between the two modal features of image and point cloud;

[0027] (2) Modal cross-projection mechanism: Introduce image-point cloud state cross-decoding matrix, embed image channel matrix into point cloud output respectively, and conversely embed point cloud channel matrix into image output to complete cross-projection;

[0028] (3) Modality-selective gating mechanism: Sigmoid gating is applied to the cross-projected features along the channel dimension to compress redundant responses, highlight key semantic signals, and output a fused feature tensor. ;

[0029] (4) Channel weighted fusion and structural reconstruction: The fusion is achieved by aligning channel weighted sums to avoid dimensionality explosion caused by splicing; the fusion result is spatially reconstructed to obtain the final unified multimodal BEV feature tensor. This serves as input for downstream task modules to call directly.

[0030] Furthermore, the occupancy prediction-target detection joint decoding module first uses multimodal BEV features Based on this, a branch path structure differentiation strategy is used to feed features into the decoders of two types of tasks: occupancy prediction and 3D object detection, which is suitable for semantic segmentation and object detection tasks with different requirements for accuracy and region consistency.

[0031] The occupancy prediction branch employs a shallow U-Net decoder: top-down two-stage upsampling progressively restores the data to... Resolution The BEV planar grid has a 2D spatial resolution, and jump connections are established between each level and the corresponding compressed feature; finally, a 1×1 convolution is mapped to... The occupancy probability is obtained by passing the height layer channel through a Sigmoid function:

[0032] (1)

[0033] in It is an element-wise Sigmoid activation function. For the number of channels, Batch size; grid The occupancy probability is denoted as Training employs a binary cross-entropy loss with dynamic class balancing weights. :

[0034] (2)

[0035] in, This represents the predicted occupancy probability of the b-th sample at BEV grid coordinates (i,j) and height layer k. The true value of this voxel is assigned a label. These are the weights for positive (occupied) and negative (idle) items, used to adaptively mitigate class imbalance based on duty cycle. This indicates the accumulation of all pixels in the batch, planar grid, and height layers, and through... Normalization and averaging are performed to ensure the stability of the loss scale;

[0036] The 3D target detection branch constructs multi-scale spatial feature maps through an adaptive resolution sampling strategy and introduces a multi-scale anchor box aggregation structure to detect the center position of various targets. ,size ,direction And its category; the detection path achieves high-precision analysis of small targets, occluded objects, and pose orientation through bilinear upsampling and region refinement modules; multimodal BEV feature tensor Heatmap generated by 3×3 convolution And Gaussian focus loss is used:

[0037] (3)

[0038] in, Let be the prediction center confidence of the b-th sample at coordinates (x, y) in the BEV plane; The true value is labeled on the heatmap at the corresponding location, therefore the loss is... When adopted Strengthen recall efforts at key locations, (Background negative samples) when using Suppressing high-confidence false positives; among which B is the focusing coefficient, used to reduce the weight of easily classified samples and highlight difficult samples; B is the batch size. This indicates that the summation and average of the positions of the batch and all BEV grids are calculated.

[0039] Furthermore, the regression head is shifted by the L1 loss prediction center. ,size and heading angle :

[0040] (4)

[0041] in, and These are the 3D bounding box regression vectors for the predicted and ground truth values, respectively, including center offset. Dimensions (l, w, h) and heading angle encoding, The number of positive samples is used to calculate the regression error only at the positive sample locations. The joint loss is calculated as follows:

[0042] (5)

[0043] in, To occupy the prediction branch loss, For heatmap loss, To detect branch loss, These are the loss weight coefficients for each branch;

[0044] In the reasoning stage, based on the heatmap Peak graticules exceeding a threshold of 0.5 are back-projected onto 3D coordinates, and the occupancy of the corresponding graticule is combined. Perform occlusion filtering, retaining only the average occupancy probability. The detection bounding boxes are used to naturally merge dense occupancy grids and sparse targets, generating an occupancy grid map and a 3D target detection set. .

[0045] Furthermore, the meta-learning adaptive training module includes an offline training phase and an online inference phase. In the offline training phase, the collected multi-domain data is first divided into several "scene tasks," denoted as:

[0046] (6)

[0047] Each of the scenario tasks It covers occupancy prediction and 3D object detection data labels under corresponding distributions, used to simulate the perception needs that vehicles may encounter in different real-world scenarios; subsequently, it divides each task into "support sets". "and query set" "Support sets for inner loop updates, and query sets for outer loop optimization;"

[0048] The system from Randomly select several domain tasks; for each task First, read its support set. Handed over to the inner loop for processing; within the inner loop, the current network parameters are... Perform a few steps of gradient descent to obtain a temporary update. The formal definition is:

[0049] (7)

[0050] in This represents the meta-initialized model parameters to be learned during the meta-learning phase. Indicates the inner loop learning rate. Indicating support set The multi-task fusion loss calculated above (usually composed of a weighted average of the occupancy prediction loss and the 3D object detection loss) is used to calculate the loss. Indicates the parameter gradient operator, Indicates in the task above utilize support set According to the gradient descent rule The task-adaptive temporary parameters are obtained after performing a few-step update to simulate few-sample adaptation.

[0051] In the outer ring, temporary parameters Put into query set Calculate loss and the original parameters Perform backpropagation to balance the few-step learning performance across multiple domains; repeat the process for different tasks. The inner and outer loops are processed to update the network until convergence, obtaining a meta-initializer with the "few-step convergence" property for multiple distributions. ;

[0052] Obtain meta-initialization parameters during offline training. Afterwards, the system is deployed to the vehicle and enters the online inference phase. When a significant change in scene distribution is detected, the system automatically extracts a small number of samples from the current frame and its neighbors to form a support set. Subsequently, the step size 1-3 is updated using the few-step gradient update rule, and the online learning rate is... To maintain consistency with the offline inner loop, Fine-tuning was performed to obtain temporary online parameters. :

[0053] (8)

[0054] in, This represents the online support set extracted from the current frame and its neighboring frames during the online phase. Indicates the online learning rate. This represents the joint / fusion loss calculated on the online support set.

[0055] After fine-tuning, the inference weights are replaced immediately to achieve rapid adaptation to the new domain; if the scenario reverts to the normal distribution, the system automatically rolls back to the previous state. This ensures stable output and long-term generalization ability.

[0056] As a preferred approach, fusion loss is used in both the inner and outer rings:

[0057] (9)

[0058] in, This represents the cross-entropy or Focal Loss of the occupied raster prediction. This represents the regression and classification losses for 3D object detection. This is the weighting coefficient. During the meta-training process, this loss is applied to both the support set and the query set for different scenario tasks, so that the network can maintain the performance of both occupancy prediction and object detection with a small number of updates.

[0059] Through multiple rounds of domain task training and iterative optimization, the system ultimately obtains initial parameter values ​​that possess "rapid generalization ability with few samples". .

[0060] Furthermore, during the system deployment and operation phases, the continuous learning and self-evolution module dynamically constructs a sample buffer and performs periodic incremental learning to address new scenarios, tasks, or knowledge drift phenomena encountered by intelligent vehicles during long-term operation, gradually improving the environmental adaptability and self-evolution capability of the 3D occupancy prediction perception system. During system operation, this module continuously monitors vehicle perception output and behavioral feedback data; if one of the following conditions is detected, the incremental learning process is triggered:

[0061] • The confidence level of the 3D occupancy prediction raster output oscillates significantly, and the prediction results show high variance;

[0062] • Inconsistent feedback exists between the perception output and the downstream path planning / behavior prediction module, resulting in continuous conflicts;

[0063] • There is a significant structural alignment shift between the perception continuous frames and multi-source information from semantic maps, IMU, and GPS;

[0064] • Sudden changes in scene visual features;

[0065] • After long-term operation, the overall performance indicators of the system show a slow decline trend;

[0066] When the triggered event meets the preset threshold, the system starts the environment buffer. The cache contains multimodal data including images, point clouds, BEV features, and prior trajectories, along with their perception prediction results, which are used as an incremental learning support set.

[0067] Using the current backbone inference model Initial occupancy prediction raster pseudo-labels are generated for the buffer data, and topological constraints are introduced. This includes consistency checks on lane lines, target trajectories, and map elements, and the construction of a joint pseudo-label supervision system. :

[0068] (10)

[0069] in, Represents cross-entropy loss, This indicates the pseudo-labels generated by the current backbone inference model and filtered and verified. This represents the occupancy prediction result output by the model for the same input sample. The weighting coefficients of the structural consistency constraint terms are represented. Indicates mean square error. This represents the structural quantities or trajectory projection quantities inferred from the model. This represents a reference structure built from prior knowledge, such as road topology, lane lines, or map features, used for... Consistency constraints are applied. This mechanism improves the accuracy of pseudo-labels while injecting cross-source structural information, thereby enhancing the quality of the supervision signal for new samples.

[0070] This module constructs a light quantum network. ,by To initialize parameters, only a portion of the structure is activated for updates, supporting small batches of training samples extracted in a concentrated manner. Perform few-step training using a combined loss method of self-supervised and pseudo-labeled loss. Optimize to ensure fast training convergence and controllable inference latency;

[0071] Meanwhile, to avoid catastrophic forgetting, the system uses soft-constraint distillation constraint loss. :

[0072] (11)

[0073] in, Indicates the KL divergence. and Let x represent the teacher model and the student model, respectively, and x represent the input sample. and Let these represent the parameter sets for the teacher model and the student model, respectively. Represents the squared norm 2 constraint term in the parameter space. This is the weighting coefficient of the consistency regularization term for this parameter, used to suppress excessive deviation of student model parameters from teacher model parameters during continuous learning and updating, thereby reducing the risk of catastrophic forgetting. For the teacher model, For the student model, the new and old models are forced to maintain a certain consistency in the output and parameter space;

[0074] This module ultimately uses a policy-gated function. Decide whether to update the student model parameters to the main model. If any of the following conditions are met, then fusion will be performed:

[0075] • Improved mean of 3D occupancy prediction raster prediction and reduced IoU or BCE loss;

[0076] • A significant decrease in the entropy value of the uncertainty heatmap indicates an increase in confidence.

[0077] • The rate of conflict in behavioral decision-making feedback has decreased or stability has increased;

[0078] If the fusion conditions are met, then parameter updates are performed using the exponential moving average (EMA) strategy:

[0079] (12)

[0080] in To integrate the smoothing coefficient and enable the gradual injection of new knowledge;

[0081] This module, in conjunction with a meta-learning adaptive mechanism and a joint task optimization strategy for 3D occupancy prediction, constructs a closed-loop evolutionary system with dynamic memory, low-supervised evolution, and stable transfer capabilities.

[0082] The beneficial effects of this invention are:

[0083] This invention provides a cross-scenario generalization and sustainable evolution intelligent vehicle multi-task environment perception system. In the BEV feature output stage, a joint decoding module of "occupancy prediction – target detection" is established based on a backbone BEV feature sharing and differentiated dual-branch structure. The occupancy prediction branch takes BEV features as input, employs high-resolution upsampling and temporal consistency constraints to generate a temporally consistent raster occupancy probability map; the target detection branch performs instance-level target attribute regression within the BEV feature space, covering center position, geometric dimensions, and orientation angle. This multi-task output module, based on a unified BEV feature representation, significantly reduces redundant computation across multiple tasks, improves the efficiency of multi-task collaborative processing, effectively solves the problems of task fragmentation, performance imbalance, and insufficient scalability in existing systems, and constructs a unified expression framework for multi-task perception.

[0084] This invention introduces meta-learning concepts into the offline training and online update processes of the system model, employing a dual-loop architecture of "meta-training-meta-testing." This allows for parameter priors to be adapted to new samples with only a small number of gradient updates. In practical use, when the intelligent vehicle travels to long-tail scenes not covered by the training samples, the system only needs a small number of samples to complete online lightweight fine-tuning, updating and replacing inference parameters in real time. This overcomes the dependence of 3D occupancy prediction perception systems on large-scale manually labeled data and long-term offline training, while avoiding the high cost, low efficiency, and poor flexibility of current large-scale retraining methods. This enhances the 3D occupancy prediction perception system's ability to adapt to long-tail scenes.

[0085] To further support scenario changes and task evolution during long-term system operation, this invention proposes a continuous learning and evolutionary architecture that integrates task self-detection, sample memory playback, and elastic weight update mechanisms. During operation, the system continuously monitors the confidence fluctuations of the perceived output, the stability of behavioral feedback, and abrupt changes in the environmental state. It automatically identifies potential new tasks, new categories, or knowledge drift events and triggers a lightweight incremental learning process. By constructing a sample cache, generating pseudo-labels using structural constraints, and building sub-models for local parameter updates, the system can achieve long-term online upgrades and continuous enhancement of semantic expression capabilities of the environmental perception system without interrupting the main inference task. This breaks the traditional "training-freeze-degeneration" paradigm of environmental perception systems, laying a technical foundation for building environmental perception systems with "autonomous learning and adaptive evolution" characteristics for intelligent vehicles. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the overall architecture of the present invention;

[0087] Figure 2 This is a schematic diagram of the multimodal sensing input processing module structure of the present invention;

[0088] Figure 3 This is a schematic diagram of the principle of the occupancy prediction-target detection joint decoding module of the present invention;

[0089] Figure 4 This is a schematic diagram of the meta-learning adaptive training module structure of the present invention;

[0090] Figure 5 This is a schematic diagram of the continuous learning self-evolution module structure of the present invention. Detailed Implementation

[0091] like Figure 1-5 As shown, this invention provides a cross-scenario generalization and sustainable evolution intelligent vehicle multi-task environment perception system, including a multimodal perception input processing module, an occupancy prediction-object detection joint decoding module, a meta-learning adaptive training module, and a continuous learning self-evolution module. The system operation includes the following steps:

[0092] (1) The multimodal perception input processing module is used to receive multi-view camera images and three-dimensional lidar point cloud data collected from intelligent vehicles, and construct structure-aligned image BEV representations and point cloud BEV representations respectively. This module extracts semantic features with strong spatiotemporal consistency through multi-scale semantic encoding and temporal recursion enhancement of image modalities. At the same time, it adopts sparse voxelization and columnar aggregation strategies to efficiently encode point clouds and generate geometric features with stable spatial structure. Finally, through unified channel, scale and spatial alignment, multimodal BEV features with complementary semantics and geometry are formed, providing a unified input basis with high expressiveness for subsequent multi-task fusion decoding and self-evolution training modules.

[0093] (2) The joint decoding module for occupancy prediction and target detection includes a 3D occupancy prediction branch and a 3D target detection branch. The two branches are decoupled based on shared multimodal BEV features. The 3D occupancy prediction branch outputs the "passage / blockage" probability for each BEV grid and generates a high-resolution, continuous temporal occupancy grid by combining time consistency constraints and appropriate upsampling. The 3D target detection branch adopts a multi-resolution resampling mechanism and anchor point aggregation structure to improve the ability to resolve small targets, occluded objects and orientation information. The two branches work together to significantly improve the multi-task robustness of the intelligent vehicle 3D occupancy prediction environment perception system.

[0094] (3) The meta-learning adaptive training module incorporates a dual-loop process of "meta-training - meta-testing" in the offline training stage, providing the network with parameter priors that can converge quickly with a small number of new samples; in the online inference stage, it performs lightweight fine-tuning based on small-scale new scene data or pseudo-labels to achieve efficient adaptation. Its main process includes: splitting various scene data into task sets, repeatedly performing inner loop small-step updates and outer loop aggregations, and converging to the initial weights that are highly sensitive to the domain distribution; after online deployment, when encountering long-tail scenes (abnormal lighting, extreme weather, or no road structure), only a very small number of scene samples are needed to quickly update the gating parameters or normalization layer gradients; this module does not affect the main inference performance of the system and significantly improves the cross-scene generalization ability of 3D occupancy prediction.

[0095] (4) The continuous learning self-evolution module is used during the system deployment and operation phases to construct a continuous learning mechanism with memory, transfer, and update capabilities, based on new scenarios, new tasks, or knowledge drift phenomena encountered during long-term vehicle operation. This enables the 3D occupancy prediction perception system to self-evolve and achieve stable optimization in dynamic environments. The module focuses on online monitoring and phased incremental learning. Through joint analysis of perception indicators such as input distribution, prediction confidence, and behavioral feedback, it dynamically identifies potential perception degradation or task imbalance. Without manual intervention, it automatically initiates low-cost update strategies such as pseudo-label generation, self-supervised optimization, and backbone parameter easing. The overall module design emphasizes a balance between knowledge retention and new knowledge absorption, effectively preventing "catastrophic forgetting" and overfitting risks, and enhancing the task adaptability and system robustness of the intelligent vehicle 3D occupancy prediction environment perception system in complex, long-term, and heterogeneous scenarios.

[0096] See Figure 1 and Figure 2 As shown, the multimodal perception input processing module receives raw data from the vehicle-mounted multi-source sensor system (including multi-view cameras and LiDAR). Through intramodal coding, temporally consistent modeling, and cross-modal information fusion, it generates a unified multimodal BEV feature representation that is structurally consistent, semantically rich, and spatiotemporally coordinated, providing high-quality, multi-dimensional basic input for subsequent 3D occupancy prediction perception tasks. This module forms the input skeleton of the perception information flow in the system and includes three sub-modules: an image semantic structure extraction and temporal modeling module, a point cloud sparse coding and geometric mapping module, and an intermodal state modeling and fusion optimization module, as detailed below:

[0097] The image semantic structure extraction and temporal modeling module is used to extract high-density semantic features from continuous image frames from a multi-view camera and construct an inter-frame temporal consistency structure. The multi-view camera is a six-view surround wide-angle camera mounted on the system, with an image sampling frequency synchronized with the LiDAR. The input image sequence is denoted as... ,in Represents a time frame. Indicates the camera number; the processing steps are as follows:

[0098] (1) Spatial semantic feature extraction: Each frame of image is first embedded by 7×7 convolution and max pooling and then enters a four-layer ResNet backbone network to generate four sets of spatial pyramid features. It possesses a multi-scale response structure that ranges from edge textures to global semantics;

[0099] (2) Semantic fusion using FPN: A lightweight feature pyramid fusion network (FPN) is introduced to perform channel alignment, bidirectional scale fusion, and spatial refinement on feature maps at different levels, forming a mid-level fused image semantic feature tensor with consistent resolution and unified channels. ;

[0100] (3) Mamba temporal state structure modeling: The state space model Mamba is used to perform bidirectional recursive modeling on the semantic feature tensor sequence of the mid-layer fused image. The temporal consistency expression is enhanced by the channel memory mechanism and the inter-frame difference mechanism, and the dynamic target trajectory and occlusion structure difference under continuous scene changes are explicitly modeled.

[0101] (4) Image-BEV mapping: The semantic features of each frame of image are mapped to a unified BEV space based on camera extrinsic parameters, pose estimation and depth guidance to form image BEV features. It possesses high semantic density, spatial consistency, and location responsiveness.

[0102] The point cloud sparse coding and geometric mapping module is used to convert sparse 3D point cloud data acquired by vehicle-mounted LiDAR into high-fidelity geometric structure features. The processing steps are as follows:

[0103] (1) Three-dimensional voxel division: the input point cloud is divided into three-dimensional voxel division. Mapping to a 3D voxel mesh, extracting the geometric and statistical features of the point set within each non-empty voxel, and generating an initial sparse tensor. ;

[0104] (2) Point cloud sparse coding: A multi-layer sparse convolution stacked structure is adopted, and batch normalization and activation function are combined to construct a high-resolution structure-preserving sparse semantic tensor. Each layer has a residual structure and a void receptive field expansion mechanism to enhance the ability to resolve detailed structures and edge contours;

[0105] (3) Point cloud BEV mapping: Compressed projection is performed in the Z-axis direction to output the BEV features of the lidar point cloud. A good balance is achieved between structural alignment and sparsity fidelity.

[0106] The aforementioned intermodal state modeling and fusion optimization module is used for image BEV features. BEV features of point cloud The following steps are involved in performing deep interaction and structure-level fusion:

[0107] (1) Mamba State Recursive Modeler: Construct a bimodal shared state space structure, perform forward and backward recursion in the channel dimension and spatial dimension respectively, and mine the dynamic evolution trajectory and structural semantic coupling relationship between the two modal features of image and point cloud;

[0108] (2) Modal cross-projection mechanism: Introduce image-point cloud state cross-decoding matrix, embed image channel matrix into point cloud output respectively, and embed point cloud channel matrix into image output to complete cross-projection and realize information transduction and complementary enhancement;

[0109] (3) Modality-selective gating mechanism: Sigmoid gating is applied to the cross-projected features along the channel dimension to compress redundant responses, highlight key semantic signals, and output a fused feature tensor. ;

[0110] (4) Channel weighted fusion and structural reconstruction: The fusion is achieved by aligning channel weighted sums to avoid dimensionality explosion caused by splicing; the fusion result is spatially reconstructed to obtain the final unified multimodal BEV feature tensor. This serves as input for downstream task modules to call directly.

[0111] See Figure 1 and Figure 3 As shown, the occupancy prediction-object detection joint decoding module includes an occupancy prediction branch and a 3D object detection branch, used to complete parallel modeling and task decoupling of two heterogeneous perception tasks, occupancy prediction and 3D object detection, based on shared BEV features. This module first uses multimodal BEV features... Based on this, a branch path structure differentiation strategy is used to feed features into the decoders of two types of tasks: occupancy prediction and 3D object detection. This is suitable for semantic segmentation and object detection tasks with different requirements for accuracy and region consistency.

[0112] The occupancy prediction branch employs a shallow U-Net decoder: top-down two-stage upsampling progressively restores the data to... Resolution The BEV planar grid has a 2D spatial resolution, and jump connections are established between each level and the corresponding compressed feature; finally, a 1×1 convolution is mapped to... The occupancy probability is obtained by passing the height layer channel through a Sigmoid function:

[0113] (1)

[0114] in It is an element-wise Sigmoid activation function. For the number of channels, Batch size; grid The occupancy probability is denoted as Training employs a binary cross-entropy loss with dynamic class balancing weights. :

[0115] (2)

[0116] in, This represents the predicted occupancy probability of the b-th sample at BEV grid coordinates (i,j) and height layer k. The true value of this voxel is assigned a label. These are the weights for positive (occupied) and negative (idle) items, used to adaptively mitigate class imbalance based on duty cycle. This indicates the accumulation of all pixels in the batch, planar grid, and height layers, and through... Normalization and averaging are performed to ensure the stability of the loss scale;

[0117] The 3D target detection branch constructs multi-scale spatial feature maps through an adaptive resolution sampling strategy and introduces a multi-scale anchor box aggregation structure to detect the center position of various targets. ,size ,direction And its category; the detection path achieves high-precision analysis of small targets, occluded objects, and pose orientation through bilinear upsampling and region refinement modules; multimodal BEV feature tensor Heatmap generated by 3×3 convolution And Gaussian focus loss is used:

[0118] (3)

[0119] in, Let (b) be the prediction center confidence of the (b)th sample at coordinates (x, y) in the BEV plane; The true value is labeled on the heatmap at the corresponding location, therefore the loss is... When adopted Strengthen recall efforts at key locations, (Background negative samples) when using Suppressing high-confidence false positives; among which B is the focusing coefficient, used to reduce the weight of easily classified samples and highlight difficult samples; B is the batch size. This indicates that the summation and average of the positions of the batch and all BEV grids are calculated.

[0120] Furthermore, the regression head is shifted by the L1 loss prediction center. ,size and heading angle :

[0121] (4)

[0122] in, and These are the 3D bounding box regression vectors for the predicted and ground truth values, respectively, including center offset. Dimensions (l, w, h) and heading angle encoding, The number of positive samples is used to calculate the regression error only at the positive sample locations. The joint loss is calculated as follows:

[0123] (5)

[0124] in, To occupy the prediction branch loss, For heatmap loss, To detect branch loss, These are the loss weight coefficients for each branch;

[0125] In the reasoning stage, according to Peak graticules exceeding a threshold of 0.5 are back-projected onto 3D coordinates, and the occupancy of the corresponding graticule is combined. Perform occlusion filtering, retaining only the average occupancy probability. The detection bounding boxes are used to naturally merge dense occupancy grids and sparse targets, generating an occupancy grid map and a 3D target detection set. .

[0126] Overall, the occupancy prediction-target detection joint decoding module seamlessly couples dense raster-based occupancy estimation with sparse bounding box-style 3D target detection through a dual-branch structure of "shared BEV features + branch path differentiation": shallow U-Net occupancy decoding and multi-scale anchor box detection are coupled in parallel. The former accurately recovers the four-dimensional occupancy probability, while the latter provides high-resolution analysis of the target's center position, geometric dimensions, and orientation angle. The two complement each other under the synergistic effect of joint loss, enabling the system to simultaneously obtain a dense occupancy map and high-precision 3D target boxes, thereby significantly improving the overall perception accuracy and robustness of the intelligent vehicle environmental perception system in complex traffic environments.

[0127] See Figure 1 and Figure 4 The meta-learning adaptive training module includes an offline training phase and an online inference phase. In the offline training phase, a meta-learning framework based on a dual-loop structure of "meta-training – meta-testing" is introduced. This module learns transferable parameter priors by simulating a rapid adaptation process for diverse scenario tasks. Therefore, after system deployment, only a small number of scenario samples and a few parameter updates are needed to achieve accuracy recovery and dynamic adaptation in the online inference phase, significantly reducing the reliance on massive annotations and long-term training.

[0128] During the offline training phase, the collected multi-domain data (including different road structures, lighting conditions, weather conditions, etc.) is first divided into several "scene tasks," denoted as:

[0129] (6)

[0130] Each of the scenario tasks It covers occupancy prediction and 3D object detection data labels under corresponding distributions, used to simulate the perception needs that vehicles may encounter in different real-world scenarios; subsequently, it divides each task into "support sets". "and query set" "Support sets for inner loop updates, and query sets for outer loop optimization."

[0131] The system from Randomly select several domain tasks; for each task First, read its support set. Handed over to the inner loop for processing; within the inner loop, the current network parameters are... Perform a few steps of gradient descent to obtain a temporary update. The formal definition is:

[0132] (7)

[0133] in This represents the meta-initialized model parameters to be learned during the meta-learning phase. Indicates the inner loop learning rate. Indicating support set The multi-task fusion loss calculated above (usually composed of a weighted average of the occupancy prediction loss and the 3D object detection loss) is used to calculate the loss. Indicates the parameter gradient operator, Indicates in the task above utilize support set According to the gradient descent rule The task-adaptive temporary parameters obtained after performing a few-step update are used to simulate few-sample adaptation.

[0134] In the outer ring, temporary parameters Put into query set Calculate loss and the original parameters Perform backpropagation to balance the few-step learning performance across multiple domains; repeat the process for different tasks. The inner and outer loops are processed to update the network until convergence, obtaining a meta-initializer with the "few-step convergence" property for multiple distributions. This demonstrates how, after iterative iteration, parameters continuously converge to the highest priority, which is sensitive to scene differences.

[0135] Obtain meta-initialization parameters during offline training. Afterwards, the system is deployed to the vehicle and enters the online inference phase. When a significant change in scene distribution is detected (such as a sudden change in lighting, absence of road structure, or extreme weather conditions), the system automatically extracts a small number of samples from the current frame and its neighbors to form a support set. Subsequently, the step size 1-3 is updated using the few-step gradient update rule, and the online learning rate is... To maintain consistency with the offline inner loop, Fine-tuning was performed to obtain temporary online parameters:

[0136] (8)

[0137] in, This represents the online support set extracted from the current frame and its neighboring frames during the online phase. Indicates the online learning rate. This represents the joint / fusion loss calculated on the online support set.

[0138] The update process only affects the gating parameters or normalization layer, without touching the main inference flow, thus not impacting the real-time frame rate; after fine-tuning, the inference weights are immediately replaced to achieve rapid adaptation to the new domain; if the scene reverts to the normal distribution, the system automatically rolls back to the normal distribution. This ensures stable output and long-term generalization ability. To balance the joint optimization of occupancy prediction and 3D object detection, this invention employs a fusion loss in both the inner and outer loops:

[0139] (9)

[0140] in, This represents the cross-entropy or Focal Loss of the occupied raster prediction. This represents the regression and classification losses for 3D object detection. is the weighting coefficient. During meta-training, this loss is applied to both the support set and query set for different scenario tasks, enabling the network to maintain both occupancy prediction and object detection performance with only a few updates.

[0141] Through multiple rounds of domain task training and iterative optimization, the system ultimately obtains initial parameter values ​​that possess "rapid generalization ability with few samples". In practical deployments, this mechanism allows the system to achieve instantaneous accuracy recovery and steady-state output when perception performance degrades or scene changes abruptly, significantly enhancing the robustness and adaptability of the 3D occupancy prediction perception system.

[0142] See Figure 1 and Figure 5As shown, the continuous learning and self-evolution module is used to dynamically construct a sample buffer and perform periodic incremental learning for new scenarios, new tasks or knowledge drift phenomena encountered by intelligent vehicles during long-term driving in the system deployment and operation phase, so as to gradually improve the environmental adaptability and self-evolution capability of the three-dimensional occupancy prediction perception system.

[0143] The core objective of this module is to break the rigid pattern of "single-stage training - long-term freezing" in existing 3D occupancy prediction systems, and to build a complete online learning chain from operation monitoring, scene sampling, weak label reconstruction, lightweight fine-tuning to knowledge fusion, so as to realize the continuous enhancement of the cognitive self-repair and expression capabilities of intelligent vehicles in unknown environments.

[0144] During system operation, this module continuously monitors the vehicle's perception output and behavioral feedback data. If any of the following conditions are detected, the incremental learning process is triggered:

[0145] • The confidence level of the 3D occupancy prediction raster output oscillates significantly, and the prediction results show high variance;

[0146] • Inconsistent feedback exists between the perception output and the downstream path planning / behavior prediction module, resulting in continuous conflicts;

[0147] • There is a significant structural alignment shift between the perception continuous frames and multi-source information from semantic maps, IMU, and GPS;

[0148] • Sudden changes in scene visual features (such as low-light conditions like rain, snow, fog, tunnels, or nighttime).

[0149] • After long-term operation, the overall performance indicators of the system show a slow decline trend.

[0150] When the triggered event meets the preset threshold, the system starts the environment buffer. The cache contains multimodal data including images, point clouds, BEV features, and prior trajectories, along with their perception prediction results, which are used as an incremental learning support set.

[0151] To alleviate the challenge of insufficient manual annotation, the system utilizes the current backbone inference model. Initial occupancy prediction raster pseudo-labels are generated for the buffer data, and topological constraints are introduced. (e.g., lane lines, target trajectories, map elements, etc.) Perform consistency checks and construct joint pseudo-label supervision:

[0152] (10)

[0153] in, Represents cross-entropy loss, This indicates the pseudo-labels generated by the current backbone inference model and filtered and verified. This represents the occupancy prediction result output by the model for the same input sample. The weighting coefficients of the structural consistency constraint terms are represented. Indicates mean square error. This represents the structural quantities or trajectory projection quantities inferred from the model. This represents a reference structure built from prior knowledge, such as road topology, lane lines, or map features, used for... Consistency constraints are applied. This mechanism improves the accuracy of pseudo-labels while injecting cross-source structural information, effectively enhancing the quality of the supervision signal for new samples.

[0154] This module constructs a light quantum network. ,by To initialize parameters, only a portion of the structure is activated for updates, supporting small batches of training samples extracted in a concentrated manner. Perform few-step training using a combined loss method of self-supervised and pseudo-labeled loss. Optimize to ensure fast training convergence and controllable inference latency;

[0155] Meanwhile, to avoid catastrophic forgetting, the system uses a soft-constraint distillation strategy:

[0156] (11)

[0157] in, Indicates the KL divergence. and Let x represent the teacher model and the student model, respectively, and x represent the input sample. and Let these represent the parameter sets for the teacher model and the student model, respectively. Represents the squared norm 2 constraint term in the parameter space. This is the weighting coefficient of the consistency regularization term for this parameter, used to suppress excessive deviation of student model parameters from teacher model parameters during continuous learning and updating, thereby reducing the risk of catastrophic forgetting. For the teacher model, the updated student model For student models, force the new and old models to maintain a certain consistency in the output and parameter space.

[0158] The system ultimately uses a policy-gated function. Decide whether to Update the parameters to the main model If any of the following conditions are met, then fusion will be performed:

[0159] • Improved mean of 3D occupancy prediction raster prediction and reduced IoU or BCE loss;

[0160] • A significant decrease in the entropy value of the uncertainty heatmap indicates an increase in confidence.

[0161] • The rate of conflict in behavioral decision-making feedback has decreased or stability has increased;

[0162] If the fusion conditions are met, parameter updates are performed using an exponential moving average (EMA) strategy:

[0163] (12)

[0164] in To achieve a smoothing coefficient (e.g., 0.95~0.99), new knowledge is gradually injected to avoid system instability caused by sudden changes.

[0165] This module, in conjunction with the meta-learning adaptive mechanism and the joint task optimization strategy of 3D occupancy prediction, constructs a closed-loop evolution system with dynamic memory, low-supervised evolution and stable transfer capabilities, which significantly improves the self-adaptability and robustness of the intelligent vehicle 3D occupancy prediction environmental perception system in complex, dynamic and long-term task environments.

Claims

1. A cross-scenario generalization and sustainable evolution intelligent vehicle multi-task environmental perception system, characterized in that: The system includes a multimodal sensing input processing module, an occupancy prediction-object detection joint decoding module, a meta-learning adaptive training module, and a continuous learning self-evolution module. The system runtime includes the following steps: The multimodal perception input processing module receives multi-view camera images and 3D LiDAR point cloud data collected from the intelligent vehicle, and constructs structure-aligned image BEV representations and point cloud BEV representations respectively, which are then used as inputs for subsequent modules. The joint decoding module for occupancy prediction and target detection includes a 3D occupancy prediction branch and a 3D target detection branch, which decouple the tasks based on shared image and point cloud BEV features; It outputs an occupancy probability grid and a three-dimensional detection target set; the occupancy probability grid, the three-dimensional detection target set, and their confidence and consistency indices serve as the basis for online optimization of the meta-learning adaptive training module and the continuous learning self-evolution module; The meta-learning adaptive training module optimizes the network parameters of the occupancy prediction-object detection joint decoding module. During offline training, it incorporates a dual-loop process of "meta-training—meta-testing," splitting various scenario data into task sets and repeatedly performing inner-loop small-step updates and outer-loop aggregation to obtain meta-initialization parameters with rapid generalization capabilities with few samples. In the online inference phase after system deployment, when a change in scenario distribution is detected, the meta-learning adaptive training module performs lightweight fine-tuning based on small-scale new scenario samples and pseudo-labeling information constructed from the joint decoding module's output, outputting adaptively updated model parameters to update the joint decoding module, achieving efficient adaptation. The continuous learning self-evolution module uses the output parameters of the meta-learning adaptive training module as initial input. During the system deployment and operation phases, it combines new scenarios, new tasks, or knowledge drift phenomena encountered during long-term vehicle operation to construct a continuous learning mechanism with memory, transfer, and update capabilities. With online monitoring and phased incremental learning as the core, it dynamically identifies potential perception degradation or task imbalance through joint analysis of perception indicators, including input distribution, prediction confidence, and behavioral feedback. Without human intervention, it automatically initiates low-cost update strategies, including pseudo-label generation, self-supervised optimization, and backbone parameter easing, and outputs the evolved model parameters back to the joint decoding module to realize the self-evolution and stable optimization of the intelligent vehicle multi-task environment perception system in dynamic environments.

2. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 1, characterized in that: The multimodal perception input processing module includes three sub-modules: an image semantic structure extraction and temporal modeling module, a point cloud sparse coding and geometric mapping module, and an intermodal state modeling and fusion optimization module. It receives raw data from vehicle-mounted multi-view cameras and LiDAR, and generates a unified multimodal BEV feature representation that is structurally consistent, semantically rich, and spatiotemporally coordinated through intramodal coding, temporally consistent modeling, and cross-modal information fusion. This provides high-quality, multi-dimensional basic input for subsequent 3D occupancy prediction perception tasks.

3. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 2, characterized in that: The image semantic structure extraction and temporal modeling module extracts high-density semantic features from continuous image frames from a multi-view camera and constructs an inter-frame temporal consistency structure. The image sampling frequency of the multi-view camera is synchronized with the LiDAR, and the input image sequence is denoted as... ,in Represents a time frame. Indicates the camera number; the processing steps are as follows: (1) Spatial semantic feature extraction: Each frame of image is first embedded by 7×7 convolution and max pooling and then enters a four-layer ResNet backbone network to generate four sets of spatial pyramid features. It possesses a multi-scale response structure that ranges from edge textures to global semantics; (2) Semantic fusion using FPN: A lightweight feature pyramid fusion network (FPN) is introduced to perform channel alignment, bidirectional scale fusion, and spatial refinement on feature maps at different levels, forming a mid-level fused image semantic feature tensor with consistent resolution and unified channels. ; (3) Mamba temporal state structure modeling: The state space model Mamba is used to perform bidirectional recursive modeling on the semantic feature tensor sequence of the mid-layer fused image. The temporal consistency expression is enhanced by the channel memory mechanism and the inter-frame difference mechanism, and the dynamic target trajectory and occlusion structure difference under continuous scene changes are explicitly modeled. (4) Image-BEV mapping: The semantic features of each frame of image are mapped to a unified BEV space based on camera extrinsic parameters, pose estimation and depth guidance to form image BEV features. .

4. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 2, characterized in that: The point cloud sparse coding and geometric mapping module transforms the sparse 3D point cloud data collected by the vehicle-mounted LiDAR into high-fidelity geometric structure features. The processing steps are as follows: (1) Three-dimensional voxel division: the input point cloud is divided into three-dimensional voxel division. Mapped to a 3D voxel mesh, where, Indicates the input point cloud set. Represent the i-th point in the point cloud set; extract the geometric statistical features of the point set within each non-empty voxel to generate the initial sparse tensor. ; (2) Point cloud sparse coding: A multi-layer sparse convolution stacked structure is adopted, and batch normalization and activation function are combined to construct a high-resolution structure-preserving sparse semantic tensor. Each layer possesses a residual structure and a cavity receptive field expansion mechanism; (3) Point cloud BEV mapping: mapping the sparse semantic tensor Compressed projection is performed along the Z-axis to output the BEV features of the LiDAR point cloud. .

5. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 2, characterized in that: The intermodal state modeling and fusion optimization module describes the BEV features of the image. BEV features of point cloud The following steps are involved in performing deep interaction and structure-level fusion: (1) Mamba State Recursive Modeler: Constructs a shared state space structure for image and point cloud dual modalities, performs forward and backward recursion in the channel dimension and spatial dimension respectively, and mines the dynamic evolution trajectory and structural semantic coupling relationship between the two modal features of image and point cloud; (2) Modal cross-projection mechanism: Introduce image-point cloud state cross-decoding matrix, embed image channel matrix into point cloud output respectively, and conversely embed point cloud channel matrix into image output to complete cross-projection; (3) Modality-selective gating mechanism: Sigmoid gating is applied to the cross-projected features along the channel dimension to compress redundant responses, highlight key semantic signals, and output a fused feature tensor. ; (4) Channel weighted fusion and structural reconstruction: The fusion is achieved by aligning channel weighted sums to avoid dimensionality explosion caused by splicing; the fusion result is spatially reconstructed to obtain the final unified multimodal BEV feature tensor. This serves as input for downstream task modules to call directly.

6. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 1, characterized in that: The occupancy prediction-target detection joint decoding module first uses the multimodal BEV feature tensor Based on this, a branch path structure differentiation strategy is used to feed features into the decoders of two types of tasks: occupancy prediction and 3D object detection, which is suitable for semantic segmentation and object detection tasks with different requirements for accuracy and region consistency. The occupancy prediction branch employs a shallow U-Net decoder: top-down two-stage upsampling progressively restores the data to... Resolution The two-dimensional spatial resolution of the BEV planar grid is determined, and jumpers are established between each level and the corresponding compressed feature; The final 1×1 convolution is mapped to The occupancy probability is obtained by passing the height layer channel through a Sigmoid function: （1） in It is an element-wise Sigmoid activation function. For the number of channels, Batch size; grid The occupancy probability is denoted as Training employs a binary cross-entropy loss with dynamic class balancing weights. : （2） in, This represents the predicted occupancy probability of the b-th sample at BEV grid coordinates (i,j) and height layer k. The true value of this voxel is assigned a label. These are the weights for positive and negative class items, used to adaptively mitigate class imbalance based on the duty cycle. This indicates the accumulation of all pixels in the batch, planar grid, and height layers, and through... Normalization and averaging are performed to ensure the stability of the loss scale; The 3D target detection branch constructs multi-scale spatial feature maps through an adaptive resolution sampling strategy and introduces a multi-scale anchor box aggregation structure to detect the center position of various targets. ,size ,direction And its category; the detection path achieves high-precision analysis of small targets, occluded objects, and pose orientation through bilinear upsampling and region refinement modules; multimodal BEV feature tensor Heatmap generated by 3×3 convolution And Gaussian focus loss is used: （3） in, Let be the prediction center confidence of the b-th sample at coordinates (x, y) in the BEV plane; The true value is labeled on the heatmap at the corresponding location, therefore the loss is... When adopted Strengthen recall efforts at key locations, When adopted Suppressing high-confidence false positives; among which B is the focusing coefficient, used to reduce the weight of easily classified samples and highlight difficult samples; B is the batch size. This indicates that the summation and average of the grid positions of the batch and all BEVs are taken. Furthermore, the regression head is shifted by the L1 loss prediction center. ,size and heading angle : （4） in, and These are the 3D bounding box regression vectors for the predicted and ground truth values, respectively, including center offset. Dimensions l, w, h and heading angle encoding, The number of positive samples is used to calculate the regression error only at the positive sample locations; the joint loss is: （5） in, To occupy the prediction branch loss, For heatmap loss, To detect branch loss, These are the loss weight coefficients for each branch; In the reasoning stage, based on the heatmap Peak graticules exceeding a threshold of 0.5 are back-projected onto 3D coordinates, and the occupancy of the corresponding graticule is combined. Perform occlusion filtering, retaining only the average occupancy probability. The detection bounding boxes are used to naturally merge dense occupancy grids and sparse targets, generating an occupancy grid map and a 3D target detection set. .

7. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 1, characterized in that: The meta-learning adaptive training module includes an offline training phase and an online inference phase. In the offline training phase, the collected multi-domain data is first divided into several "scene tasks," denoted as: （6） Each of the scenario tasks It covers occupancy prediction and 3D object detection data labels under corresponding distributions, used to simulate the perception needs that vehicles may encounter in different real-world scenarios; subsequently, it divides the "support set" for each task. "and query set" "Support sets for inner loop updates, and query sets for outer loop optimization;" The system from Randomly select several domain tasks; for each task First, read its support set. Handed over to the inner loop for processing; within the inner loop, the current network parameters are... Perform a few steps of gradient descent to obtain a temporary update. The formal definition is: （7） in This represents the meta-initialized model parameters to be learned during the meta-learning phase. Indicates the inner loop learning rate. Indicating support set The multi-task fusion loss calculated above, Indicates the parameter gradient operator, Indicates in the task above utilize support set According to the gradient descent rule The task-adaptive temporary parameters obtained after performing a few-step update are used to simulate few-sample adaptation. In the outer ring, temporary parameters Put into query set Calculate loss and the original parameters Perform backpropagation to balance the few-step learning performance across multiple domains; repeat the process for different tasks. The inner and outer loops are processed to update the network until convergence, obtaining a meta-initializer with the "convergence in a few steps" property for multiple distributions. ; Obtain meta-initialization parameters during offline training. Afterwards, the system is deployed to the vehicle and enters the online inference phase. When a significant change in scene distribution is detected, the system automatically extracts a small number of samples from the current frame and its neighbors to form a support set. Subsequently, the step size 1-3 is updated using the few-step gradient update rule, and the online learning rate is... To maintain consistency with the offline inner loop, Fine-tuning was performed to obtain temporary online parameters: （8） in, This represents the online support set extracted from the current frame and its neighboring frames during the online phase. Indicates the online learning rate. This represents the joint / fusion loss computed on the online support set; After fine-tuning, the inference weights are replaced immediately to achieve rapid adaptation to the new domain; if the scenario reverts to the normal distribution, the system automatically rolls back to the previous state. This ensures stable output and long-term generalization ability. Fusion loss is used in both the inner and outer rings: （9） in, This represents the cross-entropy or Focal Loss of the occupied raster prediction. This represents the regression and classification losses for 3D object detection. This is the weighting coefficient. During the meta-training process, this loss is applied to both the support set and the query set for different scenario tasks, so that the network can maintain the performance of both occupancy prediction and object detection with a small number of updates. Through multiple rounds of domain task training and iterative optimization, the system ultimately obtains initial parameter values ​​that possess "fast generalization ability with few samples". .

8. The intelligent vehicle multi-task environmental perception system with cross-scenario generalization and sustainable evolution as described in claim 1, characterized in that: During system deployment and operation, the continuous learning and self-evolution module dynamically constructs a sample buffer and performs periodic incremental learning to address new scenarios, tasks, or knowledge drift encountered by intelligent vehicles during long-term operation, gradually improving the environmental adaptability and self-evolution capability of the 3D occupancy prediction perception system. During system operation, this module continuously monitors vehicle perception output and behavioral feedback data; if one of the following conditions is detected, the incremental learning process is triggered: (1) The confidence level of the three-dimensional occupancy prediction raster output oscillates significantly, and the prediction results show high variance; (2) There is inconsistency between the perception output and the feedback from the downstream path planning / behavior prediction module, resulting in continuous conflicts; (3) There is a significant structural alignment shift between the perception continuous frames and the semantic map, IMU, and GPS multi-source information; (4) Sudden changes in scene visual features; (5) After long-term operation, the overall performance indicators of the system show a slow decline trend; When the triggered event meets the preset threshold, the system starts the environment buffer. The cache contains multimodal data including images, point clouds, BEV features, and prior trajectories, along with their perception prediction results, which are used as an incremental learning support set. Using the current backbone inference model Initial occupancy prediction raster pseudo-labels are generated for the buffer data, and topological constraints are introduced. This includes consistency checks on lane lines, target trajectories, and map elements, and the construction of a joint pseudo-label supervision system. （10） in, Represents cross-entropy loss, This indicates the pseudo-labels generated by the current backbone inference model and filtered and verified. This represents the occupancy prediction result output by the model for the same input sample. The weighting coefficients of the structural consistency constraint terms are represented. Indicates mean square error. This represents the structural quantities or trajectory projection quantities inferred from the model. This represents a reference structure built from prior knowledge, such as road topology, lane lines, or map features, used for... Consistency constraints are applied; this mechanism improves the accuracy of pseudo-labels while injecting cross-source structural information, thereby enhancing the quality of the supervision signal for new samples. This module constructs a light quantum network. ,by To initialize parameters, only a portion of the structure is activated for updates, supporting small batches of training samples extracted in a concentrated manner. Perform few-step training using a combined loss method of self-supervised and pseudo-labeled loss. Optimize to ensure fast training convergence and controllable inference latency; Meanwhile, to avoid catastrophic forgetting, the system uses a soft-constraint distillation strategy: （11） in, Indicates the KL divergence. and Let x represent the teacher model and the student model, respectively, and x represent the input sample. and Let these represent the parameter sets for the teacher model and the student model, respectively. Represents the squared norm 2 constraint term in the parameter space. The weight coefficients of the consistency regularization term are used to suppress excessive deviations between student model parameters and teacher model parameters during continuous learning and updating, thereby reducing the risk of catastrophic forgetting; For the teacher model, the updated student model For the student model, the new and old models are forced to maintain a certain consistency in the output and parameter space; This module ultimately uses a policy-gated function. Decide whether to update the student model parameters to the main model. If any of the following conditions are met, then fusion will be performed: (1) The mean of 3D occupancy prediction raster prediction is improved, and the IoU or BCE loss is reduced; (2) A significant decrease in the entropy value of the uncertainty heatmap indicates an increase in confidence; (3) The rate of conflict in behavioral decision-making feedback decreases or stability increases; If the fusion conditions are met, then parameter updates are performed using the exponential moving average (EMA) strategy: （12） in To integrate the smoothing coefficient and enable the gradual injection of new knowledge; This module, in conjunction with a meta-learning adaptive mechanism and a joint task optimization strategy for 3D occupancy prediction, constructs a closed-loop evolutionary system with dynamic memory, low-supervised evolution, and stable transfer capabilities.

Images