Autonomous driving perception model forgetting method based on conflict avoidance task vector

Abstract

This invention discloses a forgetting method for autonomous driving perception models based on conflict avoidance task vectors, belonging to the field of autonomous driving technology. This method uses a globally pre-trained baseline model as a unified parameter anchor point. Each batch of data to be deleted is fine-tuned and then negatively charged to generate a forgetting task vector. This vector is then filtered by amplitude to generate a sparse mask, compressed, and encrypted for storage. During aggregation, the global dominant direction is calculated along each parameter dimension, and conflicting components with opposite signs are removed to eliminate cross-batch update adversarial behavior. Finally, the aggregated vector is superimposed on the baseline model to output a compliant perception model. This invention does not iteratively modify the baseline throughout the process. After processing multiple batches, the recognition rate of forgotten samples remains low, the general perception accuracy only slightly decreases, storage overhead is significantly reduced, and it is compatible with various automotive-grade vision-language backbone networks. It covers scenarios such as outsourced road test cleanup, owner privacy deletion, test vehicle decommissioning, and regulatory compliance rectification, meeting automotive data compliance requirements.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a forgetting method for autonomous driving perception models based on conflict avoidance task vectors. Background Technology

[0002] Currently, leading domestic automakers, autonomous driving technology companies, and mobility platforms are all deploying CLIP architecture multimodal autonomous driving perception models. The model training relies on massive amounts of road test multimodal data accumulated over the years, with a data coverage period of more than 10 years. The data types include a large amount of sensitive personal information and road scenarios with compliance restrictions, such as vehicle surround view RGB images, LiDAR 3D point clouds, traffic sign text, driver faces, license plates of social vehicles, long-term commuting trajectories of car owners, and images of pedestrians along the street.

[0003] There are four types of time-series data deletion needs that OEMs routinely have, and these deletion requests are scattered and sequential, forming a long time-series task flow: 1) Batch cleanup of outsourced road test data upon expiration, requiring the destruction of highway and suburban road test samples from 5 / 10 / 15 years ago in batches; 2) Voluntary deletion of personal data by vehicle owners, with applications for deletion of privacy data submitted when vehicle owners cancel their accounts or transfer / scrap their vehicles, and the processing time is scattered and interspersed among road test cleanup tasks over many years; 3) Cleanup of internal test vehicle projects after they are decommissioned, with samples collected from dedicated test sites destroyed in batches after the verification or simulation testing of new models is completed; 4) Regulatory special compliance rectification, with the destruction of illegally collected existing road test data in multiple batches.

[0004] Existing machine forgetting solutions in the field of autonomous driving are all designed for single-batch forgetting. Directly reusing long-term, multi-batch road test data has the following drawbacks: The update direction of model parameters corresponding to different years and different road scenarios has positive and negative conflicts. Subsequent batch forgetting updates will offset the forgetting effect of older road tests, causing the recognition of forgotten pedestrians, license plates, and road features to be restored, posing a data leakage risk. Traditional solutions modify the model after processing each batch as the basis for the next round. After processing more than 10 batches of cross-year data, the recognition accuracy of general-purpose cars, pedestrians, and lane lines drops significantly. Restoring this accuracy would require hundreds of high-end GPUs and several weeks of retraining from scratch. Each batch generates a complete and dense parameter vector. Ten years and hundreds of batches correspond to tens of TB of cloud storage. Small and medium-sized car companies' low-end servers experience memory overflow and frequent backend crashes. In addition, existing algorithms are only designed for general image classification models and are not optimized for the three-modal autonomous driving architecture of image + LiDAR + traffic text. Direct porting would destroy core driving safety functions such as 3D obstacle detection and long-distance sign recognition. Meanwhile, the existing continuous learning and incremental fine-tuning technologies aim to add new knowledge without forgetting old knowledge, which is completely opposite to the technical goal of deleting specified privacy knowledge in batches over a long period of time and prohibiting the reproduction of old privacy knowledge. Therefore, they cannot be adapted to automotive data compliance scenarios.

[0005] Therefore, it is necessary to propose a continuous forgetting scheme that can unify parameter anchor points, aggregate cross-batch conflict perception, and adapt to the multimodal architecture of autonomous driving. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing single-batch forgetting tools for autonomous driving, such as the inability to handle long-term road test cleanup by year, cross-scenario forgetting update conflicts leading to the reproduction of privacy features, multiple iterations damaging driving perception accuracy, and excessive overhead of dense vector storage. This invention provides an autonomous driving perception model forgetting method based on conflict avoidance task vectors.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a forgetting method for an autonomous driving perception model based on conflict avoidance task vectors, characterized in that the autonomous driving perception model is constructed by extending a vision-language pre-trained model, and the method includes:

[0008] Step S1: Initialization of the cloud compliance platform and construction of the temporal forgetting task queue: Loading a fixed global pre-trained benchmark model. The global pre-trained baseline model weights are not modified throughout the process and serve as the unified parameter calculation benchmark for all forgetting tasks; temporal forgetting instructions are received and a temporal forgetting task queue is generated according to the order of instruction acceptance time; a new encrypted cloud-based historical task vector storage pool is created. ;

[0009] Step S2: Generate autonomous driving-specific forgetting task vectors in batches: Retrieve single batches of multimodal road test datasets to be deleted in the order of queue time sequence. Forced to use a globally pre-trained benchmark model Using the initial weights as the sole basis for fine-tuning, a short-term fine-tuning is performed on the current batch dataset to obtain a temporarily fine-tuned model. ; Calculate the parameter offset difference between the temporary fine-tuned model and the globally pre-trained baseline model. Negate the parameter offset difference to generate the original forgetting task vector corresponding to this batch of road test data. ;

[0010] Step S3: Sparse mask compression and encrypted storage of road test task vector: Sort all parameters of the original forgotten task vector by taking the absolute value, select the core perception parameters with a predetermined proportion before the amplitude to construct a binary mask, and obtain the sparse forgotten task vector by multiplying element by element; encrypt the sparse forgotten task vector and store it in the historical task vector storage pool.

[0011] Step S4, Cross-Batch Conflict Avoidance Symbol-Aware Aggregation: Traverse all historical sparse task vectors in the storage pool, and perform conflict filtering and integration for each independent parameter dimension: Accumulate all historical sparse vector components under that parameter dimension, calculate the globally dominant forgetting direction for that dimension using a sign function; filter all valid vector components with signs consistent with the globally dominant forgetting direction, and discard conflicting components with opposite signs; take the average of the filtered conflict-free valid components to generate the globally aggregated forgetting task vector for the current time series. ;

[0012] Step S5: Generate a compliant autonomous driving perception model based on a global benchmark: using a global pre-trained benchmark model Using the base vector as a foundation, and superimposing the scaled aggregated forgetting task vector, the compliance awareness model after processing the first t batches of time-series road test data is calculated: , where λ is the forgetting intensity scaling factor;

[0013] Repeat steps S2 to S5 until all batches in the time-series forgetting task queue have completed forgetting processing.

[0014] Furthermore, before each batch of road test datasets to be deleted is retrieved in step S2, it is extracted using uniform stratified sampling without replacement. The stratification dimensions include road year, road type, lighting time, vehicle type, and vehicle owner identity, so that the distribution of a single batch of samples is consistent with the distribution of the full road test data.

[0015] Furthermore, the autonomous driving perception model includes an onboard image encoding submodule, a lidar point cloud encoding submodule, a traffic text semantic encoding submodule, and a cross-modal feature fusion perception submodule;

[0016] In step S2, the fine-tuning only updates the weights of the vehicle image encoding submodule and the lidar point cloud encoding submodule, while the traffic text semantic encoding submodule is locked and frozen throughout the process to maintain the semantic recognition capability of traffic sign text even if privacy and road features are forgotten.

[0017] Furthermore, the autonomous driving perception model is compatible with four types of automotive-grade CLIP vision-language backbone networks: ViT-B / 32, ViT-L / 14, RN50, and RN101. The vehicle-mounted image encoding submodule outputs road image features with a dimension range of 256 to 2048, the lidar point cloud encoding submodule outputs three-dimensional perception features with a dimension range of 2048 to 8192, and the traffic text semantic encoding submodule outputs signage text features with a dimension range of 512 to 1024.

[0018] Furthermore, in step S4, the criterion for determining the conflicting components is: if the product of the components of any two different batches of sparse task vectors is less than 0 under the same parameter dimension, then it is determined to be an update conflicting component.

[0019] Furthermore, the time-sequential forgetting instructions include: batch cleanup instructions for outsourced road test authorizations expiring, requests for deletion of privacy data for vehicle owner account cancellation or vehicle scrapping, regulatory vehicle data compliance rectification notices, and requests for cleanup of internal test vehicle projects that have been taken offline.

[0020] Furthermore, each batch of multimodal road test datasets to be deleted includes vehicle surround-view RGB images, LiDAR 3D point clouds, traffic sign text, driver faces, vehicle license plates, and paired samples of commuting trajectories.

[0021] Furthermore, the method is adaptable to at least one of the following autonomous driving road test data cleaning business scenarios:

[0022] Scenario 1: Overdue outsourced road tests continue to be forgotten - Batch delete images and point cloud road test samples collected by outsourced companies for highways and suburbs by year;

[0023] Scenario 2: Forgotten time sequence of car owner's personal commuting data - delete privacy samples of face, license plate, and commuting trajectory collected by single user's in-vehicle camera and radar in batches according to the time sequence of car owner cancellation and vehicle scrapping;

[0024] Scenario 3: Special Cleanup of Internal Test Vehicles - After the project is completed or the simulation test cycle ends, delete the multimodal samples collected from dedicated test roads and extreme working condition tests in batches;

[0025] Scenario 4: Batch rectification and cleanup for regulatory compliance - destroying illegally collected existing road test data in multiple batches.

[0026] Furthermore, in step S2, the fine-tuning employs the AdamW optimizer;

[0027] The method is configured with the following fixed hyperparameters: the fine-tuning learning rate is 1×10⁻⁶. -5 The weight decay coefficient is 0.1, the task vector sparsity retention ratio is 30%, the forgetting intensity scaling factor λ = 0.7, the batch sample size is 128, the single batch fine-tuning iteration is 10 rounds, and the gradient warm-up steps are 200.

[0028] The present invention also provides an autonomous driving perception model forgetting system based on conflict avoidance task vectors, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-mentioned method.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] (1) Effectively suppresses the recurrence of privacy knowledge due to temporal forgetting. Existing solutions will recover and recognize face, license plate and old road features that were deleted in the early years after batch processing; the present invention automatically removes all parameter dimension reverse conflict update components through symbolic perception conflict aggregation mechanism, and the recognition accuracy of forgotten samples is still significantly reduced after processing dozens or hundreds of batches.

[0031] (2) Protect the general perception accuracy of autonomous driving throughout the process. Existing solutions modify the model as the base for the next round of forgetting each batch, and the perception accuracy drops significantly after multiple iterations; this invention locks the original pre-trained benchmark model as a unified anchor point throughout the process, and only achieves forgetting by aggregating and superimposing task vectors. After time-series processing, the general perception accuracy is only slightly reduced, and the model can be directly installed in the vehicle, saving the massive GPU computing power and time cost of retraining a large model of tens of billions.

[0032] (3) Significantly reduces the hardware threshold for cloud storage. This invention retains only 30% of the core perception parameters to generate sparse forgetting vectors, reducing storage usage by 70%. Low-end cloud servers can stably store hundreds of batches of ten years' worth of data, avoiding memory overflow and background crashes.

[0033] (4) Adapted to automotive-grade trimodal perception architecture. This invention optimizes the image + LiDAR + traffic text trimodal CLIP model, locks the traffic text encoding module during the fine-tuning stage, does not damage the core driving functions such as road sign and long-distance sign recognition, and is compatible with four types of backbone networks: ViT-B / 32, ViT-L / 14, RN50, and RN101. It does not require reconstruction of the underlying architecture and has low engineering modification requirements.

[0034] (5) Universal for all scenarios and supports ultra-long time series extension. One solution covers four types of routine cleanup business: overdue outsourced road tests, vehicle owner personal privacy, test vehicle special projects, and regulatory rectification. It fully matches the long time series business process from ten-year-old road tests to new vehicle owner data of this year. As the number of forgotten batches increases, the aggregation calculation overhead increases linearly and steadily, the target forgetting effect is stable and has no rebound, and it has the capability to implement industrial-grade ultra-long time series. Attached Figure Description

[0035] Figure 1 This is a block diagram illustrating the implementation principle of the forgetting method for autonomous driving perception models based on conflict avoidance task vectors provided by this invention. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0038] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0039] like Figure 1 As shown, this invention provides a method for forgetting autonomous driving perception models based on conflict avoidance task vectors. Its core lies in achieving continuous machine forgetting of the autonomous driving perception model in long-term, multi-batch data deletion scenarios through a globally fixed baseline model, sparse forgetting task vector construction, and a cross-batch symbolic perception conflict aggregation mechanism. The method first initializes the global baseline model and task queue, then generates forgetting task vectors batch by batch, performs sparse compression and historical storage on the task vectors, and finally generates a unified forgetting vector through symbolic perception conflict aggregation, ultimately constructing an autonomous driving perception model that meets data deletion requirements. The method is independently deployed on the vehicle manufacturer's cloud-based automotive data compliance backend, seamlessly integrating with the existing CLIP architecture autonomous driving multimodal perception training platform. The complete execution logic consists of five core steps, supported by a standardized automotive-grade hyperparameter system.

[0040] Step S1: Initialize the cloud compliance platform and construct the time-series forgetting task queue.

[0041] The autonomous driving cloud data compliance backend independently initiates this method, and the first step is to complete the global baseline model locking and task queue orchestration:

[0042] 1) Load the automotive-grade global pre-trained benchmark model that has been fully pre-trained by the automaker based on ten years of full road test data. The weights of the baseline model are locked throughout the process without any permanent modification, serving as the unified parameter calculation anchor for all time-series forgetting tasks. All batch forgetting vectors are generated based on this baseline, eliminating the accuracy decay caused by iterative modifications to the baseline from the root.

[0043] The model comprises an in-vehicle image encoding submodule, a LiDAR point cloud encoding submodule, a traffic text semantic encoding submodule, and a cross-modal feature fusion perception submodule. Specifically, the in-vehicle image encoding submodule processes RGB road images from forward / surround-view cameras; the LiDAR point cloud encoding submodule processes 3D obstacle point clouds; the traffic text semantic encoding submodule identifies road signs and traffic light text markings; and the cross-modal feature fusion perception submodule fuses multimodal features to output obstacle, pedestrian, and vehicle detection results.

[0044] 2) Receives four types of time-series road test data deletion instructions: batch cleanup instructions for outsourced road test authorizations expiring, requests for deletion of private data from vehicle owner account cancellations or vehicle scrapping, regulatory vehicle data compliance rectification notices, and internal test vehicle project decommissioning cleanup requests. The system sorts the instructions according to the order of online and offline acceptance time, automatically dividing them into four time-series task tiers: Tier 1: older highway and suburban road test batches (over 10 years old); Tier 2: mid-term urban road test batches (5-10 years old); Tier 3: urban commuting batches (nearly 5 years old); and Tier 4: new vehicle owner personal data batches (this year), generating an ordered time-series forgetting task queue.

[0045] 3) Create an independent encrypted cloud storage partition as a storage pool for historical sparse task vectors. The partitions utilize enterprise-grade encrypted hard drives to permanently store lightweight forgetting vectors corresponding to each batch of road tests; all standardized and fixed operating hyperparameters are entered: AdamW fine-tunes the learning rate. Weight decay of 0.1, vector sparsity retention ratio of 30%, and forget scaling factor. The sample size per batch is 128, the fine-tuning iterations per batch are 10 rounds, and the gradient warm-up steps are 200.

[0046] Step S2: Generate autonomous driving-specific forgetting task vectors in batches.

[0047] Retrieve complete batches of multimodal road test datasets to be deleted in sequence according to the queue time order. The dataset is stratified and sampled to include road images specific to this batch, laser point clouds, traffic sign text, driver faces, and paired samples of license plates from other vehicles. The original forgetting task vector is generated in three steps:

[0048] 1) Force the use of a globally pre-trained baseline model As the only initial weights to fine-tune, only the vehicle image encoding and LiDAR point cloud encoding sub-modules were activated for fine-tuning, while the traffic text semantic encoding sub-module remained frozen and locked throughout the process to avoid compromising the general road sign text recognition capabilities. The AdamW optimizer was used for the current batch. For short-term fine-tuning training, 128 sets of trimodal paired samples are read in a single run, and the training is repeated for 10 rounds. The first 200 steps are used for gradient warm-up to smooth the training process, and weight decay of 0.1 is used to suppress overfitting of single-batch road features. After training, a temporary fine-tuned model adapted to the specific road / vehicle owner privacy features of this batch is obtained. .

[0049] 2) Calculate the offset difference between the temporary fine-tuned model and all parameters of the global pre-trained baseline model. This difference accurately represents the model's ability to remember all parameter changes generated by the privacy data from this batch of road tests.

[0050] 3) Negate the parameter offset difference to generate the original forgetting task vector corresponding to this batch of road test data. This vector serves to offset the privacy features of the current batch of faces, license plates, and roads recorded in the baseline model.

[0051] The main innovation of this step is that the forgetting vectors for all years and all scenarios are generated based on the same invariant global benchmark model. Vectors from different batches are in a unified parameter space, which can be directly compared and aggregated in a unified manner, and there is no problem of iterative basis shift.

[0052] In this embodiment, before each batch of road test datasets to be deleted is retrieved in step S2, it is extracted using uniform stratified sampling without replacement. The stratification dimensions include road year, road type, lighting period, vehicle type, and vehicle owner identity, so that the distribution of samples in a single batch is consistent with the distribution of the full road test data, avoiding sampling imbalance that could lead to incomplete forgetting or a significant drop in general road perception accuracy. Each batch of multimodal road test datasets to be deleted includes vehicle surround-view RGB images, LiDAR 3D point clouds, traffic sign text, driver faces, vehicle license plates, and paired samples of commuting trajectories.

[0053] The fine-tuning only updates the weights of the vehicle image encoding submodule and the lidar point cloud encoding submodule, while the traffic text semantic encoding submodule is locked and frozen throughout the process to maintain the semantic recognition capability of traffic sign text even if privacy and road features are forgotten.

[0054] Preferably, the autonomous driving perception model is compatible with four types of automotive-grade CLIP vision-language backbone networks: ViT-B / 32, ViT-L / 14, RN50, and RN101. The onboard image encoding submodule outputs road image features with a dimension range of 256–2048, the LiDAR point cloud encoding submodule outputs 3D perception features with a dimension range of 2048–8192, and the traffic text semantic encoding submodule outputs signage text features with a dimension range of 512–1024.

[0055] Step S3: Sparse mask compression and encrypted storage of road test task vectors

[0056] Sort all parameters of the original forgetting task vector by absolute value, select core perception parameters with a predetermined proportion before the amplitude to construct a binary mask, and obtain the sparse forgetting task vector by element-wise multiplication; encrypt the sparse forgetting task vector and store it in the historical task vector storage pool.

[0057] The original dense forgetting task vector has the same dimension as the total parameter dimension of the perception model. Over 90% of the low-amplitude parameters within the vector contribute nothing to the memory of road and vehicle owner privacy features, constituting noisy redundancy. Direct long-term storage would consume massive amounts of cloud space, and multiple batches stacked together would exacerbate subsequent aggregation conflicts. This step uses a top-30% amplitude masking sparsification scheme to sparse the vector:

[0058] 1) For the original forgetting task vector Take the absolute value of all parameters and sort them. Select the top 30% of core sensing parameters by amplitude and generate a binary mask matrix. High-amplitude core parameters within the mask are marked as 1, while other redundant noise parameters are marked as 0.

[0059] 2) Element-wise multiplication of the original vector and the mask This yields a sparse forgetting task vector that retains only 30% of the effective components.

[0060] 3) Encrypt the sparse vector and write it to the cloud-based historical task vector storage pool, then update the storage pool set. Each batch of road / vehicle owner data corresponds to a separate set of lightweight sparse vectors;

[0061] After compression, the cloud storage usage of vector data is reduced by 70%. Small and medium-sized car companies can stably store hundreds of batches of ten years' worth of road test data on low-end cloud servers without memory overflow, background lag or crash.

[0062] Step S4: Cross-batch conflict avoidance symbol-aware aggregation

[0063] By retrieving all historical sparse task vectors from the storage pool, ranging from the earliest 10 years of old highway road tests to the current year's vehicle owner batches, and performing conflict detection, filtering, and mean aggregation operations separately for each set of independent parameter dimensions, conflicts in road scene updates across years are effectively eliminated, and privacy feature reproduction is greatly reduced.

[0064] 1) For parameter dimension i, accumulate all historical sparse vector components under that parameter dimension, and calculate the globally dominant forgetting direction for that dimension using a sign function: If the sum of all components equals 0, then this dimension does not clearly guide the direction of forgetting. .

[0065] 2) Iterate through all historical batch vectors in this dimension, filter out the effective components that are completely consistent with the global dominant forgetting direction, and discard conflicting components with opposite signs. The conflicting components are essentially reverse updates brought about by the difference in features between suburban roads in the early years and urban commuting scenarios in recent years. If the conflicting components are retained, they will cancel each other out the forgetting effect during aggregation, causing deleted faces and license plates to be re-recognized by the model.

[0066] The criterion for determining conflicting components is: if the product of the components of any two different batches of sparse task vectors is less than 0 under the same parameter dimension, it is determined to be an update conflicting component. All conflicting components are removed and do not participate in aggregation, thus addressing the compliance leakage risk of old scene face, license plate, and road features being re-identified by the model when old roads are forgotten first and recent vehicle owner data is processed later.

[0067] 3) Calculate the arithmetic mean of all conflict-free valid components after screening, and generate a global aggregated forgetting task vector that integrates all time-series batch deletion requirements. .

[0068] This step is the main innovation of the present invention. Its main innovation lies in blocking the recovery of early-year-cleared privacy knowledge from subsequent forgetting operations at the parameter layer through a symbol-aware conflict filtering mechanism, thereby solving the fatal compliance defect of temporal forgetting knowledge reproduction in existing solutions.

[0069] Step S5: Generate a compliant autonomous driving perception model based on a global benchmark.

[0070] Unlike traditional methods that iteratively modify the model as the basis for the next round, this invention uses an unmodified global pre-trained baseline model throughout the entire process. Using this as the unique basis, the globally aggregated forgetting task vector generated in the previous step is superimposed, along with a fixed forgetting intensity scaling factor. The compliant automotive-grade multimodal perception model after processing the first t batches of time-series road test data is calculated as follows:

[0071]

[0072] Scaling factor After optimization using massive amounts of road test data from highways, urban areas, and rain / snow conditions, the optimal balance between compliance oversight and driving perception accuracy has been achieved. Increased size enhances privacy feature removal, but slightly reduces general obstacle recognition capabilities; If the image is too small, it cannot completely erase sensitive features such as faces and license plates.

[0073] The output compliance model can be directly sent to the vehicle domain controller for vehicle, pedestrian, lane line and traffic light perception and reasoning during normal driving. The general road recognition accuracy fluctuates only slightly and does not affect the core safety functions of autonomous driving following and emergency avoidance.

[0074] Repeat steps S2 to S5 until all old outsourced road tests, mid-term urban road tests, and current year's vehicle owner commuting batches in the time-series forgetting task queue have been forgotten.

[0075] Preferably, the method is adapted to at least one of the following autonomous driving road test data cleaning business scenarios:

[0076] Scenario 1: Overdue outsourced road tests continue to be forgotten - Batch delete images and point cloud road test samples collected by outsourced companies for highways and suburbs by year;

[0077] Scenario 2: Forgotten time sequence of car owner's personal commuting data - delete privacy samples of face, license plate, and commuting trajectory collected by single user's in-vehicle camera and radar in batches according to the time sequence of car owner cancellation and vehicle scrapping;

[0078] Scenario 3: Special Cleanup of Internal Test Vehicles - After the project is completed or the simulation test cycle ends, delete the multimodal samples collected from dedicated test roads and extreme working condition tests in batches;

[0079] Scenario 4: Batch rectification and cleanup for regulatory compliance - destroying illegally collected existing road test data in multiple batches.

[0080] The above scenarios correspond to the core business systems of automakers' autonomous driving systems: Overdue outsourced road tests requiring adaptation to cloud-based general road pre-training models and high-speed static obstacle detection systems; forgotten owner personal data requiring adaptation to in-vehicle cockpit perception, driver behavior monitoring, and private commuting trajectory reasoning modules; forgotten test vehicle data requiring adaptation to real-vehicle-simulation joint perception training platforms and extreme rain and snow condition algorithm verification systems; and regulatory rectification and cleanup requiring adaptation to automakers' road test data compliance audit backend and multimodal data anonymization platforms.

[0081] The specific implementation process of the present invention will be further described below with reference to the embodiments.

[0082] Implementation Scenario 1: A leading automaker has been outsourcing highway road testing for ten years and has forgotten about compliance in three batches. A leading domestic automaker has been cooperating with a third-party data collection company from 2010 to 2025. The authorization for all highway and suburban road test data nationwide will expire in 2025. The compliance department requires the cleanup of the existing old road test samples in three batches: the first batch is the 15-year old highway road test from 2010 to 2015, the second batch is the mid-term suburban road test from 2016 to 2020, and the third batch is the current year's urban commuter road test from 2021 to 2025.

[0083] The complete execution flow of this invention's continuous forgetting program deployed in the cloud-based automotive data compliance backend of automakers is as follows:

[0084] S1. Program initialization:

[0085] Load the ViT-B / 32 autonomous driving multimodal benchmark model pre-trained from full road tests from 2010 to 2025, lock the weights and do not modify them; build a three-tiered temporal forgetting task queue, input all standard automotive-grade hyperparameters, and open an encrypted cloud sparse vector storage pool.

[0086] S2. Generation of the original forgetting task vector:

[0087] Prioritize processing the first batch of 15-year highway road test datasets from 2010 to 2015, fine-tuning only the image and point cloud encoding sub-modules. After fine-tuning, generate the original forgotten task vectors, filter the top-30% amplitude using a mask to obtain sparse vectors, and store them in the storage pool. Retrieve only vectors from this batch in the storage pool and aggregate them to generate the first round of compliance perception model. Verify that the accuracy of pedestrian and license plate recognition on highways from 2010 to 2015 is close to 0, with no privacy feature recognition.

[0088] S3. The second batch of time-series tasks processes the suburban road test batches from 2016 to 2020, generating corresponding sparse vectors and storing them in the storage pool; the system retrieves all sparse vectors from the first and second batches, calculates the global dominant forgetting direction on a parameter-by-parameter dimension, removes the reverse conflict components of the road scenarios in the two batches, and aggregates them to obtain a unified forgetting vector integrating the two batches of old road tests, outputting the second round of compliance model; verification shows that the fifteen-year and five-year old road test samples cannot be identified, and no knowledge reproduction occurs.

[0089] S4. Finally, process the third batch of urban commuter vehicle owner datasets from 2021-2025, generating sparse vectors and storing them in the storage pool; retrieve all historical sparse vectors from the three batches, perform symbol-aware filtering to filter all cross-batch conflicting components, and globally aggregate to generate the final forgetting task vector; overlay the global baseline model. Scaling the vectors, the output is a compliant autonomous driving perception model with all overdue outsourced road tests cleared.

[0090] S5. Implementation Verification: The recognition rate of face, license plate, and exclusive road feature models in all road tests to be deleted over the past 15 years, 5 years, and this year is close to 0, with no recurrence of the situation; the recognition accuracy of normal general-purpose cars, pedestrians, and lane lines has decreased by less than 2%, and the vehicle-side autonomous driving following and avoidance functions are completely normal, successfully passing the Cyberspace Administration of China's special audit on automotive data compliance.

[0091] Implementation Scenario 2: Personal commuting data continuously forgotten when a ride-hailing platform owner cancels their registration.

[0092] A ride-hailing company equipped with an in-vehicle multimodal perception system received a continuous stream of requests from 2022 to 2025 for account cancellation and deletion of private data related to vehicle scrapping. These requests were scattered across different timeframes, creating a long-term, batch-wise forgetting process: first, requests from existing drivers in 2022 were received, followed by requests from new drivers in 2023, 2024, and 2025, processed in more than ten batches. The ride-hailing platform's cloud-based compliance backend deployed this invention's program for sequential forgetting.

[0093] S1. Initialize and load the full-scale urban commuting road test pre-trained RN50 multimodal benchmark model of the platform, and build a time-series deletion task queue for car owners.

[0094] S2. Retrieve the complete set of images, radar, and facial datasets of the bicycle owner's commutes over many years in batches according to the order of acceptance, and generate a sparse forgetting vector for each batch for long-term encrypted storage.

[0095] S3. For each new batch of vehicle owner vectors, the system automatically retrieves all historical vehicle owner sparse vectors from 2022 to the present, removes conflicting update components of old and new commuting scenarios on a parameter-by-parameter dimension, and uniformly aggregates the global forgotten vectors.

[0096] S4. After all vehicle owner deletion tasks are completed, the faces, exclusive license plates, and long-term commuting trajectories of all deregistered vehicle owners can no longer be recognized by the vehicle perception model; the platform's daily road recognition accuracy for ride-hailing vehicles, pedestrians, and traffic signs is stable without significant decline, there is no risk of privacy leakage, and it meets the GDPR compliance requirements for the platform's overseas operations.

[0097] This invention can cover four types of time-series batch cleanup business of vehicle manufacturers, and fully supports the real industry business time sequence of "processing old historical road test data for many years first, and then processing recent vehicle owner commuting data". The scenarios, forgotten objects and adapted systems correspond one by one.

[0098] This embodiment also provides an autonomous driving perception model forgetting system based on conflict avoidance task vectors, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-described method.

[0099] This embodiment also provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method.

[0100] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0101] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A forgetting method for an autonomous driving perception model based on conflict avoidance task vectors, characterized in that, The autonomous driving perception model is constructed by extending the vision-language pre-trained model, and the method includes: Step S1: Initialization of the cloud compliance platform and construction of the temporal forgetting task queue: Loading a fixed global pre-trained benchmark model. The global pre-trained baseline model weights are not modified throughout the process and serve as the unified parameter calculation benchmark for all forgetting tasks; temporal forgetting instructions are received and a temporal forgetting task queue is generated according to the order of instruction acceptance time; a new encrypted cloud-based historical task vector storage pool is created. ; Step S2: Generate autonomous driving-specific forgetting task vectors in batches: Retrieve single batches of multimodal road test datasets to be deleted in the order of queue time sequence. Forced to use a globally pre-trained benchmark model Using the initial weights as the sole basis for fine-tuning, a short-term fine-tuning is performed on the current batch dataset to obtain a temporarily fine-tuned model. ; Calculate the parameter offset difference between the temporary fine-tuned model and the globally pre-trained baseline model. Negate the parameter offset difference to generate the original forgetting task vector corresponding to this batch of road test data. ; Step S3: Sparse mask compression and encrypted storage of road test task vector: Sort all parameters of the original forgotten task vector by taking the absolute value, select the core perception parameters with a predetermined proportion before the amplitude to construct a binary mask, and obtain the sparse forgotten task vector by multiplying element by element; encrypt the sparse forgotten task vector and store it in the historical task vector storage pool. Step S4, Cross-Batch Conflict Avoidance Symbol-Aware Aggregation: Traverse all historical sparse task vectors in the storage pool, and perform conflict filtering and integration for each independent parameter dimension: Accumulate all historical sparse vector components under that parameter dimension, calculate the globally dominant forgetting direction for that dimension using a sign function; filter all valid vector components with signs consistent with the globally dominant forgetting direction, and discard conflicting components with opposite signs; take the average of the filtered conflict-free valid components to generate the globally aggregated forgetting task vector for the current time series. ; Step S5: Generate a compliant autonomous driving perception model based on a global benchmark: using a global pre-trained benchmark model Using the base vector as a superposition, and superimposing the scaled aggregated forgetting task vector, the compliance awareness model after processing the first t batches of time-series road test data is calculated: , where λ is the forgetting intensity scaling factor; Repeat steps S2 to S5 until all batches in the time-series forgetting task queue have completed forgetting processing.

2. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, Before each batch of road test datasets to be deleted is retrieved in step S2, it is extracted using uniform stratified sampling without replacement. The stratification dimensions include road year, road type, lighting time, vehicle type, and vehicle owner identity, so that the distribution of a single batch of samples is consistent with the distribution of the full road test data.

3. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, The autonomous driving perception model includes an on-board image encoding submodule, a lidar point cloud encoding submodule, a traffic text semantic encoding submodule, and a cross-modal feature fusion perception submodule. In step S2, the fine-tuning only updates the weights of the vehicle image encoding submodule and the lidar point cloud encoding submodule, while the traffic text semantic encoding submodule is locked and frozen throughout the process.

4. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 3, characterized in that, The autonomous driving perception model is compatible with four types of automotive-grade CLIP vision-language backbone networks: ViT-B / 32, ViT-L / 14, RN50, and RN101. The vehicle-mounted image encoding submodule outputs road image features with a dimension range of 256 to 2048, the lidar point cloud encoding submodule outputs three-dimensional perception features with a dimension range of 2048 to 8192, and the traffic text semantic encoding submodule outputs signage text features with a dimension range of 512 to 1024.

5. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, In step S4, the criterion for determining the conflicting components is: if the product of the components of any two different batches of sparse task vectors is less than 0 under the same parameter dimension, then it is determined to be an update conflicting component.

6. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, The time-sequenced forgetting instructions include: batch cleanup instructions for outsourced road test authorizations expiring, requests for deletion of private data for vehicle owner account cancellation or vehicle scrapping, regulatory vehicle data compliance rectification notices, and requests for cleanup of internal test vehicle projects that have been taken offline.

7. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, Each batch of multimodal road test datasets to be deleted contains RGB images of vehicle surround view, 3D point clouds of LiDAR, traffic sign text, driver faces, vehicle license plates, and paired samples of commuting trajectories.

8. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, The method is applicable to at least one of the following autonomous driving road test data cleaning business scenarios: Scenario 1: Overdue outsourced road tests continue to be forgotten - Batch delete images and point cloud road test samples collected by outsourced companies for highways and suburbs by year; Scenario 2: Forgotten time sequence of car owner's personal commuting data - delete privacy samples of face, license plate, and commuting trajectory collected by single user's in-vehicle camera and radar in batches according to the time sequence of car owner cancellation and vehicle scrapping; Scenario 3: Special Cleanup of Internal Test Vehicles - After the project is completed or the simulation test cycle ends, delete the multimodal samples collected from dedicated test roads and extreme working condition tests in batches; Scenario 4: Batch rectification and cleanup for regulatory compliance - destroying illegally collected existing road test data in multiple batches.

9. The forgetting method for autonomous driving perception models based on conflict avoidance task vectors according to claim 1, characterized in that, In step S2, the fine-tuning uses the AdamW optimizer; The method is configured with the following fixed hyperparameters: the fine-tuning learning rate is 1×10⁻⁶. -5 The weight decay coefficient is 0.1, the task vector sparsity retention ratio is 30%, the forgetting intensity scaling factor λ = 0.7, the batch sample size is 128, the single batch fine-tuning iteration is 10 rounds, and the gradient warm-up steps are 200.

10. A forgetting system for an autonomous driving perception model based on conflict avoidance task vectors, characterized in that, It includes a memory, a processor, and computer program instructions stored in the memory and executable by the processor, wherein when the processor executes the computer program instructions, it can implement the method as described in any one of claims 1-9.

Images