Model iteration method and device, electronic equipment and storage medium

Abstract

The application relates to a model iteration method and device, electronic equipment and a storage medium, comprising: obtaining original sensor data collected by a vehicle terminal in a preset time window before and after a trigger time in response to a preset event trigger; clustering the obtained multiple groups of original sensor data using an unsupervised clustering algorithm, identifying an outlier cluster from the clustering result, and marking the original sensor data corresponding to the outlier cluster as long-tail candidate data; extracting images from the long-tail candidate data, using a cloud perception model and a vehicle terminal perception model to respectively infer the images, screening out images with inconsistent inference results of the two perception models as difficult example images, and labeling the difficult example images according to the inference result of the cloud perception model; optimizing and training the vehicle terminal perception model using the labeled difficult example images, and deploying the optimized perception model to the vehicle terminal. Thus, the problem of insufficient performance of the perception model in a long-tail scene is effectively solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to a model iteration method, apparatus, electronic device, and storage medium. Background Technology

[0002] The iteration of current autonomous driving perception models heavily relies on the collection and annotation of real-world road data. Existing technologies sometimes compare the analysis results of lightweight on-vehicle models with high-precision cloud-based models. When the two are inconsistent or the confidence level of the on-vehicle model's output is too low, the current data is identified as a valuable long-tail scenario and saved. However, the selection logic of these solutions depends on differences in model output, resulting in a data pool containing a large number of redundant or low-value samples. This low data value density makes it difficult to support precise and targeted improvements in model capabilities.

[0003] Other technical approaches combine feature extraction with recognition models to filter long-tail data: first, multi-source sensor data is cleaned and features are extracted; then, a classification or recognition model is trained based on the extracted features; finally, this model is used to filter long-tail scenarios. The limitation of this method is that its performance is highly dependent on manually preset feature sets. Essentially, it uses human cognitive boundaries to define the machine's detection range, failing to identify entirely new and unknown risk patterns whose features exceed the predefined range. Furthermore, some solutions rely excessively on the confidence level or performance limitations of the vehicle-side perception model to be optimized, resulting in a "model capability trap": a model with perceptual defects struggles to accurately identify scenarios it is "unknown" or "cognitively flawed," and may completely fail during cold starts or when facing unknown domains.

[0004] In summary, existing technologies generally suffer from problems such as reliance on prior knowledge or the model's own capabilities in long-tail data mining, and a single value dimension. This leads to the perception model's performance being consistently insufficient in long-tail scenarios, making it difficult to effectively cope with rare and complex driving risks in real-world roads. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve them, this application provides a model iteration method, apparatus, electronic device, and storage medium.

[0006] Firstly, this application provides a model iteration method, the method comprising: Acquire raw sensor data collected by the vehicle in response to a preset event, within a preset time window before and after the trigger time; An unsupervised clustering algorithm is used to cluster the multiple sets of the original sensor data. Outlier clusters are identified based on the clustering results, and the original sensor data corresponding to the outlier clusters are marked as long-tail candidate data. Images are extracted from the long-tail candidate data, and the images are inferred using the cloud perception model and the vehicle perception model respectively. Images whose inference results from the two perception models are inconsistent are selected as difficult examples, and the difficult examples are labeled with the inference results of the cloud perception model. The vehicle-side perception model is optimized and trained using labeled difficult example images, and the optimized perception model is deployed to the vehicle.

[0007] In one possible implementation, the step of using an unsupervised clustering algorithm to cluster the acquired sets of original sensor data, and identifying outlier clusters based on the clustering results, includes: High-dimensional global feature extraction is performed on the raw sensor data within the preset time window to obtain the corresponding high-dimensional global feature vector. An unsupervised clustering algorithm is used to cluster the high-dimensional global feature vectors of the multiple sets of original sensor data. Noisy sample points and clusters with a sample number less than a preset minimum sample number in the clustering results are identified as outlier clusters.

[0008] In one possible implementation, the step of extracting high-dimensional global features from the raw sensor data within the preset time window to obtain a high-dimensional global feature vector includes: According to the preset sliding window length and sliding step size, the preset time window is divided into multiple sequentially adjacent sliding sub-windows; For each type of raw sensor data within each sliding sub-window, multiple statistical features are calculated, and the statistical features corresponding to all sliding sub-windows of the same type of raw sensor data are concatenated in chronological order to form a sub-feature matrix of the raw sensor data. The sub-feature matrices of all the original sensor data are concatenated to form the high-dimensional global feature vector.

[0009] In one possible implementation, the method further includes, before concatenating the sub-feature matrices of all the raw sensor data: Based on the preset event that triggers data acquisition, determine the weighting coefficient corresponding to each type of raw sensor data; Each element in the sub-feature matrix of the original sensor data is multiplied by the corresponding weighting coefficient.

[0010] In one possible implementation, selecting images whose inference results from two perceptual models are inconsistent as difficult example images includes: For each target category in each image, if the number of bounding boxes for that target category detected by the cloud-based perception model is not equal to the number of bounding boxes detected by the vehicle-side perception model, then the image is determined to be a difficult example image. If the number of images is equal, but the maximum intersection-union ratio of any cloud bounding box in the target category with all vehicle bounding boxes is less than a preset threshold, then the image is identified as a difficult example image.

[0011] In one possible implementation, the method further includes: Extract the signal features of the long-tail candidate data, and generate signal rules based on the signal features to describe the driving behavior patterns within the corresponding preset time window; Configure the signal rule as a newly added preset event on the vehicle side for subsequent data collection triggering.

[0012] In one possible implementation, after extracting images from the long-tailed candidate data, the method further includes: The extracted image is deredundant processed using at least one of the following methods: The perceptual hashing algorithm is used to calculate the hash value of each image, and only one image with the same hash value is kept. Monochrome block detection is performed on the image, and the area ratio of a single color region in the image is calculated. If the area ratio exceeds a preset ratio threshold, the image is removed.

[0013] In one possible implementation, before optimizing the training of the vehicle-side perception model using the labeled hard example images, the method further includes: The annotation results are pushed to the manual review interface; In response to manual correction instructions for the annotation results, the standard results are corrected.

[0014] In one possible implementation, the method further includes: After the optimized perception model is deployed to the vehicle, the steps of the method described in any one of claims 1-8 are continued to be executed to form a continuous closed-loop iteration of the vehicle perception model; In addition, the vehicle-side perception model is monitored and iterated to improve its perception performance indicators under the preset events. When the perception performance indicators improve beyond a preset threshold, the data collection trigger frequency corresponding to the preset events is reduced.

[0015] Secondly, this application provides a model iteration apparatus, the apparatus comprising: The data acquisition module is used to acquire raw sensor data collected by the vehicle in response to a preset event, within a preset time window before and after the trigger time; The long-tail identification module is used to cluster the acquired multiple sets of original sensor data using an unsupervised clustering algorithm, identify outlier clusters based on the clustering results, and mark the original sensor data corresponding to the outlier clusters as long-tail candidate data. The difficult example filtering module is used to extract images from the long-tail candidate data, use the cloud perception model and the vehicle perception model to reason about the images respectively, filter out images whose reasoning results of the two perception models are inconsistent as difficult example images, and label the difficult example images with the reasoning results of the cloud perception model. The model optimization module is used to optimize and train the vehicle-side perception model using labeled difficult example images, and then deploy the optimized perception model to the vehicle.

[0016] In one possible implementation, the long-tail recognition module includes: The feature extraction unit is used to perform high-dimensional global feature extraction on the raw sensor data within the preset time window to obtain the corresponding high-dimensional global feature vector. The clustering unit is used to cluster the high-dimensional global feature vectors of the multiple sets of original sensor data obtained by using an unsupervised clustering algorithm, and to identify the noisy sample points and the clusters with a sample number less than a preset minimum sample number in the clustering results as the outlier clusters.

[0017] In one possible implementation, the feature extraction unit includes: According to the preset sliding window length and sliding step size, the preset time window is divided into multiple sequentially adjacent sliding sub-windows; For each type of raw sensor data within each sliding sub-window, multiple statistical features are calculated, and the statistical features corresponding to all sliding sub-windows of the same type of raw sensor data are concatenated in chronological order to form a sub-feature matrix of the raw sensor data. The sub-feature matrices of all the original sensor data are concatenated to form the high-dimensional global feature vector.

[0018] In one possible implementation, the device further includes: The weighting module is used to determine the weighting coefficients corresponding to each type of original sensor data according to the preset event that triggers data acquisition before splicing the sub-feature matrices of all the original sensor data. Each element in the sub-feature matrix of the original sensor data is multiplied by the corresponding weighting coefficient.

[0019] In one possible implementation, the difficult example screening module is specifically used for: For each target category in each image, if the number of bounding boxes for that target category detected by the cloud-based perception model is not equal to the number of bounding boxes detected by the vehicle-side perception model, then the image is determined to be a difficult example image. If the number of images is equal, but the maximum intersection-union ratio of any cloud bounding box in the target category with all vehicle bounding boxes is less than a preset threshold, then the image is identified as a difficult example image.

[0020] In one possible implementation, the device further includes: The rule extraction module is used to extract the signal features of the long-tail candidate data and generate signal rules based on the signal features to describe the driving behavior patterns within the corresponding preset time window. Configure the signal rule as a newly added preset event on the vehicle side for subsequent data collection triggering.

[0021] In one possible implementation, the device further includes: The redundancy removal module is used to perform redundancy removal processing on the extracted images after extracting images from the long-tail candidate data, using at least one of the following methods: The perceptual hashing algorithm is used to calculate the hash value of each image, and only one image with the same hash value is kept. Monochrome block detection is performed on the image, and the area ratio of a single color region in the image is calculated. If the area ratio exceeds a preset ratio threshold, the image is removed.

[0022] In one possible implementation, before optimizing the vehicle-side perception model using the labeled hard example images, the device further includes: The manual review module is used to push the annotation results to the manual review interface; In response to manual correction instructions for the annotation results, the standard results are corrected.

[0023] In one possible implementation, the device further includes: The closed-loop iteration module is used to continue executing the steps in any of the methods in the first aspect after the optimized perception model is deployed to the vehicle, thereby forming a continuous closed-loop iteration of the vehicle perception model. An adaptive adjustment module is used to monitor the perception performance index of the vehicle-side perception model after iteration under the preset event. When the perception performance index improves beyond the preset threshold, the data collection trigger frequency corresponding to the preset event is reduced.

[0024] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the model iteration methods described in the first aspect.

[0025] Fourthly, this application provides a computer-readable storage medium storing a program for a model iteration method, wherein when the program for the model iteration method is executed by a processor, it implements the steps of the model iteration method described in any of the first aspects.

[0026] The technical solution provided in this application has the following beneficial effects: The technical solution provided in this application collects raw sensor data within a preset time window triggered by a preset event on the vehicle. An unsupervised clustering algorithm is then used to cluster multiple data sets to automatically identify outliers, thereby labeling data corresponding to long-tail scenarios as long-tail candidate data. This process does not rely on any preset scenario rules or model confidence levels, and can discover unknown rare patterns based on the statistical distribution of the data itself, overcoming the limitations of existing technologies that rely on prior knowledge or the model's own capabilities. Furthermore, images are extracted from the long-tail candidate data. The cloud model and the vehicle model are used to infer and filter images with inconsistent results as hard examples. The cloud model results are used for annotation, ensuring annotation quality by utilizing the high precision of the cloud model and automatically filtering hard examples through dual-model collaborative comparison, avoiding the high cost of full manual annotation. Finally, the annotated hard example images are used to perform targeted optimization training on the vehicle model and deployed back to the vehicle, forming a complete data-driven closed loop. Therefore, this solution can systematically solve the problem of insufficient performance of perception models in long-tail scenarios. Furthermore, each closed-loop iteration optimizes the real difficult cases discovered in actual road tests, enabling the vehicle-side model to continuously evolve with data accumulation. At the same time, it significantly reduces the reliance on manual annotation and prior knowledge, thereby improving the robustness and safety of the autonomous driving system in extreme and rare scenarios. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating an embodiment of a model iteration method provided in this application; Figure 2 A flowchart illustrating an embodiment of another model iteration method provided in this application; Figure 3A flowchart illustrating another embodiment of the model iteration method provided in this application; Figure 4 A block diagram illustrating an embodiment of a model iteration device provided in this application; Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Figure 1 This is a flowchart illustrating an embodiment of a model iteration method provided in this application. Figure 1 As shown, the method includes the following steps: Step 101: Obtain the raw sensor data collected by the vehicle in response to the triggering of a preset event, which is located within a preset time window before and after the triggering time.

[0032] Preset events refer to specific triggering conditions that occur during vehicle operation, including but not limited to driver takeover, system malfunction (such as sensor defocusing or computing unit overheating), serious accidents (such as collisions or rollovers), and sudden changes in road conditions (such as construction zones or animals crossing the road). These events often indicate that the current scenario has special characteristics or potential perception difficulties.

[0033] The preset time window is a time interval centered on the trigger time t0, taking a fixed duration forward and backward, denoted as [t0-Δt1, t0+Δt2]. Here, Δt1 and Δt2 are both, for example, 5 seconds each, or Δt1 is 8 seconds and Δt2 is 5 seconds. This window is used to collect continuous data before and after the preset event occurs, capturing the complete evolution of the event.

[0034] Raw sensor data consists of unprocessed signals directly from various vehicle sensors, including but not limited to camera images, LiDAR point clouds, millimeter-wave radar echoes, and GPS / IMU information. Preserving the originality avoids feature loss caused by preprocessing, providing maximum information for subsequent data mining. However, anonymization is necessary to protect privacy and security.

[0035] In step 101, when any preset event is triggered during vehicle operation, the vehicle-side system immediately records the current moment and automatically extracts all raw sensor data within a preset time window before and after that moment. This data, along with the timestamp and the type of the triggering event, is then encapsulated into a data packet associated with the preset event. This event-driven acquisition strategy transforms the data acquisition method from traditional "random sampling" or "full-coverage recording" to "targeted acquisition of key events," thereby ensuring that high-value extreme cases (i.e., long-tail scenarios) are not overlooked.

[0036] Subsequently, the vehicle-side system packages the raw sensor data collected within the preset time window and uploads it to the cloud. This allows the cloud to obtain the raw sensor data collected by the vehicle in response to a preset event, which is located within the preset time window before and after the trigger time, for subsequent cluster analysis and long-tail data mining.

[0037] Step 102: Use an unsupervised clustering algorithm to cluster the multiple sets of raw sensor data, identify outlier clusters based on the clustering results, and mark the raw sensor data corresponding to the outlier clusters as long-tail candidate data.

[0038] Unsupervised clustering algorithms are machine learning methods that can divide data into several "clusters" without labels, resulting in high similarity among samples within the same cluster and low similarity among samples in different clusters. Commonly used algorithms include density-based DBSCAN (Density-Based Spatial Clustering of Applications with Noise) and distance-based K-means.

[0039] Multiple sets of raw sensor data refer to multiple event data packets collected in step 101. Each data packet corresponds to a preset trigger event and the raw sensor data within the time window before and after it.

[0040] Outlier clusters refer to small clusters with extremely few samples and far distances from other clusters after clustering. The scenarios corresponding to these clusters occur very infrequently in the training data, making them typical candidates for the "long tail problem".

[0041] In step 102, the raw sensor data from the multiple event data packets uploaded to the cloud in step 101 are first converted into feature vectors. Then, an unsupervised clustering algorithm (such as DBSCAN) is used to cluster the massive feature vectors, automatically dividing the samples into several clusters. Among them, large-scale dense clusters representing common driving modes correspond to conventional scenarios, while noise points or small clusters representing rare modes, i.e., clusters with significantly fewer samples and far from the center of the mainstream clusters, are marked as "outlier clusters". The raw sensor data corresponding to these outlier clusters constitute a first-level long-tail candidate set, thereby filtering out abnormal scenarios with high potential risk.

[0042] This automatic filtering method eliminates the need for manual labeling of massive amounts of data to find long-tail scenarios, significantly improving mining efficiency and ensuring that high-value extreme cases are not overlooked.

[0043] Step 103: Extract images from long-tail candidate data, use cloud-based perception model and vehicle-side perception model to reason about the images respectively, select images whose reasoning results of the two perception models are inconsistent as difficult examples, and label the difficult examples with the reasoning results of cloud-based perception model.

[0044] Cloud-based perceptual models are high-performance large models deployed on cloud servers. They typically have deeper network structures, richer training data, and more powerful computing support, and their inference results are considered to be closer to reality (as teacher models).

[0045] The vehicle-side perception model is a lightweight model deployed on the vehicle's edge computing unit. Due to limitations in computing power and storage, its accuracy is typically lower than that of cloud-based models. This model is the target that this solution needs to continuously optimize.

[0046] Difficult example images refer to image samples where two models output inconsistent results for the same image. This means that the vehicle-side perception model has cognitive biases or blind spots in that scenario. Such samples are extremely valuable for improving the robustness of the model.

[0047] Labeling difficult example images with the inference results of cloud-based perception models means using the output of cloud-based perception models (such as target categories, bounding boxes, and semantic segmentation labels) as ground truth to automatically generate high-quality labeled data without human intervention.

[0048] In step 103, image frames are first extracted from the long-tail candidate data (if the sensor includes a camera, it is extracted directly; if it only includes LiDAR, the point cloud can be projected as an image or the original range image can be used). Then, the same image is fed into both the cloud-based perception model and the vehicle-side perception model for inference, and their outputs are compared. When inconsistencies are found (e.g., the cloud detects an obstacle while the vehicle does not, or the category predictions differ), the image is classified as a hard example, and the output of the cloud-based perception model is used as the label for automatic annotation. This step leverages the differences between models to accurately pinpoint the weaknesses of the vehicle-side perception model, while simultaneously utilizing the capabilities of the cloud-based perception model to achieve fully automatic annotation, forming a "model self-supervision" closed loop.

[0049] In one embodiment, an exemplary implementation of selecting images with inconsistent inference results from two perception models as difficult examples includes: for each target category in each image, if the number of bounding boxes detected by the cloud perception model for that target category is not equal to the number of bounding boxes detected by the vehicle perception model, then the image is determined as a difficult example image; if the number is equal but the maximum intersection-union ratio of any cloud bounding box under that target category with all vehicle bounding boxes is less than a preset threshold, then the image is determined as a difficult example image.

[0050] In this embodiment, the automatic difficult example screening step aims to automatically identify images with poor perception performance of the vehicle-side perception model through collaborative reasoning between the cloud-based perception model (high-quality perception model) and the vehicle-side perception model (lightweight perception model), and generate high-quality pseudo-labels for them using the results of the cloud-based perception model, for subsequent manual review and model optimization.

[0051] The specific implementation steps of this embodiment are as follows: The same image is fed into both a cloud-based perception model and a vehicle-based perception model to obtain their respective perception outputs. Due to its advantages in computing power and data, the output of the cloud-based perception model is treated as a pseudo-label (ground reference); the output of the vehicle-based perception model is a predicted bounding box.

[0052] For each target category in the image (such as vehicles, pedestrians, traffic signs, etc.), the following judgment logic is executed sequentially: Quantity comparison: First, compare the number of bounding boxes of this category detected by the cloud-based perception model with the number of bounding boxes of this category detected by the vehicle-side perception model. If the two are not equal, it indicates that the vehicle-side perception model has obvious missed detections (few detections) or false detections (many detections), and the image is directly identified as a hard case and included in the hard case candidate set.

[0053] Quality Comparison (When Quantities Are Equal): When the number of bounding boxes detected by the cloud-based perception model and the vehicle-based perception model is the same, the location matching degree is further verified. An Intersection over Union (IoU) threshold θ (e.g., 0.5) is set. For each pseudo-label bounding box output by the cloud-based perception model, its IoU value with all predicted bounding boxes from the vehicle-based perception model is calculated, and the maximum value is taken. If this maximum value is less than the threshold θ, it means that the vehicle-based perception model does not have a predicted bounding box that sufficiently overlaps with the pseudo-label at that location, i.e., there is a missed detection or localization error. This image is also classified as a hard case.

[0054] For any image in any category that meets any of the above conditions (unequal number or maximum IoU less than the threshold), the system automatically saves the pseudo-labels generated by the cloud perception model, the original prediction results of the vehicle perception model, and the difference report to the designated difficult example folder, thus completing the automatic screening and pre-labeling of difficult examples.

[0055] Through the above mechanism, it is possible to efficiently and accurately locate samples (i.e., difficult examples) that are difficult to perceive by the vehicle-side perception model, and use the output of the cloud-based perception model as the basis for automatic annotation, which greatly reduces the cost of manual annotation and provides high-quality training data for subsequent model optimization.

[0056] Step 104: Optimize and train the vehicle-side perception model using the labeled difficult example images, and deploy the optimized perception model to the vehicle.

[0057] The labeled difficult example images refer to the high-quality labeled data generated in step 103, where each image is labeled with the correct perception result.

[0058] Optimized training refers to adding difficult example images to the training set of the vehicle perception model by using incremental learning, fine-tuning, or retraining, and adjusting the model parameters to reduce the prediction error on difficult examples.

[0059] In step 104, the vehicle-side perception model is specifically optimized using the difficult example images and their automatic annotation results obtained in step 103. For example, since the amount of difficult example data is usually small, training can be conducted using a small-batch, multi-iteration approach, focusing on enabling the model to learn and correct its original erroneous perception judgments. After optimization, the new model is packaged and deployed to the vehicle through version management and security testing processes (e.g., via OTA (Over-The-Air) technology to replace the old model and activate its operation), providing real-time service to the autonomous driving or assisted driving system.

[0060] The technical solution provided in this application collects raw sensor data within a preset time window triggered by a preset event on the vehicle. An unsupervised clustering algorithm is then used to cluster multiple data sets to automatically identify outliers, thereby labeling data corresponding to long-tail scenarios as long-tail candidate data. This process does not rely on any preset scenario rules or model confidence levels, and can discover unknown rare patterns based on the statistical distribution of the data itself, overcoming the limitations of existing technologies that rely on prior knowledge or the model's own capabilities. Furthermore, images are extracted from the long-tail candidate data. The cloud model and the vehicle model are used to infer and filter images with inconsistent results as hard examples. The cloud model results are used for annotation, ensuring annotation quality by utilizing the high precision of the cloud model and automatically filtering hard examples through dual-model collaborative comparison, avoiding the high cost of full manual annotation. Finally, the annotated hard example images are used to perform targeted optimization training on the vehicle model and deployed back to the vehicle, forming a complete data-driven closed loop. Therefore, this solution can systematically solve the problem of insufficient performance of perception models in long-tail scenarios. Furthermore, each closed-loop iteration optimizes the real difficult cases discovered in actual road tests, enabling the vehicle-side model to continuously evolve with data accumulation. At the same time, it significantly reduces the reliance on manual annotation and prior knowledge, thereby improving the robustness and safety of the autonomous driving system in extreme and rare scenarios.

[0061] Figure 2 A flowchart illustrating an embodiment of another model iteration method provided in this application. Figure 2 The process shown is in Figure 1 Based on the illustrated process, an exemplary implementation is described that uses an unsupervised clustering algorithm to cluster multiple sets of raw sensor data, and identifies outlier clusters based on the clustering results. For example... Figure 2 As shown, it includes the following steps: Step 201: Extract high-dimensional global features from the raw sensor data within the preset time window to obtain the corresponding high-dimensional global feature vector.

[0062] "High-dimensional" means that the feature dimension is much larger than the original data volume or traditional handmade features; "global" means that the feature covers the entire time window, rather than a single moment or local segment. Specifically, a high-dimensional global feature vector refers to a feature vector with a high dimension (usually hundreds or thousands of dimensions) obtained by extracting and encoding all the original sensor data within the entire preset time window. This feature vector attempts to characterize the overall characteristics of the driving scene within the time window from a global perspective (such as vehicle motion patterns, changes in the surrounding environment, sensor response statistics, etc.), so as to be used for subsequent unsupervised clustering.

[0063] In one embodiment, an exemplary implementation of extracting high-dimensional global features from raw sensor data within a preset time window to obtain a high-dimensional global feature vector includes: dividing the preset time window into multiple sequentially adjacent sliding sub-windows according to a preset sliding window length and sliding step size; calculating multiple statistical features for each type of raw sensor data within each sliding sub-window, and concatenating the statistical features corresponding to all sliding sub-windows of the same type of raw sensor data in chronological order to form a sub-feature matrix of the raw sensor data; and concatenating the sub-feature matrices of all raw sensor data within a preset time window to form a high-dimensional global feature vector.

[0064] In one embodiment, before concatenating the sub-feature matrices of all the original sensor data, the method further includes: determining the weighting coefficient corresponding to each type of original sensor data according to a preset event that triggers data acquisition; and multiplying the elements in the sub-feature matrix of each type of original sensor data by the corresponding weighting coefficient.

[0065] In the above embodiments, the preset time window is first divided into multiple sequentially adjacent sliding sub-windows according to the set sliding window length and sliding step size. For example, the window length is 5 seconds and the sliding step size is 0.5 seconds. Adjacent sliding windows may have overlapping parts to maintain time continuity.

[0066] Then, for each type of raw sensor data (such as camera data, LiDAR point cloud, millimeter-wave radar echo, GPS / IMU information, etc.) within each sliding sub-window, multiple statistical features are calculated. For example, for the t-th sliding sub-window within the i-th signal, its mean μ_i(t), standard deviation σ_i(t), maximum value Max_i(t), minimum value Min_i(t), extreme value Range_i(t), linear fitting slope Slope_i(t), and approximate entropy Entropy_i(t) are calculated, which together constitute the sub-feature vector of the signal in the t-th sliding sub-window: f_i(t)=[μ_i(t),σ_i(t),Max_i(t),Min_i(t),Range_i(t),Slope_i(t),Entropy_i(t)].

[0067] Then, the sub-feature vectors calculated for the same signal across all sliding sub-windows are concatenated vertically in chronological order to form the sub-feature matrix of that signal:

[0068] Where w is the total number of sliding child windows.

[0069] Furthermore, to reflect the differences in the importance of various signals under different events (for example, in a "system malfunction" event, the weight of fault diagnosis-related signals may be higher than that of regular driving signals; while in a "driver takeover" event, the weight of steering wheel angle and brake pedal signals may be higher), a corresponding weighting coefficient can be assigned to each type of raw sensor data according to the preset event type that triggers data acquisition. The feature matrices of each signal are weighted and merged to obtain weighted sub-feature matrices. :

[0070] Finally, the weighted sub-feature matrices corresponding to all the original sensor data within the same preset time window are horizontally concatenated to form a high-dimensional global feature vector of the original sensor data within the preset time window.

[0071] The high-dimensional global feature vectors obtained through the above embodiments not only retain the local dynamic characteristics within the time window (by sequentially splicing through a sliding window), but also highlight the sensor information most relevant to specific events through weighting, providing a unified, compact, and discriminative input for subsequent unsupervised clustering.

[0072] Step 202: Use an unsupervised clustering algorithm to cluster the high-dimensional global feature vectors of the multiple sets of original sensor data. Identify the noisy sample points and clusters with a sample number less than the preset minimum sample number in the clustering results as outlier clusters.

[0073] Step 202 uses the high-dimensional global feature vectors of all event data obtained in step 201 as input and calls an unsupervised clustering algorithm (such as DBSCAN) to cluster them. After clustering, the number of samples in each cluster and the noise points that are not assigned to any cluster are analyzed. The specific identification rule is as follows: all sample points judged as noise by the clustering algorithm, as well as those clusters that form independent clusters but whose total number of samples is less than a preset minimum number of samples, are marked as "outlier clusters". The original sensor data corresponding to the outlier clusters are the long-tail candidate data.

[0074] This mechanism does not rely on manual labeling. By automatically detecting sparsely distributed, small-scale, or isolated samples, it quickly filters out abnormal scenarios that are significantly different from mainstream driving modes, effectively narrowing the scope of subsequent difficult case mining and improving overall processing efficiency.

[0075] Figure 2 The process shown is in Figure 1Based on the illustrated process, high-dimensional global feature extraction transforms multi-source sensor data within a preset time window into a unified high-dimensional feature vector. An unsupervised clustering algorithm is then used to automatically identify outlier clusters, thereby achieving efficient screening of long-tail candidate data. This approach eliminates the need for manually defined scene rules, prior features, or confidence outputs from vehicle-side models. It directly discovers rare patterns based on the statistical distribution of the data itself, avoiding the problem of traditional methods missing unknown risk scenarios due to insufficient model capabilities or cognitive limitations.

[0076] Figure 3 A flowchart illustrating another embodiment of the model iteration method provided in this application. Figure 3 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: Step 301: Obtain the raw sensor data collected by the vehicle in response to the triggering of a preset event, which is located within a preset time window before and after the triggering time.

[0077] Step 302: Use an unsupervised clustering algorithm to cluster the multiple sets of raw sensor data, identify outlier clusters based on the clustering results, and mark the raw sensor data corresponding to the outlier clusters as long-tail candidate data.

[0078] For a detailed description of steps 301 and 302, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0079] Step 303: Extract images from the long-tail candidate data and perform redundancy removal on the extracted images.

[0080] Step 303 aims to convert the video information in the raw sensor data uploaded by the vehicle into image frames that can be processed by the subsequent perception model, and to remove redundant and low-quality images to reduce the consumption of cloud storage and computing resources.

[0081] First, the video stream captured by the camera is extracted from the raw sensor data that has been labeled as long-tail candidate data. To automate the conversion of the video stream into an image sequence, a dedicated Docker image is constructed in this embodiment. This image integrates the ROS (Robot Operating System) environment, the zplayer tool, and necessary dependency libraries, and encapsulates a script. This script can automatically read video files, perform video stream extraction and frame decomposition, and output image folders named according to timestamps. In this way, a continuous video stream can be converted into a discrete image sequence for subsequent analysis.

[0082] Secondly, the above image sequence undergoes redundancy removal processing. Since there are often many continuous static or minimally changing elements between adjacent frames in a driving scene, directly processing all frames would generate a large amount of redundant data, wasting storage and computing resources. Therefore, in one embodiment, at least one of the following two algorithms is used to filter the images: The perceptual hashing algorithm is used to calculate the hash value of each image, and only one image with the same hash value is kept. Monochrome block detection is performed on the image, and the area ratio of a single color region in the image is calculated. If the area ratio exceeds a preset ratio threshold, the image is removed.

[0083] Among them, the perceptual hashing algorithm is not sensitive to subtle changes in images (such as brightness, translation, etc.) and can quickly identify redundant frames with essentially the same content.

[0084] As an optional implementation, for images retained by the perceptual hashing algorithm, monochrome block detection is further performed. The ratio of the area of ​​a single-color region (e.g., pure black, pure white, or a single hue) in the image to the total area of ​​the entire image is calculated. If this ratio exceeds a preset threshold (e.g., 90%), the image is determined to be a low-quality image (e.g., scenes where the camera is obstructed, the lens is damaged, or there is no lighting at night), and it is removed from the dataset.

[0085] After the above redundancy removal process, the amount of remaining image data is significantly reduced, but key change information in the scene is preserved. These images will then be used in subsequent model collaborative inference and hard example selection stages.

[0086] Step 304: Use the cloud-based perception model and the vehicle-side perception model to reason about the images respectively, select images whose reasoning results from the two perception models are inconsistent as difficult examples, and label the difficult examples with the reasoning results of the cloud-based perception model.

[0087] Step 305: Push the annotation results to the manual review interface; respond to the manual correction instructions for the annotation results and correct the standard results.

[0088] Step 305 aims to manually review the pre-labeled results generated by the aforementioned model collaborative annotation in order to eliminate possible biases or errors in automatic annotation, thereby obtaining a high-quality gold standard dataset and providing reliable ground truth for subsequent model training.

[0089] In one specific embodiment, this step is implemented as follows: First, the difficult example images selected in step 304, along with their corresponding cloud-based model pre-annotation results (including the bounding box position, category label, and confidence score for each target), are pushed to a manual review interface. This interface can be a web-based annotation platform with intuitive image display, overlay display of annotation results, and interactive editing functions. Reviewers can view each image individually on the interface and see the overlaid pre-annotated boxes.

[0090] Secondly, in response to the correction instructions issued by the reviewers, the pre-annotation results are corrected accordingly. Correction instructions include, but are not limited to: Boundary box adjustment: When reviewers find that the pre-annotated box deviates from the actual target outline, they can adjust the position and size of the boundary box by dragging, scaling or redrawing it with the mouse.

[0091] Category Correction: If the pre-labeled category is incorrect (e.g., "pedestrian" is labeled as "cyclist"), the reviewer can select the correct category from the drop-down menu.

[0092] Adding missed targets: If the pre-annotation misses targets in the image, the reviewer can manually add new bounding boxes and specify the category.

[0093] Delete falsely identified targets: If a non-target (such as a shadow or sign) is incorrectly labeled as a target, the reviewer can delete the label box.

[0094] The system captures the actions of the reviewers in real time and updates the corresponding image annotations based on the instructions. After the review is completed, these manually corrected annotations are saved as the final gold standard dataset.

[0095] The aforementioned manual review mechanism not only compensates for the shortcomings of purely automatic annotation in rare and complex scenarios, ensuring the quality of training data, but also significantly reduces the cost and time of fully manual annotation because pre-annotation has been largely completed by large cloud models, requiring only manual review and fine-tuning.

[0096] Step 306: Optimize and train the vehicle-side perception model using the labeled difficult example images, and deploy the optimized perception model to the vehicle; return to step 301 to form a continuous closed-loop iteration of the vehicle-side perception model.

[0097] In step 306, the original perception model on the vehicle is incrementally optimized using high-quality labeled data (Gold Standard dataset) that has been manually verified, and the optimized model is redeployed to the vehicle to achieve continuous model evolution. At the same time, through a closed-loop feedback mechanism, the system can gradually improve its perception capability for long-tail scenarios through multiple iterations.

[0098] In one specific embodiment, the implementation process of this step includes the following sub-steps: Step 1: Incremental Model Training The gold-standard dataset obtained in step 305 (containing difficult example images and their corresponding precise annotations) will be used as training samples to incrementally train the perception model currently deployed on the vehicle. In one embodiment, to avoid catastrophic forgetting (i.e., new data causing a decline in the model's ability to recognize old scenes), the following strategy is adopted: Mixed sampling: The gold standard dataset is mixed with the original regular training dataset at a preset ratio (e.g., 1:3) to construct a small batch of training samples.

[0099] Transfer learning: Load the existing weights of the vehicle-side model as initial parameters and fine-tune the model with a low learning rate (e.g., 1 / 10 of the original learning rate). Depending on the specific situation, the first few layers of the model (used to extract general features) can be frozen, and only the parameters of the last few layers (used for specific category classification and regression) can be updated.

[0100] Class balance: When there is a large difference in the number of samples of different classes in the gold standard data, a weighted loss function or oversampling strategy can be used to prevent the model from favoring the class with more samples.

[0101] Early stopping strategy: When the performance on the validation set no longer improves or the improvement is less than a preset threshold, training is terminated early to avoid overfitting.

[0102] After training, an optimized vehicle-side perception model is obtained. This model, based on its original capabilities, has stronger robustness in recognizing long-tail scenarios (such as difficult examples corresponding to outlier clusters) identified in step 302.

[0103] Step 2, Model Deployment The trained and optimized perception model is deployed to the vehicle via OTA (Over-The-Air) technology. Specifically, the cloud platform packages the model parameter file (or the complete model file) and pushes it to the vehicle network via an encrypted channel. Upon receiving the file, the vehicle performs integrity verification and decryption, replacing the original perception model file. The model loading module restarts or dynamically loads the new model to ensure that the optimized perception capabilities are used in subsequent vehicle operations.

[0104] Finally, return to step 301 to form a continuous closed-loop iteration: After the model is deployed, the system does not stop working, but automatically returns to step 301 to continue the entire process of data collection triggered by preset events on the vehicle, long-tail mining in the cloud, annotation training and redeployment.

[0105] By repeatedly executing the above loop, each iteration generates corresponding hard case data for newly discovered long-tail scenarios and optimizes the vehicle-side model accordingly. As the number of iterations increases, the coverage and recognition accuracy of the vehicle-side perception model for various long-tail scenarios gradually improves, and the system performance tends to saturate, thus achieving self-evolution of the driver assistance perception model.

[0106] also, Figure 3 The closed-loop process shown also includes an effect monitoring sub-step: monitoring the perception performance index of the iterated vehicle-side perception model under a preset event; when the perception performance index improves beyond a preset threshold, reducing the data collection trigger frequency corresponding to that preset event. In a specific embodiment, this effect monitoring sub-step is implemented as follows: After deploying the optimized perception model to the vehicle, the system continuously collects perception output data when the vehicle encounters preset events during actual driving (such as driver intervention, system malfunctions, collisions, etc.). For each type of preset event, the system statistically analyzes key performance indicators of the vehicle-side perception model within the time window before and after the event trigger, including but not limited to: Detection accuracy: The proportion of target categories (vehicles, pedestrians, obstacles, etc.) that the model correctly identifies; Detection rate (recall rate): The proportion of actual targets that are correctly detected by the model; Mean Intersection over Union (mIoU): The degree of overlap between the detection bounding box and the ground truth bounding box; Confidence distribution: The mean and variance of the confidence scores of the model output reflect the determinism of the model.

[0107] The above indicators can be obtained by comparing the perception results uploaded by the vehicle with the "gold standard true values" (from historical verification data or high-precision offline models) stored in the cloud, or they can be indirectly estimated through unsupervised methods (such as multi-sensor consistency verification).

[0108] For each type of preset event, a performance improvement threshold is preset (e.g., accuracy improvement of less than 2% or recall rate stable above 95%). When the perceived performance index corresponding to a certain type of event is found to be lower than the threshold in multiple consecutive iterations (e.g., 3 times), it is determined that the model capability for that type of event has reached saturation, meaning that the marginal contribution of continuing to collect data for this type of scenario to the model performance is very low.

[0109] Based on the performance saturation determination result, the system automatically adjusts the vehicle's triggering strategy for this type of preset event: Reduce trigger frequency: Lower the trigger sensitivity of the corresponding event. For example, change the mode of uploading data every time a takeover event is encountered to uploading according to the sampling rate (such as uploading only 10% of the events), or directly close the upload channel for this type of event.

[0110] Resource reallocation: Transfer the freed computing and storage resources to other pre-defined events that have not yet reached performance saturation (especially long-tail scenarios that are still difficult to improve after multiple iterations), increase the data acquisition density of these scenarios, so that the system can focus on tackling the "hardest challenges".

[0111] Furthermore, even if a certain type of event is down-produced, the system continues to monitor the model's performance in that scenario at a lower sampling frequency (e.g., uploading only once out of every 100 events). If a performance regression is detected (e.g., an accuracy drop of more than 5%), the system can automatically restore the standard triggering frequency for that type of event and restart data collection and iteration.

[0112] Through the aforementioned effect monitoring sub-steps, this application achieves the self-awareness and adaptive scheduling capabilities of the closed-loop system, which avoids wasting resources in high-performance scenarios and ensures that bottleneck scenarios receive sufficient attention, thereby achieving the greatest overall performance gain with minimal resource cost.

[0113] Figure 3 The process shown presents a complete closed-loop iterative path from data acquisition, long-tail mining, image deduplication, model collaborative annotation, manual review to incremental model training and OTA deployment, providing a complete, efficient, and scalable solution for the continuous self-evolution of autonomous driving perception models in real long-tail scenarios.

[0114] Finally, in one embodiment, the method provided in this application further includes: extracting signal features from long-tail candidate data, and generating signal rules based on the signal features to describe the driving behavior pattern within the corresponding preset time window; configuring the signal rules as preset events added to the vehicle for subsequent data collection triggering.

[0115] Specifically, for the preset time window (i.e. the original data segment identified as an outlier cluster) marked by the unsupervised clustering algorithm as long-tail candidate data in the aforementioned steps, the original values ​​and status information of all sensor signals (such as vehicle speed, acceleration, brake pedal opening, steering wheel angle, yaw rate, gear information, driver takeover signals, etc.) within the window are extracted, and readable signal rules are generated by manually or semi-automatically summarizing them.

[0116] For example, for an abnormal driving pattern discovered through clustering, logical combination rules such as "emergency braking signal activated with absolute deceleration > 0.5g, and steering wheel angle change rate > 100° / s" can be generated, transforming the "black box" pattern discovered by the machine into a chain of signal evidence that engineers can understand and verify. Subsequently, these signal rules are configured as newly added preset events on the vehicle side, and the vehicle status is monitored in real time through the rule engine; when the vehicle-side sensor signals meet the rule conditions, a new data acquisition is triggered, and the original sensor data before and after the trigger time are packaged and uploaded in the same way as the original preset events.

[0117] In this way, on the one hand, the results of cluster mining are solidified into triggering rules that can be used in the future, and on the other hand, a closed-loop enhancement is achieved from "passively waiting for events" to "actively mining and predicting long-tail scenarios", enabling the system to continuously self-evolve and cover a wider range of driving risk patterns.

[0118] Figure 4 This is a block diagram illustrating an embodiment of a model iteration device provided in this application. Figure 4 As shown, the device includes: Data acquisition module 41 is used to acquire raw sensor data collected by the vehicle in response to a preset event, which is located within a preset time window before and after the trigger time; The long-tail identification module 42 is used to cluster the acquired multiple sets of original sensor data using an unsupervised clustering algorithm, identify outlier clusters based on the clustering results, and mark the original sensor data corresponding to the outlier clusters as long-tail candidate data. The difficult example filtering module 43 is used to extract images from the long-tail candidate data, use the cloud perception model and the vehicle perception model to reason about the images respectively, filter out images whose reasoning results of the two perception models are inconsistent as difficult example images, and label the difficult example images with the reasoning results of the cloud perception model. The model optimization module 44 is used to optimize and train the vehicle-side perception model using labeled difficult example images, and then deploy the optimized perception model to the vehicle.

[0119] In one possible implementation, the long-tail recognition module 42 includes: The feature extraction unit is used to perform high-dimensional global feature extraction on the raw sensor data within the preset time window to obtain the corresponding high-dimensional global feature vector. The clustering unit is used to cluster the high-dimensional global feature vectors of the multiple sets of original sensor data obtained by using an unsupervised clustering algorithm, and to identify the noisy sample points and the clusters with a sample number less than a preset minimum sample number in the clustering results as the outlier clusters.

[0120] In one possible implementation, the feature extraction unit includes: According to the preset sliding window length and sliding step size, the preset time window is divided into multiple sequentially adjacent sliding sub-windows; For each type of raw sensor data within each sliding sub-window, multiple statistical features are calculated, and the statistical features corresponding to all sliding sub-windows of the same type of raw sensor data are concatenated in chronological order to form a sub-feature matrix of the raw sensor data. The sub-feature matrices of all the original sensor data are concatenated to form the high-dimensional global feature vector.

[0121] In one possible implementation, the device further includes: The weighting module is used to determine the weighting coefficients corresponding to each type of original sensor data according to the preset event that triggers data acquisition before splicing the sub-feature matrices of all the original sensor data. Each element in the sub-feature matrix of the original sensor data is multiplied by the corresponding weighting coefficient.

[0122] In one possible implementation, the difficult example screening module 43 is specifically used for: For each target category in each image, if the number of bounding boxes for that target category detected by the cloud-based perception model is not equal to the number of bounding boxes detected by the vehicle-side perception model, then the image is determined to be a difficult example image. If the number of images is equal, but the maximum intersection-union ratio of any cloud bounding box in the target category with all vehicle bounding boxes is less than a preset threshold, then the image is identified as a difficult example image.

[0123] In one possible implementation, the device further includes: The rule extraction module is used to extract the signal features of the long-tail candidate data and generate signal rules based on the signal features to describe the driving behavior patterns within the corresponding preset time window. Configure the signal rule as a newly added preset event on the vehicle side for subsequent data collection triggering.

[0124] In one possible implementation, the device further includes: The redundancy removal module is used to perform redundancy removal processing on the extracted images after extracting images from the long-tail candidate data, using at least one of the following methods: The perceptual hashing algorithm is used to calculate the hash value of each image, and only one image with the same hash value is kept. Monochrome block detection is performed on the image, and the area ratio of a single color region in the image is calculated. If the area ratio exceeds a preset ratio threshold, the image is removed.

[0125] In one possible implementation, before optimizing the vehicle-side perception model using the labeled hard example images, the device further includes: The manual review module is used to push the annotation results to the manual review interface; In response to manual correction instructions for the annotation results, the standard results are corrected.

[0126] In one possible implementation, the device further includes: The closed-loop iteration module is used to continue executing the steps in any of the methods in the first aspect after the optimized perception model is deployed to the vehicle, thereby forming a continuous closed-loop iteration of the vehicle perception model. An adaptive adjustment module is used to monitor the perception performance index of the vehicle-side perception model after iteration under the preset event. When the perception performance index improves beyond the preset threshold, the data collection trigger frequency corresponding to the preset event is reduced.

[0127] In another embodiment of this application, an electronic device is also provided, such as... Figure 5 As shown, it includes a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. Memory 1130 is used to store computer programs; The processor 1110 is used to implement the model iteration method described in any of the foregoing method embodiments when executing a program stored in the memory.

[0128] The electronic device provided in this application embodiment effectively solves the problem of insufficient performance of the perception model in long-tail scenarios by executing the program stored in the memory. Moreover, each closed-loop iteration is optimized for real difficult cases mined in actual road tests, so that the capabilities of the vehicle-side model can continuously self-evolve with the accumulation of data. At the same time, it significantly reduces the dependence on manual annotation and prior knowledge, and improves the robustness and safety of the autonomous driving system in extreme and rare scenarios.

[0129] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0130] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.

[0131] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0132] The processor 1110 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0133] In another embodiment of this application, a computer-readable storage medium is also provided, on which a program for a model iteration method is stored, wherein when the program for the model iteration method is executed by a processor, the program implements the steps of the model iteration method described in any of the foregoing method embodiments.

[0134] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A model iteration method, characterized in that, The method includes: Acquire raw sensor data collected by the vehicle in response to a preset event, within a preset time window before and after the trigger time; An unsupervised clustering algorithm is used to cluster the multiple sets of the original sensor data. Outlier clusters are identified based on the clustering results, and the original sensor data corresponding to the outlier clusters are marked as long-tail candidate data. Images are extracted from the long-tail candidate data, and the images are inferred using the cloud perception model and the vehicle perception model respectively. Images whose inference results from the two perception models are inconsistent are selected as difficult examples, and the difficult examples are labeled with the inference results of the cloud perception model. The vehicle-side perception model is optimized and trained using labeled difficult example images, and the optimized perception model is deployed to the vehicle.

2. The method according to claim 1, characterized in that, The step of using an unsupervised clustering algorithm to cluster the acquired sets of original sensor data, and identifying outlier clusters based on the clustering results, includes: High-dimensional global feature extraction is performed on the raw sensor data within the preset time window to obtain the corresponding high-dimensional global feature vector. An unsupervised clustering algorithm is used to cluster the high-dimensional global feature vectors of the multiple sets of original sensor data. Noisy sample points and clusters with a sample number less than a preset minimum sample number in the clustering results are identified as outlier clusters.

3. The method according to claim 2, characterized in that, The step of extracting high-dimensional global features from the raw sensor data within the preset time window to obtain a high-dimensional global feature vector includes: According to the preset sliding window length and sliding step size, the preset time window is divided into multiple sequentially adjacent sliding sub-windows; For each type of raw sensor data within each sliding sub-window, multiple statistical features are calculated, and the statistical features corresponding to all sliding sub-windows of the same type of raw sensor data are concatenated in chronological order to form a sub-feature matrix of the raw sensor data. The sub-feature matrices of all the original sensor data are concatenated to form the high-dimensional global feature vector.

4. The method according to claim 3, characterized in that, Before concatenating the sub-feature matrices of all the original sensor data, the method further includes: Based on the preset event that triggers data acquisition, determine the weighting coefficient corresponding to each type of raw sensor data; Each element in the sub-feature matrix of the original sensor data is multiplied by the corresponding weighting coefficient.

5. The method according to claim 1, characterized in that, The selection of images whose inference results from two perceptual models are inconsistent as difficult example images includes: For each target category in each image, if the number of bounding boxes for that target category detected by the cloud-based perception model is not equal to the number of bounding boxes detected by the vehicle-side perception model, then the image is determined to be a difficult example image. If the number of images is equal but the maximum intersection-union ratio of any cloud bounding box in the target category with all vehicle bounding boxes is less than a preset threshold, then the image is identified as a difficult example image.

6. The method according to claim 1, characterized in that, The method further includes: Extract the signal features of the long-tail candidate data, and generate signal rules based on the signal features to describe the driving behavior patterns within the corresponding preset time window; Configure the signal rule as a newly added preset event on the vehicle side for subsequent data collection triggering.

7. The method according to claim 1, characterized in that, After extracting images from the long-tailed candidate data, the method further includes: The extracted image is deredundant processed using at least one of the following methods: The perceptual hashing algorithm is used to calculate the hash value of each image, and only one image with the same hash value is kept. Monochrome block detection is performed on the image, and the area ratio of a single color region in the image is calculated. If the area ratio exceeds a preset ratio threshold, the image is removed.

8. The method according to claim 1, characterized in that, Before optimizing and training the vehicle-side perception model using the labeled hard example images, the method further includes: The annotation results are pushed to the manual review interface; In response to manual correction instructions for the annotation results, the standard results are corrected.

9. The method according to claim 1, characterized in that, The method further includes: After the optimized perception model is deployed to the vehicle, the steps of the method described in any one of claims 1-8 are continued to be executed to form a continuous closed-loop iteration of the vehicle perception model; In addition, the vehicle-side perception model is monitored and iterated to improve its perception performance indicators under the preset events. When the perception performance indicators improve beyond a preset threshold, the data collection trigger frequency corresponding to the preset events is reduced.

10. A model iteration device, characterized in that, The device includes: The data acquisition module is used to acquire raw sensor data collected by the vehicle in response to a preset event, within a preset time window before and after the trigger time; The long-tail identification module is used to cluster the acquired multiple sets of original sensor data using an unsupervised clustering algorithm, identify outlier clusters based on the clustering results, and mark the original sensor data corresponding to the outlier clusters as long-tail candidate data. The difficult example filtering module is used to extract images from the long-tail candidate data, use the cloud perception model and the vehicle perception model to reason about the images respectively, filter out images whose reasoning results of the two perception models are inconsistent as difficult example images, and label the difficult example images with the reasoning results of the cloud perception model. The model optimization module is used to optimize and train the vehicle-side perception model using labeled difficult example images, and then deploy the optimized perception model to the vehicle.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the model iteration method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for a model iteration method, which, when executed by a processor, implements the steps of the model iteration method according to any one of claims 1-9.

Images