An automatic driving data storage optimization method, device and system

Abstract

The application provides a kind of automatic driving data storage optimization method, device and system, the method includes collecting the multimodal data of vehicle, scene perception feature and event information;Evaluate the scene value and event trigger probability of vehicle in each data segment;According to the scene value, event trigger probability, compliance constraint and the resource state of vehicle, generate a candidate strategy set, and filter out the optimal strategy;Optimization processing and transmission are carried out to the optimal strategy, and difference synchronization and strategy closed loop update are carried out in network interruption and recovery scene;Solved at least one of the following problems existing in the storage and upload of the existing automatic driving data: extensive resource allocation, low-value data squeezing bandwidth leading to key evidence loss;Depend on hard threshold, cannot cover potential high-risk fragments;Multi-modal independent coding, high redundancy, poor compression effect;Compliance rules solidification, difficult to adapt across regions;Weak network transmission has no priority, backhaul is unreliable;Strategy has no resource closed loop, system efficiency is unstable.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a method, apparatus, and system for optimizing the storage of autonomous driving data. Background Technology

[0002] As autonomous driving technology evolves from assisted driving to highly automated driving, onboard systems need to collect, process, and transmit massive amounts of multi-source heterogeneous perception data in real time, including video, point cloud, milli-radar, CAN bus data, and driving logs. This data serves as the input for autonomous driving decision-making and control, as well as core digital evidence for accident retrospective analysis, risk analysis, algorithm iteration, and compliance evidence collection.

[0003] Currently, most mainstream solutions for autonomous driving data storage and uploading in the industry adopt fixed configurations and event-driven models. For example, fixed bit rates and fixed sampling rates are commonly used for video, point clouds, and radar data, with the acquisition quality only temporarily improved or caching enabled after hard threshold events such as collisions or sudden deceleration are triggered. Data retention strategies are mostly based on vehicle models or regions, with retention duration and quality limits set uniformly, without distinguishing between scenario complexity and risk levels. Each perception modality is compressed and encoded independently, without cross-modal information collaboration.

[0004] However, existing technologies for storing and uploading autonomous driving data suffer from at least one of the following problems: inefficient resource allocation, low-value data crowding out bandwidth and causing the loss of critical evidence; reliance on hard thresholds, failing to cover potentially high-risk segments; multimodal independent encoding, resulting in high redundancy and poor compression; rigid compliance rules, making cross-regional adaptation difficult; weak network transmission lacks priority, leading to unreliable backhaul; and the lack of resource closure in the strategy results in unstable system performance. Summary of the Invention

[0005] The technical problem solved by this invention is that the storage and uploading of autonomous driving data in the prior art suffers from at least one of the following issues: inefficient resource allocation, low-value data crowding out bandwidth and causing the loss of key evidence; reliance on hard thresholds, which cannot cover potentially high-risk segments; multimodal independent encoding, resulting in high redundancy and poor compression effect; rigid compliance rules, making cross-regional adaptation difficult; weak network transmission without priority, leading to unreliable backhaul; and lack of resource closure in the strategy, resulting in unstable system performance.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a method for optimizing the storage of autonomous driving data, the method comprising:

[0007] Collect multimodal data of vehicles and extract scene perception features and event information of vehicles;

[0008] Based on scene perception features and event information, the scene value and event trigger probability of the vehicle in each data segment are evaluated;

[0009] Load a pre-defined compliance template to determine compliance constraints for data processing;

[0010] Based on scenario value, event trigger probability, compliance constraints, and vehicle resource status, a set of candidate strategies that conform to multimodal data is generated.

[0011] Based on the vehicle's current mode, the optimal strategy is selected from the candidate strategy set;

[0012] The system performs optimized processing and transmission of the optimal strategy, and performs differential synchronization and closed-loop policy updates in network interruption and recovery scenarios.

[0013] Compared with existing technologies, the technical effects achieved by this solution are as follows: This solution extracts scene perception features and event information from vehicle multimodal data, evaluates the scene value and event triggering probability of data segments, combines compliance constraints of compliance templates with real-time vehicle resource status, generates and filters the optimal strategy suitable for the current modality, and then performs targeted optimization processing and transmission. With the addition of differential synchronization and strategy closed-loop update mechanisms after network interruption recovery, it achieves fine-grained allocation of resources for autonomous driving data storage and uploading. This not only breaks through the limitations of hard threshold triggering and achieves effective coverage of potentially high-risk segments, but also abandons the multimodal independent encoding mode, reducing data redundancy and improving compression effect.

[0014] Furthermore, this solution achieves flexible adaptation of compliance rules through pre-set compliance templates, reducing the adaptation costs across regions and businesses. It also improves the reliability of data backhaul in weak network scenarios by relying on priority transmission and differential synchronization. Moreover, through dynamic closed-loop updates of resource status and policies, it avoids system performance fluctuations and ensures stable system operation. Thus, with limited in-vehicle storage and network bandwidth resources, it solves the problems of extensive resource allocation, easy loss of key evidence, high redundancy of multimodal data, difficulty in compliance adaptation, unreliable transmission in weak networks, and unstable system performance in existing technologies. It improves the retention quality and transmission reliability of key data, taking into account the intelligence, refinement, compliance, and stability of autonomous driving data processing, and enhances the overall performance of the autonomous driving data storage and upload system.

[0015] In one embodiment of the present invention, performing optimization processing and transmission of the optimal strategy includes:

[0016] Perform adaptive coding and cross-modal redundancy removal according to the optimal strategy;

[0017] According to the priority sequence determined by the optimal strategy, the optimization results of adaptive coding and cross-modal redundancy removal are placed into different transmission queues;

[0018] Based on real-time network feedback, select the corresponding data segments for transmission according to priority.

[0019] In the event of network congestion or weak network conditions, the transmission priority of event segments and high-value segments is automatically increased, while low-value segments are proactively discarded or downgraded.

[0020] Compared with existing technologies, the technical effects achieved by this solution are as follows: This solution performs adaptive coding and cross-modal redundancy removal on multimodal data through optimal strategies, realizing collaborative optimization processing of multimodal data. It abandons the independent coding mode of multimodal data in existing technologies, eliminates information redundancy between modalities, and reduces the coding resource overhead of accurately matching each data segment (any data segment in multimodal data). It also reduces the data storage and transmission volume while ensuring the fidelity of key data.

[0021] Furthermore, the optimized data is divided into transmission queues according to the data priority sequence specified by the optimal strategy, and data is sent according to data priority based on real-time network feedback. At the same time, in the case of network congestion or weak network conditions, the transmission priority of event fragments and high-value fragments is increased in a targeted manner, while low-value fragments are actively discarded or downgraded. This changes the current situation in existing technologies where data transmission lacks fine-grained priority and there is no differentiated processing in weak network conditions. It avoids low-value data crowding out limited network bandwidth, ensures that event and high-value key data are transmitted preferentially in complex network scenarios, improves the reliability of data transmission and the efficiency of network bandwidth utilization, and achieves fine-grained management of resources from the two dimensions of data processing and transmission scheduling. It solves the problems of high redundancy of multimodal data, coarse allocation of bandwidth resources, and easy loss of key evidence in weak network transmission in existing technologies, and ensures the efficient processing and priority backhaul of key data for autonomous driving.

[0022] In one embodiment of the present invention, selecting the optimal strategy from the candidate strategy set includes:

[0023] Solve for the maximum value integral under resource state constraints and output the optimization results for multimodal data;

[0024] The objective function for maximizing the value integral is:

[0025] Formula 1: ;

[0026] The constraints on resource status are:

[0027] Formula 2:

[0028] ;

[0029] In the formula, Indicates the maximum value. Indicates the value of a scenario. Indicates the probability of the event being triggered. i Represents the first in multimodal data iLet w represent a set of n data segments, M represent a set of multimodal data, and m represent a subset of M. Indicates that subset m is at the th order. i The strategy decision vector for each data segment; α represents the coefficient of scenario value, and β represents the coefficient of event trigger probability. Let λ represent the resource cost function required to execute the optimal strategy, and let λ represent the loss weight of the resource cost. This represents the sum of storage usage for the optimal strategy. This represents the summation of uplink bandwidth usage for the optimal strategy. This indicates the vehicle's current remaining available storage capacity. This indicates the vehicle's current available uplink bandwidth. This indicates the set of policies allowed by the compliance template.

[0030] Compared with existing technologies, the technical effects achieved by this solution are as follows: This solution constructs a mathematical optimization model with the goal of maximizing value integrals and constrained by the real-time resource status of the vehicle. It quantitatively integrates scene value, event trigger probability, and resource cost to solve for the optimal strategy. This abandons the extensive approach of existing technologies that rely on experience or fixed rules to formulate data processing strategies, and realizes the quantification, intelligence, and optimal solution of autonomous driving data processing strategies. At the same time, it decomposes multimodal data into refined decision-making units by segment and by mode. Through policy decision vectors, it realizes customized policy configuration for each data segment and each mode, so that resource allocation and data processing strategies are accurately matched with the actual value of each data segment and the system resource status, thereby improving the resource utilization efficiency and the scientificity and accuracy of policy formulation of the autonomous driving data processing system.

[0031] In one embodiment of the present invention, the scene value is determined comprehensively based on scene perception features;

[0032] The event trigger probability is predicted by a lightweight time series model based on the event information, and is used to predict whether the event will be triggered in the first time period in the future.

[0033] Scene perception features include at least one of the following: scene complexity, density of interactive subjects, near-collision index, system boundary proximity, key area coverage, collision time threshold, peak lateral and longitudinal acceleration of vehicles, number of target objects, density of target objects, road structure, lighting conditions, weather conditions, occlusion status, and vehicle-side safety self-assessment level.

[0034] Event information includes at least one of the following: collision, sudden deceleration, activation of driver assistance systems, and driver takeover.

[0035] Compared with existing technologies, the technical effects achieved by this solution are as follows: This solution comprehensively determines the value of a scene by selecting core scene perception features such as collision time threshold, deceleration index, and the number and density of target objects. It accurately quantifies the evidence collection and risk value of different data segments. At the same time, by combining collision and rapid deceleration event information, it predicts the probability of event triggering in the first time period in the future (i.e., whether the event will be triggered in the future Δt) through a lightweight time series model. This enables early prediction of potentially high-risk scenarios and overcomes the limitations of existing technologies that rely on hard thresholds to trigger data processing and cannot cover potentially high-risk segments. It avoids the lag and one-sidedness of single hard threshold judgment and adapts to the real-time inference requirements of the vehicle end through a lightweight model. This allows the vehicle's autonomous driving system to adjust its data processing strategy in advance during the risk escalation stage, achieving effective coverage of potentially high-risk segments (such as low-frequency accidents and near-miss collisions).

[0036] Furthermore, by using scenario value and event trigger probability as the core basis for formulating data processing strategies, the storage, encoding, and transmission resources of data are tilted towards high-value and high-risk data. This solves the problem of extensive allocation of existing technology resources and the loss of key evidence due to low-value data crowding out resources. Under limited onboard resources, it maximizes the high-fidelity retention of key data and improves the targeting and effectiveness of autonomous driving data processing.

[0037] In one embodiment of the present invention, the compliance constraints include at least one of the following: minimum data fidelity rule, data retention duration rule, privacy information anonymization level rule, and event forensics window rule;

[0038] Among them, the minimum data fidelity rule limits the lower limits of video resolution and frame rate, and the lower limit of point cloud data density within the event window;

[0039] The data retention period rule distinguishes the minimum storage period between ordinary data fragments and event data fragments;

[0040] The rules for anonymization levels of privacy information define the standards for the levels of desensitization processing of privacy information.

[0041] The event evidence collection window rules stipulate that data within a preset time period before and after the event occurrence time must be retained in full with the highest fidelity.

[0042] Compared with existing technologies, the technical effects achieved by adopting this solution are as follows: This solution clarifies compliance constraints as rules for minimum data fidelity, retention time, anonymization level of privacy information, and event evidence collection window, and refines and quantifies each rule. It replaces the compliance configurations fixed in the code in existing technologies with standardized and parameterized compliance requirements. This not only ensures that there are clear compliance bottom lines to follow throughout the entire process of storing, encoding, and transmitting autonomous driving data, but also solves the problem that traditional compliance configurations require modification of core code when adapting to cross regions and businesses. This reduces the cost and difficulty of compliance adaptation and improves the versatility and deployment efficiency of the vehicle autonomous driving system.

[0043] Furthermore, the compliance rules cover the core compliance dimensions of event data forensics, general data retention, and privacy protection. They ensure the validity of forensic evidence collection of key event data by setting a minimum fidelity threshold for the event window and ensuring full high-fidelity retention of the forensic data window. They also achieve reasonable utilization of storage resources under compliance by differentiating the retention time of general and event data. In addition, they avoid the risk of privacy leakage through privacy information anonymization rules. This allows the vehicle autonomous driving system to fully meet compliance requirements while optimizing data processing, solving the problems of rigid compliance rules, difficulty in adaptation, and difficulty in balancing compliance and resource utilization in existing technologies.

[0044] In one embodiment of the present invention, the multimodal data includes: vehicle video data, radar data, point cloud data, CAN signals, and vehicle log data;

[0045] Performing adaptive coding and cross-modal redundancy removal according to the optimal strategy includes:

[0046] Perform at least one of the following on-vehicle video data: adaptive bitrate control, adaptive GOP setting, adaptive ROI QP adjustment, and scene switching I-frame densification processing;

[0047] Perform adaptive adjustment of voxel rasterization ratio and / or octree sparsity ratio on point cloud data;

[0048] Perform adaptive adjustment of the reflection intensity threshold and / or sparsification of tracking trajectory points on the radar data;

[0049] Perform differential coding and / or semantic deduplication on CAN signals and log data.

[0050] Compared with existing technologies, the technical effects achieved by this solution are as follows: This solution adopts an adaptive coding strategy to address the differences in characteristics between vehicle video, radar, point cloud, CAN signal, and driving logs. It abandons the crude approach of using fixed coding and independent coding within each modality in existing technologies, and achieves precise matching between the coding strategy of each modality and its own data characteristics and actual value requirements. Specifically, the adaptive control of bitrate, GOP, ROI, and QP of vehicle video data, the dynamic adjustment of sparsity ratio of point cloud and radar data, and the differential coding and semantic redundancy removal processing of CAN signal and log data not only ensure the fidelity of high-value and key data, but also perform targeted resource compression on low-value data, reducing the redundancy of single-modality data, and avoiding over-compression or resource waste caused by a uniform coding method for some modal data.

[0051] Furthermore, the combination of multimodal differentiated adaptive coding lays the foundation for subsequent cross-modal redundancy removal, further improving the overall compression and resource utilization efficiency of multimodal data. It solves the problems of high redundancy, poor compression effect, and mismatch between coding strategy and data characteristics in existing technologies for multimodal data coding. Under limited on-board storage and bandwidth resources, it achieves refined and efficient management of multimodal data coding, maximizing both data fidelity and resource utilization.

[0052] In one embodiment of the present invention, performing adaptive coding and cross-modal deduplication according to an optimal strategy includes:

[0053] Geometric information and motion vectors of vehicles are extracted based on point cloud data and radar data, and video ROI masks are generated.

[0054] Reduce the bitrate of the non-ROI region of the video based on the video ROI mask, and perform redundancy control on the non-ROI region of the video by increasing the quantization parameters or reducing the frame rate.

[0055] Compared with existing technologies, the technical effects achieved by this solution are as follows: This solution overcomes the limitations of existing technologies that process multimodal data independently and fail to explore the information correlation between modalities. It extracts geometric information and motion vectors based on the spatial geometric characteristics of point cloud and radar data to generate accurate video ROI masks, realizing information synergy and complementarity between multimodal data. Furthermore, based on the video ROI mask, it implements targeted redundancy control on non-ROI areas of the video by reducing the bit rate, increasing quantization parameters, or reducing the frame rate. It accurately identifies and removes background redundant information in the video that is irrelevant to autonomous driving evidence collection and risk assessment. While ensuring high fidelity in key areas of the video, it reduces the amount of redundant video data and further improves the overall compression efficiency of multimodal data.

[0056] Furthermore, by using objective geometric and motion information from point clouds and radar to determine video redundancy areas, the subjectivity and one-sidedness of redundancy removal from a single video modality are avoided. This makes redundancy removal processing more accurate and better suited to the actual value requirements of autonomous driving data, solving the problems of high redundancy in multimodal data, underutilization of cross-modal information, and bandwidth and storage resources being squeezed out by redundant data in existing technologies.

[0057] In one embodiment of the present invention, any candidate strategy in the candidate strategy set includes at least one of the following: encoding quality level of the corresponding modal data, sparsity of point cloud, data granularity of radar, anonymization level, data retention duration, data upload priority, and privacy desensitization level.

[0058] Compared with existing technologies, the technical effects achieved by adopting this solution are as follows: This solution clarifies that each candidate strategy in the candidate strategy set includes at least one of the following: coding quality level, point cloud sparsity, radar data granularity, anonymization level, data retention duration, data upload priority, and privacy desensitization level. This enables refined and quantifiable control over the entire process of multimodal data processing, abandoning the extensive mode of single data processing strategy and fixed parameter configuration in existing technologies, and improving the targeting and flexibility of autonomous driving data processing.

[0059] Secondly, the present invention also provides a storage optimization apparatus for autonomous driving data. The storage optimization apparatus is used to implement the storage optimization method for autonomous driving data as described in any of the above examples. The storage optimization apparatus for autonomous driving data includes:

[0060] The data extraction unit is used to collect multimodal data of the vehicle and extract the vehicle's scene perception features and event information.

[0061] The data evaluation unit is used to evaluate the scene value and event trigger probability of a vehicle in each data segment based on scene perception features and event information.

[0062] Template loading unit: This unit is used to load a preset compliance template to determine the compliance constraints for data processing.

[0063] The strategy generation unit is used to generate a set of candidate strategies that conform to multimodal data based on scenario value, event triggering probability, compliance constraints, and vehicle resource status.

[0064] The strategy selection unit is used to select the optimal strategy from the candidate strategy set based on the current mode of the vehicle.

[0065] The strategy execution unit is used to optimize and transmit the best strategy, and to perform differential synchronization and closed-loop policy updates in network interruption and recovery scenarios.

[0066] Compared with existing technologies, the technical effects achieved by adopting this technical solution are as follows: it can achieve the technical effects corresponding to any of the above examples, which will not be elaborated here.

[0067] Thirdly, the present invention also provides a storage optimization system for autonomous driving data, which is applied to the storage optimization apparatus for autonomous driving data in any of the above examples to execute the storage optimization method for autonomous driving data in any of the above examples.

[0068] Compared with existing technologies, the technical effects achieved by adopting this technical solution are as follows: it can achieve the technical effects corresponding to any of the above examples, which will not be elaborated here.

[0069] By adopting the technical solution of the present invention, the following technical effects can be achieved:

[0070] This invention extracts scene perception features and event information from vehicle multimodal data, evaluates the scene value and event triggering probability of data segments, combines compliance constraints of compliance templates with real-time vehicle resource status, generates and filters the optimal strategy adapted to the current modality, and then performs targeted optimization processing and transmission. With the addition of differential synchronization after network interruption recovery and a closed-loop update mechanism for strategy, it achieves fine-grained allocation of resources for autonomous driving data storage and uploading. This not only breaks through the limitations of hard threshold triggering and achieves effective coverage of potentially high-risk segments, but also abandons the multimodal independent encoding mode, reducing data redundancy and improving compression effect.

[0071] Furthermore, this invention achieves flexible adaptation of compliance rules through preset compliance templates, reducing the adaptation costs across regions and businesses. It also improves the reliability of data backhaul in weak network scenarios by relying on priority transmission and differential synchronization. Moreover, through dynamic closed-loop updates of resource status and policies, it avoids system performance fluctuations and ensures stable system operation. Thus, with limited in-vehicle storage and network bandwidth resources, it solves the problems of extensive resource allocation, easy loss of key evidence, high redundancy of multimodal data, difficulty in compliance adaptation, unreliable transmission in weak networks, and unstable system performance in existing technologies. It improves the retention quality and transmission reliability of key data, taking into account the intelligence, refinement, compliance, and stability of autonomous driving data processing, and enhances the overall performance of the autonomous driving data storage and upload system. Attached Figure Description

[0072] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 A flowchart illustrating an optimized storage method for autonomous driving data provided in an embodiment of the present invention;

[0074] Figure 2 A block diagram of an autonomous driving data storage optimization device provided in an embodiment of the present invention.

[0075] Explanation of reference numerals in the attached figures:

[0076] 100. Data extraction unit; 200. Data evaluation unit; 300. Template loading unit; 400. Strategy generation unit; 500. Strategy filtering unit; 600. Strategy execution unit. Detailed Implementation

[0077] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0078] Firstly, such as Figure 1 As shown, the present invention provides a method for optimizing the storage of autonomous driving data, the method comprising:

[0079] S100: Collects multimodal data of the vehicle and extracts the vehicle's scene perception features and event information;

[0080] S200: Based on scene perception features and event information, evaluate the scene value and event trigger probability of the vehicle in each data segment;

[0081] S300: Load a pre-defined compliance template to determine compliance constraints for data processing;

[0082] S400: Generates a set of candidate strategies that conform to multimodal data based on scenario value, event trigger probability, compliance constraints, and vehicle resource status;

[0083] S500: Select the optimal strategy from the candidate strategy set based on the vehicle's current mode;

[0084] S600: Performs optimized processing and transmission of the optimal policy, and performs differential synchronization and policy closed-loop update in network interruption and recovery scenarios.

[0085] Understandably, this solution extracts scene perception features and event information from vehicle multimodal data, evaluates the scene value and event trigger probability of data segments, combines compliance constraints of compliance templates with the vehicle's real-time resource status, generates and filters the optimal strategy suitable for the current modality, and then performs targeted optimization processing and transmission. Coupled with differential synchronization after network interruption recovery and a closed-loop strategy update mechanism, it achieves fine-grained resource allocation for autonomous driving data storage and uploading. This not only overcomes the limitations of hard threshold triggering, achieving effective coverage of potentially high-risk segments, but also abandons the multimodal independent encoding mode, reducing data redundancy and improving compression performance. The vehicle's current modality includes available storage space, available network bandwidth, temperature, and power supply.

[0086] Furthermore, this solution achieves flexible adaptation of compliance rules through pre-set compliance templates, reducing the adaptation costs across regions and businesses. It also improves the reliability of data backhaul in weak network scenarios by relying on priority transmission and differential synchronization. Moreover, through dynamic closed-loop updates of resource status and policies, it avoids system performance fluctuations and ensures stable system operation. Thus, with limited onboard storage and network bandwidth resources, it solves the problems of extensive resource allocation, easy loss of key evidence, high redundancy of multimodal data, difficulty in compliance adaptation, unreliable transmission in weak networks, and unstable system performance in existing technologies. It improves the retention quality and transmission reliability of key data, balancing the intelligence, refinement, compliance, and stability of autonomous driving data processing, and enhancing the overall performance of the autonomous driving data storage and upload system. Specifically, the dynamic closed-loop update operation of the policy is as follows: after completing a policy execution and data upload, the autonomous driving data processing system updates the current modality, including remaining storage capacity, network bandwidth usage, and system operating environment status. If a new compliance template is issued from the cloud, the autonomous driving data processing system can update it immediately, allowing compliance constraints to be adjusted in real time without interrupting the operation of the autonomous driving data processing system.

[0087] In some embodiments provided by this invention, the optimization processing and transmission of the optimal strategy include:

[0088] Perform adaptive coding and cross-modal redundancy removal according to the optimal strategy;

[0089] According to the priority sequence determined by the optimal strategy, the optimization results of adaptive coding and cross-modal redundancy removal are placed into different transmission queues;

[0090] Based on real-time network feedback, select the corresponding data segments for transmission according to priority.

[0091] In the event of network congestion or weak network conditions, the transmission priority of event segments and high-value segments is automatically increased, while low-value segments are proactively discarded or downgraded.

[0092] Understandably, this solution performs adaptive coding and cross-modal redundancy removal on multimodal data through optimal strategies, thereby achieving collaborative optimization processing of multimodal data. It abandons the independent coding mode of multimodal data in existing technologies, eliminates information redundancy between modalities, and reduces the coding resource overhead of accurately matching each data segment (any data segment in multimodal data). While ensuring the fidelity of key data, it reduces the data storage and transmission volume.

[0093] Furthermore, the optimized data is divided into transmission queues according to the data priority sequence specified by the optimal strategy. For example, event data > high SAV or high TPR data > data at the lower limit specified by compliance constraints > other data. Combined with real-time network feedback, such as round-trip latency, packet loss rate, available bandwidth, and storage capacity between the vehicle and the cloud, data is sent according to data priority. At the same time, in the case of network congestion or weak network conditions, the transmission priority of event fragments and high-value fragments is increased, and low-value fragments are actively discarded or downgraded. This changes the status quo of data transmission without fine-grained priority and without differentiated processing in weak network conditions in the existing technology. It avoids low-value data crowding out limited network bandwidth and ensures that event and high-value key data are transmitted first in complex network scenarios. It improves the reliability of data transmission and the efficiency of network bandwidth utilization. It realizes fine-grained management of resources from the two dimensions of data processing and transmission scheduling. It solves the problems of high redundancy of multimodal data, coarse allocation of bandwidth resources, and easy loss of key evidence in weak network transmission in the existing technology, and ensures efficient processing and priority backhaul of key data for autonomous driving.

[0094] In some embodiments provided by the present invention, selecting the optimal strategy from the candidate strategy set includes:

[0095] Solve for the maximum value integral under resource state constraints and output the optimization results for multimodal data;

[0096] The objective function for maximizing the value integral is:

[0097] Formula 1: ;

[0098] The constraints on resource status are:

[0099] Formula 2:

[0100] ;

[0101] In the formula, Indicates the maximum value. Indicates the value of a scenario. This represents the probability of an event being triggered (i.e., the probability that an event may occur within the next Δt, such as a collision, sudden deceleration, triggering of driver assistance systems, or driver takeover). i Represents the first in multimodal datai The data segments (e.g., a 1-second data window or a 500ms data segment), w represents the set of n data segments, M represents the set of multimodal data, and m represents a subset of M (e.g., M = {vehicle video data, radar data, point cloud data, CAN signal and driving log data}). Indicates that subset m is at the th order. i The strategy decision vector for each data segment; α represents the coefficient of scenario value, and β represents the coefficient of event trigger probability. Let λ represent the resource cost function required to execute the optimal strategy (e.g., storage usage, bandwidth usage, additional computing power, additional energy consumption), where λ represents the loss weight of the resource cost. This represents the sum of storage usage for the optimal strategy. This represents the summation of uplink bandwidth usage for the optimal strategy. This indicates the vehicle's current remaining available storage capacity. This indicates the vehicle's current available uplink bandwidth. This indicates the set of policies allowed by the compliance template.

[0102] Understandably, this solution constructs a mathematical optimization model with the goal of maximizing value integrals and constrained by the real-time resource status of the vehicle. It quantitatively integrates scenario value, event trigger probability, and resource cost to solve for the optimal strategy. This abandons the extensive approach of relying on experience or fixed rules to formulate data processing strategies in existing technologies, and realizes the quantification, intelligence, and optimal solution of autonomous driving data processing strategies. At the same time, it decomposes multimodal data into refined decision-making units by segment and by mode. Through policy decision vectors, it realizes customized policy configuration for each data segment and each mode. This allows resource allocation and data processing strategies to accurately match the actual value of each data segment with the system resource status. Under the premise of ensuring storage / bandwidth / compliance, it maximizes the "Scenario Value (SAV) + Risk Prediction (Event Trigger Probability TPR)" of multimodal data while minimizing resource costs, thereby improving the resource utilization efficiency and the scientific and accurate nature of strategy formulation in the autonomous driving data processing system.

[0103] In some embodiments provided by the present invention, the scene value is determined comprehensively based on scene perception features;

[0104] The event trigger probability is predicted by a lightweight time series model based on the event information, and is used to predict whether the event will be triggered in the first time period in the future.

[0105] Scene perception features include at least one of the following: scene complexity, density of interactive subjects, near-collision index, system boundary proximity, key area coverage, collision time threshold, peak lateral and longitudinal acceleration of vehicles, number of target objects, density of target objects, road structure, lighting conditions, weather conditions, occlusion status, and vehicle-side safety self-assessment level.

[0106] Event information includes at least one of the following: collision, sudden deceleration, activation of driver assistance systems, and driver takeover.

[0107] Understandably, this solution comprehensively determines the value of a scenario by selecting core scene perception features such as collision time threshold, deceleration index, and the number and density of target objects. It accurately quantifies the evidence collection and risk value of different data segments. At the same time, it combines collision and rapid deceleration event information and uses a lightweight time series model to predict the probability of event triggering in the first time period in the future (i.e., whether the event will be triggered in the future Δt). This enables early prediction of potentially high-risk scenarios and overcomes the limitations of existing technologies that rely on hard thresholds to trigger data processing and cannot cover potentially high-risk segments. It avoids the lag and one-sidedness of single hard threshold judgment and adapts to the real-time inference requirements of the vehicle end through a lightweight model. This allows the vehicle's autonomous driving system to adjust its data processing strategy in advance during the risk escalation stage, achieving effective coverage of potentially high-risk segments (such as low-frequency accidents and near-miss collisions).

[0108] Furthermore, by using scenario value and event trigger probability as the core basis for formulating data processing strategies, the storage, encoding, and transmission resources of data are tilted towards high-value and high-risk data. This solves the problem of extensive allocation of existing technology resources and the loss of key evidence due to low-value data crowding out resources. Under limited on-board resources, it maximizes the high-fidelity retention of key data and improves the targeting and effectiveness of autonomous driving data processing.

[0109] The formula for calculating the value of a scene is as follows:

[0110] Formula 3: ;

[0111] In the formula, X k It represents at least one of the following features: scene complexity, density of interactive subjects, near-collision index, system boundary proximity, key area coverage, collision time threshold, peak lateral and longitudinal acceleration of vehicles, number of target objects, density of target objects, road structure, lighting conditions, weather conditions, occlusion status, and vehicle-side safety self-assessment level. V k These are configurable weights corresponding to the scene perception features mentioned above;

[0112] In some embodiments provided by the present invention, compliance constraints include at least one of the following: minimum data fidelity rule, data retention duration rule, privacy information anonymization level rule, and event evidence collection window rule.

[0113] Among them, the minimum data fidelity rule limits the lower limits of video resolution and frame rate, and the lower limit of point cloud data density within the event window;

[0114] The data retention period rule distinguishes the minimum storage period between ordinary data fragments and event data fragments;

[0115] The rules for anonymization levels of privacy information define the standards for the levels of desensitization processing of privacy information.

[0116] The event evidence collection window rules stipulate that data within a preset time period before and after the event occurrence time must be retained in full with the highest fidelity.

[0117] Understandably, this solution clarifies compliance constraints as minimum data fidelity (e.g., event window video: resolution ≥ 1080p, frame rate ≥ 30 frames / second; point cloud ≥ Xpts / s, where X is a specific quantitative value, such as 100,000 points / second), retention duration (e.g., ordinary segments ≥ Y days; event segments ≥ Z days), anonymization level of privacy information (e.g., license plate, face, or address), and rules for event evidence collection windows (e.g., data within a preset time before and after the event occurrence must be retained in full with the highest fidelity). Furthermore, it refines and quantifies each rule, replacing the compliance configurations fixed in the code of existing technologies with standardized and parameterized compliance requirements. This not only ensures that the entire process of storing, encoding, and transmitting autonomous driving data has clear compliance bottom lines to follow, but also solves the problem of needing to modify core code when adapting traditional compliance configurations across regions and businesses. This reduces the cost and difficulty of compliance adaptation and improves the versatility and deployment efficiency of vehicle autonomous driving systems.

[0118] Furthermore, the compliance rules cover the core compliance dimensions of event data forensics, general data retention, and privacy protection. They ensure the validity of forensic evidence of key event data through rules that guarantee the minimum fidelity of the event window and retain all data with high fidelity during the forensics window. They also achieve reasonable utilization of storage resources under compliance premise by differentiating the retention time of general and event data. In addition, they avoid the risk of privacy leakage through rules that desensitize privacy information. This allows the autonomous driving system of the vehicle to fully meet compliance requirements while optimizing data processing, and solves the problems of rigid compliance rules, difficulty in adaptation, and difficulty in balancing compliance and resource utilization in existing technologies.

[0119] In some embodiments provided by the present invention, the multimodal data includes: vehicle video data, radar data, point cloud data, CAN signals, and driving log data;

[0120] Performing adaptive coding and cross-modal redundancy removal according to the optimal strategy includes:

[0121] Perform at least one of the following on-vehicle video data: adaptive bitrate control, adaptive GOP setting, adaptive ROI QP adjustment, and scene switching I-frame densification processing;

[0122] Perform adaptive adjustment of voxel rasterization ratio and / or octree sparsity ratio on point cloud data;

[0123] Perform adaptive adjustment of the reflection intensity threshold and / or sparsification of tracking trajectory points on the radar data;

[0124] Perform differential coding and / or semantic deduplication on CAN signals and log data.

[0125] Understandably, this solution employs an adaptive encoding strategy to address the unique characteristics of in-vehicle video, radar, point cloud, CAN signals, and driving logs. It abandons the crude approach of fixed encoding and independent encoding within each modality used in existing technologies, achieving precise matching between the encoding strategy for each modality and its own data characteristics and actual value requirements. Specifically, the adaptive adjustment of bitrate, GOP, and ROI QP for in-vehicle video data, the dynamic adjustment of sparsity ratios for point cloud and radar data, and the differential encoding and semantic redundancy removal processing for CAN signals and log data ensure the fidelity of high-value, critical data while selectively compressing low-value data, reducing the redundancy of single-modality data, and avoiding over-compression or resource waste caused by a uniform encoding method for some modalities. Here, GOP stands for Group of Pictures; ROI QP stands for Region of Interest Quantization Parameter; and CAN stands for Controller Area Network.

[0126] Furthermore, the combination of multimodal differentiated adaptive coding lays the foundation for subsequent cross-modal redundancy removal, further improving the overall compression and resource utilization efficiency of multimodal data. It solves the problems of high redundancy, poor compression effect, and mismatch between coding strategy and data characteristics in existing technologies for multimodal data coding. Under limited on-board storage and bandwidth resources, it achieves refined and efficient management of multimodal data coding, maximizing both data fidelity and resource utilization.

[0127] In some embodiments provided by the present invention, performing adaptive coding and cross-modal redundancy removal according to the optimal strategy includes:

[0128] Geometric information and motion vectors of vehicles are extracted based on point cloud data and radar data, and video ROI masks are generated.

[0129] Reduce the bitrate of the non-ROI region of the video based on the video ROI mask, and perform redundancy control on the non-ROI region of the video by increasing the quantization parameters or reducing the frame rate.

[0130] Understandably, this solution overcomes the limitations of existing technologies that process multimodal data independently and fail to explore the information correlation between modalities. It extracts geometric information and motion vectors based on the spatial geometric characteristics of point cloud and radar data, thereby generating accurate video ROI masks and achieving information synergy and complementarity between multimodal data. Furthermore, based on the video ROI masks, it implements targeted redundancy reduction controls on non-ROI areas of the video, such as reducing bitrate, increasing quantization parameters, or reducing frame rate. This accurately identifies and removes redundant background information in the video that is irrelevant to autonomous driving evidence collection and risk assessment. While ensuring high fidelity in key video areas, it reduces the amount of redundant video data and further improves the overall compression efficiency of multimodal data. Here, ROI stands for Region of Interest.

[0131] Furthermore, by using objective geometric and motion information from point clouds and radar to determine video redundancy areas, the subjectivity and one-sidedness of redundancy removal from a single video modality are avoided. This makes redundancy removal processing more accurate and better suited to the actual value requirements of autonomous driving data. It solves the problems of high redundancy in multimodal data, underutilization of cross-modal information, and bandwidth and storage resources being squeezed out by redundant data in existing technologies.

[0132] In some embodiments provided by the present invention, any candidate strategy in the candidate strategy set includes at least one of the following: the encoding quality level of the corresponding modal data, the sparsity of the point cloud, the data granularity of the radar, the anonymization level, the data retention duration, the data upload priority, and the privacy desensitization level.

[0133] Understandably, this solution achieves refined and quantifiable control over the entire multimodal data processing process by specifying that each candidate strategy in the candidate strategy set includes at least one of the following: coding quality level, point cloud sparsity, radar data granularity, anonymization level, data retention duration, data upload priority, and privacy desensitization level. This abandons the extensive mode of single data processing strategy and fixed parameter configuration in existing technologies, and improves the targeting and flexibility of autonomous driving data processing.

[0134] Secondly, such as Figure 2As shown, the present invention also provides a storage optimization device for autonomous driving data. This storage optimization device is used to implement the autonomous driving data storage optimization method as described in any of the above embodiments. The autonomous driving data storage optimization device includes a data extraction unit 100, a data evaluation unit 200, a template loading unit 300, a strategy generation unit 400, a strategy filtering unit 500, and a strategy execution unit 600. Specifically, the data extraction unit 100 is used to collect multimodal data of the vehicle and extract the vehicle's scene perception features and event information. The data evaluation unit 200 is used to evaluate the scene value and event triggering probability of the vehicle in each data segment based on the scene perception features and event information. The template loading unit 300 is used to load a preset compliance template to determine the compliance constraints for data processing. The strategy generation unit 400 is used to generate a candidate strategy set that conforms to the multimodal data based on the scene value, event triggering probability, compliance constraints, and the vehicle's resource status. The strategy filtering unit 500 is used to filter the optimal strategy from the candidate strategy set based on the vehicle's current modality. The strategy execution unit 600 is used to perform optimization processing and transmission on the optimal strategy, and to perform differential synchronization and strategy closed-loop update in network interruption and recovery scenarios.

[0135] Thirdly, the present invention also provides an autonomous driving data storage optimization system, which is applied to the autonomous driving data storage optimization apparatus as described in any of the above embodiments to execute the autonomous driving data storage optimization method as described in any of the above embodiments.

[0136] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method for optimizing the storage of autonomous driving data, characterized in that, The storage optimization method includes: Collect multimodal data of the vehicle and extract the scene perception features and event information of the vehicle; Based on the scene perception features and the event information, the scene value and event trigger probability of the vehicle in each data segment are evaluated; Load a pre-defined compliance template to determine compliance constraints for data processing; Based on the scenario value, the event trigger probability, the compliance constraints, and the vehicle's resource status, a set of candidate strategies that conform to the multimodal data is generated. Based on the current mode of the vehicle, the optimal strategy is selected from the candidate strategy set; The optimal strategy is optimized and transmitted, and differential synchronization and closed-loop strategy update are performed in network interruption and recovery scenarios. The step of selecting the optimal strategy from the candidate strategy set includes: Solve for the maximum value integral under the resource state constraints and output the optimization results of the multimodal data; The objective function for maximizing the value integral is: Official 1: ; The constraints on the resource status are: Formula 2: ； In the formula, Indicates the maximum value. This indicates the value of the scenario. This indicates the probability of the event being triggered. i Indicates the first in the multimodal data i Let w represent the set of n data segments, M represent the set of multimodal data, and m represent a subset of M. Indicates that subset m is at the th order. i The strategy decision vector for each data segment; α represents the coefficient of the scenario value, and β represents the coefficient of the event trigger probability. Let λ represent the resource cost function required to execute the optimal strategy, and let λ represent the loss weight of the resource cost. This represents the sum of storage usage for the optimal strategy. This represents the summation of uplink bandwidth usage for the optimal strategy. This indicates the current remaining available storage capacity of the vehicle. This indicates the current available uplink bandwidth on the network for the vehicle. This indicates the set of policies allowed by the compliance template.

2. The storage optimization method according to claim 1, characterized in that, The optimization processing and transmission of the optimal strategy include: Perform adaptive coding and cross-modal redundancy removal according to the optimal strategy described above; According to the priority sequence determined by the optimal strategy, the optimization results of the adaptive coding and the cross-modal redundancy removal are placed into different transmission queues; Based on real-time network feedback, select the corresponding data segments for transmission according to priority. In the event of network congestion or weak network conditions, the transmission priority of event segments and high-value segments is automatically increased, while low-value segments are proactively discarded or downgraded.

3. The storage optimization method according to claim 1, characterized in that, The value of the scene is determined comprehensively based on the scene perception features; The event trigger probability is predicted by a lightweight time series model based on the event information, and is used to predict whether the event will be triggered in the first time period in the future. The scene perception features include at least one of the following: scene complexity, density of interactive subjects, near-collision index, system boundary proximity, key area coverage, collision time threshold, peak lateral and longitudinal acceleration of vehicles, number of target objects, density of target objects, road structure, lighting conditions, weather conditions, occlusion status, and vehicle-side safety self-assessment level. The event information includes at least one of the following: collision, sudden deceleration, triggering of driver assistance systems, and driver takeover.

4. The storage optimization method according to any one of claims 2-3, characterized in that, The compliance constraints include at least one of the following: minimum data fidelity rule, data retention duration rule, privacy information anonymization level rule, and event evidence collection window rule; Among them, the minimum data fidelity rule limits the lower limits of video resolution and frame rate, and the lower limit of point cloud data density within the event window; The data retention duration rule distinguishes the minimum storage period between ordinary data segments and event data segments; The privacy information anonymization level rules define the standards for the desensitization processing level of privacy information; The event evidence collection window rules stipulate that data within a preset time period before and after the event occurrence time must be retained in full with the highest fidelity.

5. The storage optimization method according to claim 4, characterized in that, The multimodal data includes: vehicle video data, radar data, point cloud data, CAN signals, and driving log data; Performing adaptive coding and cross-modal redundancy removal according to the optimal strategy includes: Perform at least one of the following on the vehicle video data: adaptive bitrate control, adaptive GOP setting, adaptive ROI QP adjustment, and scene switching I-frame densification processing; Perform adaptive adjustment of voxel rasterization ratio and / or octree sparsity ratio on the point cloud data; The radar data is subjected to adaptive adjustment of the reflection intensity threshold and / or sparsification of tracking trajectory points; Differential coding and / or semantic deduplication are performed on the CAN signals and log data.

6. The storage optimization method according to claim 5, characterized in that, Performing adaptive coding and cross-modal redundancy removal according to the optimal strategy includes: Based on the point cloud data and the radar data, the geometric information and motion vector of the vehicle are extracted, and a video ROI mask is generated. The bitrate of the non-ROI region of the video is reduced based on the video ROI mask, and redundancy control is performed on the non-ROI region of the video to increase quantization parameters or reduce frame rate.

7. The storage optimization method according to claim 4, characterized in that, Any candidate strategy in the candidate strategy set includes at least one of the following: encoding quality level of the corresponding modal data, sparsity of the point cloud, data granularity of the radar, anonymization level, data retention duration, data upload priority, and privacy desensitization level.

8. A storage optimization device for autonomous driving data, characterized in that, The storage optimization device is used to implement the storage optimization method for autonomous driving data as described in any one of claims 1-7, wherein the storage optimization device for autonomous driving data comprises: A data extraction unit (100) is used to collect multimodal data of the vehicle and extract the scene perception features and event information of the vehicle. A data evaluation unit (200) is used to evaluate the scene value and event triggering probability of the vehicle in each data segment based on the scene perception features and the event information. Template loading unit (300), the template loading unit (300) is used to load a preset compliance template to determine the compliance constraints of data processing; The strategy generation unit (400) is used to generate a set of candidate strategies that conform to the multimodal data based on the scenario value, the event trigger probability, the compliance constraints and the resource status of the vehicle. A strategy filtering unit (500) is used to filter the optimal strategy from the candidate strategy set according to the current mode of the vehicle. The strategy execution unit (600) is used to perform optimization processing and transmission of the optimal strategy, and to perform differential synchronization and strategy closed-loop update in network interruption and recovery scenarios.

9. A storage optimization system for autonomous driving data, characterized in that, The autonomous driving data storage optimization system is applied to the autonomous driving data storage optimization apparatus as described in claim 8 to execute the autonomous driving data storage optimization method as described in any one of claims 1-7.

Images