Data processing method, storage medium and vehicle

Abstract

The invention relates to the technical field of intelligent driving, in particular to a data processing method and a vehicle. By calculating the computing power gap of the main chip of the driving assistance system, when the driving assistance system predicts that the driving assistance system has the computing power gap, the task of the driving assistance system which can be unloaded is transmitted to the intelligent cabin system for calculation. According to the data processing method provided by the embodiment of the invention, the cockpit computing power is borrowed, so that the function degradation of the driving assistance system caused by insufficient local resources is avoided, the sensing and decision-making performance stability is ensured, and meanwhile, the computing power utilization rate of the intelligent cockpit system can be improved.

Description

Data processing methods, storage media and vehicles Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a data processing method, storage medium, and vehicle. Background Technology

[0002] Advanced Driver Assistance Systems (ADAS) provide assistance and even partial automation functions based on driver control through sensors, algorithms, and actuators. The computational demands of ADAS increase exponentially with system complexity, the number of sensors, and algorithm evolution. From L1 to L2+ / L3, ADAS places higher demands on computing power (TOPS), memory bandwidth, real-time performance, power consumption, and functional safety. In extreme scenarios, computational overload can easily occur.

[0003] The cockpit system (SoC) primarily serves tasks such as voice recognition, in-vehicle display systems (e.g., instrument panel, central control screen, passenger entertainment screen), and multi-screen interaction. These tasks are characterized by "high burstiness and low sustainability." During daily driving, when the cockpit system receives a task, its computing power is rapidly increased; when there is no task, a large amount of computing power is idle, resulting in a significantly low overall utilization rate. Summary of the Invention

[0004] To address the problem of uneven computing power across different functional domains in existing driver assistance systems, this application provides a data processing method, a storage medium, and a vehicle.

[0005] To address the above problems, this application provides the following technical solution.

[0006] The first aspect of this application provides a data processing method, comprising the following steps: calculating the computing power required by the main chip of a driving assistance system; determining whether there is a computing power gap in the main chip based on the peak computing power of the main chip and the computing power required by the main chip; if so, the main chip analyzes the driving assistance system tasks that can be offloaded among all driving assistance system tasks; the main chip offloads the offloadable driving assistance system tasks to an intelligent cockpit system to calculate and generate a task execution strategy; the main chip receives the task execution strategy fed back by the intelligent cockpit system and executes it.

[0007] In this embodiment, by quantifying the computing power gap, when the ADAS main chip's own computing power is insufficient to support safe operation in the current driving scenario, the equivalent inference computing power temporarily borrowed from the outside is required. Suitable tasks are then offloaded to the cockpit system, achieving a closed-loop management of "computing power shortage warning - precise task offloading - external computing power supplementation," fundamentally preventing the degradation or failure of intelligent driving functions due to computing power shortages. Simultaneously, it also achieves collaborative utilization of computing power between the driver assistance system and the intelligent cockpit system, breaking the resource barrier of traditional "single-system dedicated computing power." On the one hand, there is no need to blindly upgrade the hardware specifications of the ADAS chip to meet computing power demands in extreme scenarios; the computing power shortage of the ADAS chip can be compensated by reusing the redundant computing power of the intelligent cockpit system, reducing the overall vehicle hardware procurement cost and R&D investment. On the other hand, the quantitative design of the computing power gap makes task offloading more targeted, transferring only "non-core and transferable" tasks (such as environmental image redundancy analysis, driving scenario pre-classification, user habit data statistics, etc.) to the cockpit system, avoiding invalid computing power occupation, achieving dynamic and balanced allocation of the vehicle's computing power resources, and improving resource utilization.

[0008] In conjunction with the first aspect, in some embodiments of this application, calculating the computing power required by the main chip of the driving assistance system includes: calculating the current real-time load of the driving assistance system; predicting the scenario threat level of the driving assistance system in a set future time period, converting the scenario threat level into a predicted demand load, wherein the scenario threat level is used to characterize the impact of driving scenario complexity on intelligent driving strategies; using the sum of the current real-time load and the predicted demand load as the predicted total load of the driving assistance system; and calculating the required computing power based on the predicted total load.

[0009] This application proposes a method for calculating computing power gap. This method calculates the predicted load of the ADAS system by combining the current real-time load level with the predicted load increase caused by threats. ADAS, by fusing high-precision maps, sensor inputs, and V2X information, predicts the upcoming entry into complex scenarios (such as unprotected left turns, construction zones, and dense pedestrian areas) 500ms to 1s in advance, and accordingly predicts the increase in computing power demand; it also anticipates the load gap in advance and proactively initiates a computing power borrowing request from the cockpit before local computing power becomes insufficient, ensuring that high-load tasks always operate at full specifications. In conjunction with the first aspect, in some embodiments of this application, the method for converting the scene threat level into predicted demand load includes: defining a threat load conversion coefficient, and using the product of the scene threat level and the threat load conversion coefficient as the predicted demand load for scene threat level conversion.

[0010] In this embodiment of the application, by introducing a threat load conversion coefficient, the abstract "scenario threat level" (such as low, medium, high, and critical) is quantified into specific computing power requirements, thereby realizing the mapping from semantic-level risk perception to physical-level resource scheduling.

[0011] In conjunction with the first aspect, in some embodiments of this application, the prediction of the scene threat level of the driving assistance system within a set future time period includes: setting an environmental threat feature vector, the environmental threat feature vector including multiple environmental threat parameters; and taking the sum of the products of each environmental threat parameter and its corresponding weight as the scene threat level.

[0012] In this embodiment, the driving assistance system constructs an environmental threat feature vector and introduces a parameter weighting mechanism to abstract the complex driving environment into a quantifiable and computable scene threat level value. This method not only improves the scientific rigor and flexibility of threat assessment.

[0013] In conjunction with the first aspect, in some embodiments of this application, the calculation of the current real-time load of the driving assistance system includes: obtaining the utilization rate of the central processing unit, the utilization rate of the neural network processor, the utilization rate of the graphics processor, and the memory bandwidth utilization rate of the driving assistance system; and using the sum of the products of two or more of the central processing unit utilization rate, the neural network processor utilization rate, the graphics processor utilization rate, and the memory bandwidth utilization rate, and the corresponding weights of the indicators, as the current real-time load of the driving assistance system.

[0014] In this embodiment, the driver assistance system no longer relies on a single indicator (such as NPU utilization alone) to roughly estimate the load. Instead, it comprehensively considers the real-time utilization of multiple key hardware resources, such as CPU, NPU, GPU, and memory bandwidth, and generates a unified "current real-time load" through weighted fusion. This method significantly improves the comprehensiveness and accuracy of load perception. Traditional solutions often only monitor the utilization of a single processor, but in reality: the NPU may be idle while the CPU is occupied by post-processing tasks (such as target tracking and trajectory planning); or memory bandwidth may be saturated, causing the NPU to frequently wait for data, and its computing power cannot be utilized. This application accurately identifies the true bottleneck of the system through multi-dimensional indicator fusion; the computing power gap prediction is closer to the actual execution capacity, avoiding the "misjudgment of having spare capacity" leading to rejection of lending, or the "misjudgment of being overloaded" leading to excessive degradation due to one-sided monitoring.

[0015] In conjunction with the first aspect, in some embodiments of this application, the method for analyzing driver assistance system tasks that can be offloaded includes: the driver assistance system setting evaluation indicators for each task, the evaluation indicators including computational intensity indicators, data locality indicators, and time criticality indicators; evaluating whether the task is suitable for offloading based on any one evaluation indicator, or the sum of the products of any number of evaluation indicators and their corresponding weights; wherein, the computational intensity refers to the amount of computation required to measure the task; the data locality indicator is used to characterize whether the data required by the task is already located in cross-domain shared memory or the amount of task data migration overhead; the time criticality indicator is used to characterize the sensitivity of the task to the impact of latency on the driver assistance system.

[0016] In this embodiment, considering that not all tasks are suitable for offloading to the intelligent cockpit system, a method for evaluating task offloading feasibility is proposed. By comprehensively considering three core indicators—computational intensity, data locality, and time criticality—and generating a unified "offloading priority" or "offloading feasibility score" through weighted fusion, a refined offloading decision at the task level can be achieved, avoiding the impact of task offloading on the normal operation of the two systems and maximizing the benefits of cross-domain scheduling.

[0017] In conjunction with the first aspect, in some embodiments of this application, the main chip offloads the unloadable driving assistance system tasks to the intelligent cockpit system for calculating and generating task execution strategies, including: determining whether the intelligent cockpit system resources are available; if not, rejecting the unloading request of the driving assistance system tasks to be unloaded; if so, calculating a priority queue of the driving assistance system tasks to be unloaded; analyzing the borrowing records of the driving assistance system tasks to be unloaded in the priority queue, wherein the borrowing records are used to characterize the impact of the task's past borrowing behavior on other tasks; and passing the driving assistance system tasks to be unloaded with compliant borrowing records and high priority to the intelligent cockpit system for calculation.

[0018] In this embodiment of the application, after determining that a task can be unloaded, the driving assistance system does not directly send it to the smart cockpit for execution. Instead, it introduces a collaborative decision-making mechanism to avoid ineffective scheduling and resource contention, improve system certainty, and prevent tasks from failing to execute after being unloaded. At the same time, it ensures the scheduling of high-priority tasks.

[0019] In conjunction with the first aspect, in some embodiments of this application, the main chip offloads the unloadable driving assistance system tasks to the intelligent cockpit system to calculate and generate task execution strategies. This further includes: transferring the unloadable driving assistance system tasks to the computing resource area of ​​the intelligent cockpit system, where the computing resource area can be used for data processing of both the intelligent cockpit system and the driving assistance system; calculating the available computing power of the cockpit resources based on the intelligent cockpit system's computing power configuration, real-time load, and predicted demand load; and allocating different proportions of computing power to the tasks of different priorities within the limited range of available computing power available for borrowing.

[0020] In this embodiment, considering that the driver assistance system may simultaneously assign multiple tasks to the intelligent cockpit system, or that multiple offloaded tasks may be waiting for the cockpit system to process in a queue at the same time, the available computing power resources are allocated according to task priority to improve the rationality of task execution and resource allocation. High-priority tasks (such as AEB-related perception): up to 80% of the available computing power can be obtained; medium- and low-priority tasks (such as parking post-processing): up to 50% or less can be obtained. The driver assistance system not only offloads tasks to the intelligent cockpit, but also provides differentiated and controlled resource supply for tasks of different priorities within the limited and dynamically changing available computing power of the cockpit through a refined computing power quota allocation mechanism.

[0021] A second aspect of this application provides a computer-readable storage medium storing program code, which is executed by one or more processors. When the program code runs on the processor, it causes an apparatus including one or more processors to perform the data processing method described in the above embodiments.

[0022] A third aspect of this application provides a vehicle that can execute the data processing method provided in the first aspect of this application, or include the data processing system provided in the second aspect of this application.

[0023] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more apparent, specific embodiments of this application are given below. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 is a schematic flowchart of the data processing method according to an embodiment of this application.

[0026] Figure 2 is a logical diagram of the data processing method according to an embodiment of this application.

[0027] Figure 3 is a logical structure diagram of the driving assistance system according to an embodiment of this application.

[0028] Figure 4 is a flowchart of the data processing of the intelligent cockpit system according to an embodiment of this application.

[0029] Figure 5 is a vehicle logic structure diagram of an embodiment of this application. Detailed Implementation

[0030] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0031] In this application's embodiments, prefixes such as "first" and "second" are used merely to distinguish different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0032] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0033] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system.

[0034] Intelligent driving vehicles are divided into multiple functional domains according to their functions and safety levels, including driver assistance systems, intelligent cockpit systems, powertrain systems, and network systems.

[0035] Driver assistance systems (ADAS) are responsible for environmental perception, decision-making and planning, and vehicle control, enabling Level 1–Level 4 autonomous driving functions. ADAS places extremely high demands on the performance, real-time capabilities, power consumption, and safety of the computing platform. ADAS typically employs a heterogeneous computing architecture, integrating different types of processors such as a Central Processing Unit (CPU), a Neural Processing Unit (NPU), and a Graphics Processing Unit (GPU) into the main chip to efficiently perform tasks such as perception, decision-making, planning, and control.

[0036] The CPU is responsible for scheduling and task management of the driver assistance system, running the operating system, executing logic control, state machine, vehicle communication, and high-level decision-making after fusing data from multiple sensors; the NPU is responsible for efficiently executing deep learning inference tasks (such as object detection, semantic segmentation, lane line recognition), processing raw sensor data such as camera and radar point clouds, and inferring AI models; the GPU is responsible for parallel image processing (such as distortion correction and image stitching), assisting in the operation of some deep learning models (especially in training or high-precision inference scenarios), and visualization rendering (such as 3D vehicle surrounding view and HMI interface).

[0037] In typical weather conditions with sparse targets, structured roads, and good lighting, lane lines are clear, requiring only lightweight lane detection; the number of surrounding traffic participants is small, resulting in a light load on the target detection model; and there are no obstructions or extreme lighting conditions, eliminating the need to activate image enhancement modules. Under these conditions, the driver assistance system operates under ideal conditions of low complexity and high predictability.

[0038] In extreme scenarios such as nighttime, heavy rain, and dense fog, the driver assistance system (ADAS) needs to activate multimodal fusion (camera + millimeter-wave radar + lidar), initiate image enhancement modules (for rain removal, fog removal, and low-light enhancement), run high-precision networks to reconstruct 3D space, perform dense trajectory prediction, and the decision-making module needs to consider more game-theoretic strategies (such as avoiding "ghost pedestrians") to generate safe intelligent assisted driving strategies. In these conditions, the computing power required for the ADAS system increases significantly compared to normal weather. If the ADAS chip's computing power is insufficient, risks such as decreased perception frame rate, increased decision latency, and functional degradation may occur.

[0039] In one specific embodiment, the ADAS-specific chip uses chips such as the Mobileye EyeQ series and Horizon J5. Although these chips can provide 4~128 TOPS of computing power, their neural network processor (NPU) architecture is highly optimized for standard convolutional neural network (CNN) inference. However, they have low efficiency in supporting emerging models such as Transformer, 3D sparse convolution, and dynamic graph neural networks, resulting in a significant reduction in actual effective computing power.

[0040] In existing technologies, to address the problem of computing power surges caused by extreme scenarios, the common approach is to simply upgrade the ADAS chip specifications and adopt high-performance ADAS chips to improve the chip's computing power, resulting in a surge in costs. Moreover, in some extreme scenarios, it is still impossible to guarantee that the computing power is sufficient for use.

[0041] The intelligent cockpit system is responsible for human-computer interaction, infotainment, driver status monitoring, and multi-screen collaboration. The intelligent cockpit system is based on a heterogeneous multi-core System-on-Chip (SoC), such as the Qualcomm 8295 or Kirin 990A chip system.

[0042] By achieving a balance between security and user experience through hardware isolation and virtualization / containerization, coupled with high-speed communication and a layered software stack, it supports complex scenarios such as multi-screen interaction, AI perception, and intelligent driving collaboration, and is moving from domain-specific to centralized computing and cockpit-driver integration. The intelligent cockpit system also includes three core computing engines: CPU, GPU, and NPU.

[0043] The computing power metrics of intelligent cockpit systems are designed for extreme scenarios. For example, the computing power required for intelligent cockpit systems to simultaneously perform screen rendering and control (including instrument panel, central control screen, rear entertainment screen, etc.), 3D navigation, voice assistant, camera, and gaming functions. These scenarios will not all operate at full load simultaneously. The intelligent cockpit system's SoC computing power is "designed for peak performance but operated at average performance," resulting in the cockpit system processor remaining idle for extended periods, leading to wasted computing power and redundant costs.

[0044] To address the imbalance in computing power between driver assistance systems (ADAS) and smart cockpit systems, methods for cross-domain computing power coordination can be explored. However, in current technologies, smart cockpit systems and ADAS are hardware-isolated, resulting in a rigid architecture. Cross-domain communication between the two systems is limited to signal interaction, failing to achieve real-time migration and collaboration of computing tasks. The lack of a secure, real-time, and efficient computing power coordination mechanism hinders the global optimization and dynamic emergency response of the vehicle's computing resources.

[0045] Furthermore, the driver assistance system cannot predict peak computing power demands in advance, leading to delays due to passive responses; the lack of a safety barrier between the driver assistance system and the intelligent cockpit system means that the computing power of the cockpit system, as a non-safe domain, cannot be accessed by the safe domain in a manner that meets functional safety requirements. All of these issues affect the balanced scheduling of computing power between the intelligent cockpit system and the driver assistance system.

[0046] To address the above issues, this application proposes a data processing method aimed at resolving the computing power bottleneck in driver assistance systems. This application's embodiments focus on the essence of the peak computing power bottleneck in driver assistance systems, rather than simply increasing chip configuration, aiming instead to build an intelligent computing ecosystem capable of sensing the environment and coordinating operations.

[0047] The data processing method provided in this application embodiment, with reference to Figures 1 and 2, includes the following steps.

[0048] S1: Calculates the computing power required for the main chip of the driver assistance system.

[0049] The calculation of step S1 is completed by the driver assistance system, or it can be completed by the main chip itself.

[0050] The computing power required by the main chip refers to the total computational load required by each computing unit to calculate and generate the vehicle's assisted driving strategy based on vehicle environmental data, vehicle driving data, and other factors under the current driving conditions. The computing power currently required by the main chip affects the computational level of the current vehicle assisted driving strategy.

[0051] Specifically, the main chip of a driver assistance system includes multiple computing units, and the computing power required by these units varies depending on the vehicle's driving conditions. For example, in low-speed congested urban conditions, the vehicle needs to start and stop frequently, resulting in a high frequency of perception updates, thus requiring high computing power from the main chip. In high-speed cruising conditions, the vehicle's driving environment is simple, with fewer surrounding targets, thus requiring lower computing power from the main chip.

[0052] When the main chip requires high computing power, the driver assistance system may have a risk of errors in the calculation of driving strategies.

[0053] S2: Determine whether there is a computing power gap in the main chip based on the peak computing power of the main chip and the computing power required by the processor.

[0054] The calculation of step S2 is completed by the driver assistance system, or it can be completed by the main chip itself.

[0055] Computing power gap refers to a state of insufficient computing resources in which the actual computing power required by the main chip of the driving assistance system (i.e. the instantaneous or peak computing power requirement calculated in step S1) exceeds the effective computing power resources that the main chip can currently provide, resulting in the system being unable to complete all computing tasks on time, which may lead to functional delays, degradation or even failure.

[0056] The computing power gap metric can quantify the computing reliability of the main chip. When there is a computing power gap, or the computing power gap is too large, the computing reliability of the main chip of the driving assistance system decreases.

[0057] S3: If yes, the main chip analyzes which driver assistance system tasks can be offloaded from all driver assistance system tasks. It should be understood that if no, the main chip of the driver assistance system continues to perform the current computational task.

[0058] Driving assistance systems have a large number of tasks, but not all tasks need to be executed by the main chip. Some tasks can be executed by other systems or computing units, which can still meet the normal functions of the driving assistance system.

[0059] In this embodiment, "unloadable driving assistance system tasks" refers to computational tasks among all tasks to be executed by the driving assistance system that can be executed by other systems or data computing units. Unloading these tasks can reduce the computational burden on the main chip of the driving assistance system, while still ensuring the normal operation of the driving assistance system.

[0060] S4: The main chip off the unloadable driving assistance system tasks and assigns them to the intelligent cockpit system to calculate and generate task execution strategies.

[0061] In order to optimize resource utilization, reduce power consumption, improve system robustness, or address the computing power gap of the main chip, this application proposes that some assisted driving system tasks can be "offloaded" to other computing units or dedicated hardware for execution.

[0062] S5: The main chip receives the task execution strategy from the intelligent cockpit system and executes it.

[0063] The main chip offloads driver assistance system tasks from the intelligent driving domain to the intelligent cockpit system for processing. The cockpit system then generates task execution strategies, which are fed back to the main chip for execution. This achieves "domain fusion" between the driver assistance system and the cockpit system, improving the reliability of driver assistance strategy generation while ensuring functional safety and data real-time performance.

[0064] In some embodiments of this application, the method for calculating the computing power required by the driver assistance system processor includes: calculating the current real-time level of the driver assistance system; predicting the scenario threat level of the driver assistance system in a set future time period, converting the scenario threat level into a predicted demand load, wherein the scenario threat level is used to characterize the impact of driving scenario complexity on intelligent driving strategies; taking the sum of the current real-time load and the predicted demand load as the predicted total load of the driver assistance system, and predicting the computing power gap of the driver assistance system based on the predicted total load of the driver assistance system and the computing power of the driver assistance system.

[0065] The current real-time load is used to characterize the weighted overall utilization rate of each computing unit (CPU, NPU, GPU, memory bandwidth) of the main chip of the driver assistance system, and to characterize the current computing power occupancy of the system.

[0066] Scene threat refers to a combination of potential hazards or events that may pose a risk to the functional safety, intended functional safety, or cybersecurity of an autonomous driving system in a specific traffic or environmental scenario. Examples include: an unexpected pedestrian suddenly darting out from behind an obstruction (ghost pedestrian), a camera's inability to recognize stationary obstacles under strong glare, the number and types of objects, and changes in weather conditions. The scene threat level can characterize the demand load on the driver assistance system over a future period.

[0067] The scenario threat level characterizes the impact of the complexity of the driving scenario on the computing power requirements of intelligent driving within a future set time period (such as 0.5s / 1s / 2s). Its value range can be set from 0 to 1. The larger the value, the more complex the scenario and the higher the computing power requirements.

[0068] In some embodiments of this application, the method for calculating the scene threat level is as follows.

[0069] Define environmental threat feature vectors The environmental threat feature vector includes multiple environmental threat parameters; the sum of the products of each environmental threat parameter and its corresponding weight is taken as the scene threat level.

[0070] Environmental threat feature vectors can be selected based on the actual operating environment and scenario characteristics of the vehicle. The parameters can be one or more.

[0071] For example, in the embodiments of this application, the environmental threat feature vector The settings are as follows:

[0072] The total number of targets within the sensing range; it should be understood that the size of the sensing range can be set according to requirements. The total number of valid targets (vehicles, pedestrians, non-motorized vehicles, obstacles, etc.) can be obtained by the sensing system (camera CPU utilization / CPU utilization / CPU utilization / CPU utilization / LiDAR fusion) mounted on the vehicle.

[0073] Target type diversity index; target diversity characterizes the richness of perceived target types. More types result in higher computational overhead (different targets require different detection CPU utilization / CPU utilization tracking models). Perceived target types include motor vehicles, non-motor vehicles, pedestrians, cyclists, cones, guardrails, etc. For example, the concept of entropy can be used to calculate the diversity index. Count the number of each target type within the current perception range (e.g., 150 meters ahead, 50 meters to the left and right). Calculate the proportion of each target type. Calculate the diversity index using the entropy formula and normalize it to the range of 0-1.

[0074] The weather impact coefficient is used to characterize the effect of weather on perception quality and computing power. Severe weather requires stronger computing power. In this embodiment, the impact coefficient is set according to the type of weather, for example: sunny weather. =0.1, heavy rain / dense fog =0.9; Road complexity reflects the impact of road geometry, traffic participant density, and rule uncertainty on the difficulty of driving decisions. To simplify processing, this embodiment uses road geometry to evaluate road complexity. Parameter values ​​are assigned to each road type, for example: straight road = 0.1, complex intersection = 0.9, curve = 0.5, roundabout = 0.7, etc.

[0075] : The countdown to the next critical driving decision (mapped to 0-1). This refers to the time remaining until the next critical driving decision. For example, the vehicle may need to turn at an intersection, change lanes, or brake due to an obstacle ahead. The timeframe for these critical decision points can be obtained through high-precision maps and navigation route planning. In threat assessment, it's generally believed that the shorter the time remaining until a critical decision, the higher the threat level; that is, the more urgent the situation, the greater the threat. If we directly use time (Time to Event), then the shorter the time, the smaller the value, but the greater the threat, which is the opposite of the increasing relationship with other indicators (such as the number of targets, weather impact, etc.). To maintain consistency (i.e., the larger the value, the greater the threat), we use a reciprocal form. Thus, when the time is shorter (urgent), the larger the reciprocal, the higher the threat level score.

[0076] In some embodiments of this application, a corresponding weight value is further set for each environmental threat parameter. The parameter values ​​for each environmental threat parameter are obtained based on the aforementioned steps. Normalization is performed using normalization parameters. And further calculate the threat level of the scenario. .

[0077]

[0078] For example: , , , , .

[0079] In some embodiments of this application, the scenario threat level is further obtained based on calculation. The value of the value is used to classify the threat level of a scenario, and a scenario risk classification mechanism based on quantitative indicators is proposed.

[0080] Specifically, the normalized threat level Mapped to four predefined threat level ranges: low threat level ( ):when At this time, it indicates that the current scenario structure is simple, the behavior of traffic participants is predictable, the environmental conditions are good, and the system is operating within a normal safety margin; medium threat level ( ):when At this time, the representation exhibits a certain degree of road complexity or dynamic uncertainty, requiring the maintenance of standard perception and decision-making performance; high threat level ( ):when When the scene is in an edge condition such as high interactivity or low visibility, the system should activate an enhanced perception strategy and prepare for human-machine collaboration; the critical threat level ( ):when At times, the characterization system faces extremely high safety risks (such as dense construction areas + strong glare + emergency intervention), and must immediately trigger degraded operation, minimum risk strategy or issue an emergency takeover request to the driver.

[0081] In driver assistance systems, higher hazard levels typically require greater computing power, stronger real-time performance, and higher redundancy. Threat level classification results are output to the vehicle's central decision-making module, functional safety monitoring unit, or human-machine interface system for dynamically adjusting control strategies, computing power allocation, or user prompt levels.

[0082] By combining the current load and scenario threat levels, we can predict future computing power gaps, anticipate insufficient computing power in advance, avoid latency exceeding limits for hard real-time tasks, improve system robustness and security, and provide a basis for dynamic task scheduling CPU utilization / CPU utilization model degradation CPU utilization / CPU utilization computing power reservation.

[0083] In some embodiments of this application, the method for calculating the current real-time load of the driving assistance system further includes the following steps.

[0084] Obtain the utilization rates of the central processing unit (CPU), neural network processor (NN), graphics processing unit (GPU), and memory bandwidth of the driver assistance system. Use the sum of two or more of these metrics, along with their corresponding weights, as the current real-time load of the driver assistance system.

[0085] As described in the aforementioned embodiments, the driver assistance system adopts an architecture consisting of a central processing unit (CPU), a neural network processing unit (NPU), and a graphics processing unit (GPU). Wherein: CPU utilization... This can also be called CPU average utilization, which is a positive number ranging from 0 to 1, and can be obtained by reading the operating system performance counters; it represents the utilization rate of the central processing unit. Assign CPU utilization weights Neural network processor utilization , is a positive number ranging from 0 to 1, obtained through a dedicated driver interface; represents the neural network processor utilization. Assigning neural network processor utilization weights Graphics processor utilization : A positive number between 0 and 1, obtained through the graphics driver API; : Graphics processor utilization. Assign graphics processor utilization weights Memory bandwidth utilization : A positive number ranging from 0 to 1, calculated using the memory controller performance counter; represents memory bandwidth utilization. Assign memory bandwidth utilization weights .

[0086] The current real-time load of the driving assistance system is calculated by using the sum of the products of two or more of the above four indicators and their corresponding weights.

[0087] For example, CPU utilization can be monitored. Weighted by CPU utilization The product of neural network processors and their utilization Weights related to neural network processor utilization The product of GPU utilization and GPU utilization weights Multiply the products, sum them up, and obtain the current real-time load of the driving assistance system. .

[0088] As a preferred implementation method, four metrics are used to calculate the current real-time load. Specifically:

[0089] In a specific embodiment, the values ​​of each weight coefficient are as follows: Central Processing Unit Utilization , , This reflects the differences in importance of each computing unit in ADAS tasks.

[0090] In some embodiments, the current real-time load can be... The system is categorized to determine the computing power usage of the driver assistance system. A low-load threshold is set. (Can be set to 0.4), Medium load threshold (Can be 0.65), High load threshold (Can be 0.8) and critical load threshold (0.9 can be used).

[0091] 0≤ <0.4: The driver assistance system is operating under low load, allowing for the addition of new tasks. CPU utilization / CPU frequency reduction / CPU utilization / CPU power reduction indicates sufficient computing power; 0.4≤ <0.65: The driver assistance system is operating normally under medium load; monitor trends; computing power is sufficient for computational needs; 0.65≤ <0.8: The driver assistance system is operating under high load, and it is necessary to consider limiting non-critical tasks; 0.8≤ <0.9: The driver assistance system is operating under high load. Given the current computing power configuration of ADAS, non-critical tasks need to be downgraded, such as: CPU utilization (AI CPU utilization / Rendering), ensuring CPU utilization (Perception CPU utilization / Control). 0.9≤ ≤1: The driver assistance system is operating under critical load, requiring mandatory safety measures to be implemented. Further non-critical tasks should be shut down to ensure hard real-time tasks are performed.

[0092] In some embodiments of this application, the method for converting scenario threat levels into predicted demand load includes the following steps.

[0093] Define threat load conversion factor This is used to reflect the contribution of the threat level to future load. The product of the scenario threat level and the threat load conversion coefficient is used as the predicted demand load for scenario threat level conversion.

[0094] The predicted demand load and the current real-time load of the driver assistance system are converted from the scenario threat level. The sum of these values ​​serves as the predicted load for driver assistance systems. .

[0095]

[0096] For example, A value of 0.25 can be used, which means "each unit of threat will increase the system load by an additional 25%".

[0097] By combining the current real-time load with the predicted demand load obtained from future threat predictions, the required computing power for borrowing is quantified, and an accurate borrowing request is given.

[0098] In some embodiments of this application, the method for calculating the computing power gap specifically includes the following steps.

[0099] The computing power gap is related to the processing power of the driver assistance system chip, and since the computing power gap usually occurs under high load, a load threshold is introduced. To calculate the computing power gap.

[0100] Specifically, the computing power gap is calculated according to the following formula.

[0101]

[0102] in, This indicates a computing power gap; This represents the nominal peak computing power (unit: TOPS) of the ADAS chip. It is an inherent performance parameter of the chip and is usually given in the chip datasheet. It is a known fixed value.

[0103] Load threshold The value can be set according to requirements, and can be calibrated based on factors such as the ADAS chip model, vehicle configuration, and actual test data. For ease of calculation, in some embodiments, a load threshold is used. A high load threshold can be used directly. (Can be 0.8) and critical load threshold (A value such as 0.9 can be used) is calculated.

[0104] In some embodiments of this application, the method for analyzing driver assistance system tasks that can be offloaded includes the following steps.

[0105] The driver assistance system sets evaluation metrics for each task, including computational intensity metrics, data locality metrics, and time criticality metrics. Based on any one of these metrics, or the sum of the products of any number of metrics and their corresponding weights, the system evaluates whether the task is suitable for offloading. Computational intensity refers to the computational workload of a task, typically represented by the number of computational operations required per unit of data processed (e.g., FLOPs). High computational intensity tasks require significant computational resources; offloading them to the cockpit domain can fully utilize the cockpit's parallel computing capabilities (e.g., NPU / GPU). Computational intensity is obtained by pre-determining the typical computational workload of each task through task analysis (e.g., model complexity analysis, code profiling) and normalizing it to the [0,1] interval. For example, the computational intensity of a lightweight model might be 0.3, while that of a large neural network model might be 0.9.

[0106] Data locality is used to characterize whether the data required for a task is already in cross-domain shared memory or the size of the task data migration overhead. A task with high data locality means that its input data already exists in a shared memory area accessible to both the ADAS and cockpit domains, reducing data transmission latency and bandwidth consumption. It is obtained by evaluating the data's location (private memory or shared memory) and size before task execution. For example, if the task input data is already in shared memory, then Data_Locality = 1.0; if the data needs to be copied from ADAS private memory to shared memory, a lower value is obtained by normalizing the copy time overhead.

[0107] Time-criticality metrics are used to characterize the sensitivity of a task to the impact of delays on driver assistance systems. Tasks with high time criticality must be completed within a very short timeframe (e.g., emergency collision avoidance algorithms) and are not suitable for offloading to the cockpit domain because cross-domain communication and additional scheduling introduce delays. Acquisition method: Determined based on the task's position in the ADAS function chain and its deadline requirements. For example, forward collision warning tasks have high time criticality (e.g., 0.9), while background map updates have low time criticality (e.g., 0.1).

[0108] In different embodiments, the suitability of a task to be offloaded can be evaluated based on any evaluation metric, or the sum of the products of any number of evaluation metrics and their corresponding weights. This determines whether a computational task is suitable for safe and efficient migration from the main autonomous driving chip to an external computing node.

[0109] For example, the suitability of a task for uninstallation can be evaluated based on time-criticality indicators; tasks with high time-criticality are not suitable for uninstallation. Thresholds can be set for each evaluation indicator, and the suitability of a task for uninstallation can be determined by comparing it to these thresholds.

[0110] For example, the suitability of an evaluation task to be unloaded can also be determined by the sum of the products of any number of evaluation indicators and their corresponding weights. For instance, two or three indicators can be selected for evaluation. As a preferred embodiment, in this application, three indicators are used to calculate whether the evaluation task is suitable to be unloaded.

[0111]

[0112] Evaluation metrics for whether a task can be uninstalled. A threshold can be set, and the calculation results, combined with the threshold, evaluate whether the task is suitable for uninstallation.

[0113] Computational intensity (0-1), high computational tasks are more suitable for offloading. Its weight.

[0114] Data locality (0-1): Tasks whose input data is already in shared memory are more suitable for unloading. Its weight.

[0115] Tasks with time criticality (0-1) and extreme sensitivity to latency are not suitable for uninstallation. Its weight.

[0116] For example, weighting coefficients: =0.5, =0.3, =0.2.

[0117] Furthermore, based on any evaluation metric, or the sum of the products of any number of evaluation metrics and their corresponding weights, the task set can be divided into a set of tasks suitable for uninstallation and a set of tasks unsuitable for uninstallation.

[0118] The perception system of the ADAS chip is defined as a perception pipeline, and all tasks of the ADAS chip are defined as a task graph. , : : Represents a set of task nodes, where each node represents an executable subtask (such as object detection, semantic segmentation, feature extraction, etc.).

[0119] : Represents the set of task-dependent edges, where each edge represents the dependency relationship between tasks (e.g., if the output of task A is the input of task B, then there is a directed edge from A to B).

[0120] : indicates each task.

[0121] Calculate the task weight for each task:

[0122] in, It is a comprehensive indicator, calculated based on multiple metrics including density, data locality, time-criticality, and local resource bottlenecks. For example, for each indicator, it can be normalized and then multiplied by its weight. The results of all project indicators are then summed to obtain the comprehensive evaluation index.

[0123] The tasks are sorted according to their weight. Based on the principle of maximum and minimum values, the task graph G is divided into two low-weight subgraphs. and high-weight subgraph This ensures that at least one of the following conditions is met: the total communication cost of the cut edge is minimized (reducing cross-device data transmission); the weight difference between the two parts is minimized (avoiding too little or too much offloading).

[0124] Only high-weight subgraphs The task marked as "to be unloaded" indicates that it can be transferred to the intelligent cockpit system for computation.

[0125] Since driver assistance systems and smart cockpit systems typically operate on different hardware platforms, operating systems, and security levels, the process of offloading tasks from driver assistance systems to smart cockpit systems requires consideration of cross-domain interaction security issues.

[0126] The communication protocol stack between the driver assistance system and the intelligent cockpit system is designed as follows: Application layer: loan request / response messages; a set of communication semantics and message formats defined at the application logic level to achieve temporary sharing of computing resources or task capabilities (i.e., "loan") between the two domains.

[0127] Security layer: AES-256-GCM encryption + HMAC-SHA256 authentication are used to prevent man-in-the-middle attacks, data tampering, and replay attacks.

[0128] Transport layer: Zero-copy data transmission based on shared memory, achieving high-speed bus with minimized communication overhead.

[0129] Physical layer: High-speed serial bus between domains (such as PCIe or 10G Ethernet), strictly meeting latency budgets at each stage.

[0130] Message encapsulation format: serializes task descriptions (such as subsets of the task graph) into standard messages.

[0131] The above communication protocol stack design enables secure, reliable, and low-overhead cross-domain task lending communication.

[0132] For example, the message header uses a lightweight binary protocol header: magic: 0xADAS2COCKPIT, used to quickly identify legitimate messages and filter noise or erroneous data; version: 0x10, used to indicate whether floating-point numbers, two bytes (major version.minor version), or BCD encoding are available; msg_type: REQUEST / RESPONSE / DATA, used to distinguish message types, including requests, responses, pure data streams, etc.; msg_length: used to indicate the length of the data portion; checksum: provides CRC32 verification, used to detect transmission or memory errors.

[0133] In some embodiments of this application, a resource scheduling arbitrator is used to achieve dynamic sharing of computing resources between the driver assistance system and the intelligent cockpit system. The resource scheduling arbitrator can be configured in the intelligent driver assistance system, the intelligent cockpit system, or an intermediate switching unit independent of both.

[0134] As the "scheduling brain" of the system, the resource scheduling arbitrator responds to resource lending requests from the driving assistance system. Based on the real-time available computing resource status on the cockpit side, the preset global scheduling strategy, and the requester's historical usage quota, it executes arbitration decisions on lending requests. The arbitration decisions of the driving assistance system include: approving all requested resources, rejecting the request, or approving some resources (i.e., degrading services). The global scheduling strategy includes at least one of the following mechanisms: a priority ranking mechanism based on security threat level, a time-sensitive decay factor, and single and cumulative computing power lending quota limits for each intelligent driving instance.

[0135] In some embodiments of this application, the main chip offloads the unloadable driver assistance system tasks to the intelligent cockpit system to calculate and generate task execution strategies. This also includes a resource scheduling step, specifically: determining the availability of intelligent cockpit system resources; the arbitrator first obtains the current available computing resource status of the intelligent cockpit system and determines whether its idle resources are below a preset minimum safety threshold. If idle resources are insufficient, the cockpit resources are determined to be unavailable, and the borrowing request for the driver assistance system task is immediately rejected to ensure the safe operation of the core functions of the cockpit system. If the cockpit resources are determined to be available, the system further determines how to respond to the borrowing request of the driver assistance system.

[0136] Provided that cockpit resources are available, the priority queue of driver assistance system tasks to be unloaded is calculated first.

[0137] Furthermore, the loan records of the driving assistance system tasks to be unloaded in the priority queue are analyzed. The loan records are used to characterize the impact of the task's past loan behavior on other tasks. The driving assistance system tasks to be unloaded that have compliant loan records and high priority are passed to the intelligent cockpit system for calculation.

[0138] Specifically, lending records can be viewed as a "credit system." A past lending activity of an ADAS task instance directly affects the approval results, available resource limits, and priority of its current and future requests. Its core impacts include: Fairness: Preventing a single instance from "monopolizing" resources, i.e., avoiding a situation where a particular ADAS function module task (such as surround-view perception) continuously consumes a large amount of cockpit computing power due to high load, leaving other critical functions (such as forward collision warning) without resources when truly needed.

[0139] Efficiency: Encourages "prudent" borrowing behavior. Specifically, it guides ADAS to more accurately predict computing power and break down tasks, requesting only necessary resources, reducing ineffective or excessive borrowing, and improving overall resource utilization. Observability: Provides data for system optimization. Specifically, long-term borrowing records are a data source for analyzing system load patterns, identifying "computing power starvation" modules, and optimizing global quota strategies.

[0140] In some embodiments of this application, the method for calculating the priority queue of driving assistance system tasks to be unloaded is as follows.

[0141] First, set the parameters used by the priority calculation function.

[0142] For example, the priority calculation function includes threat level. and timestamp .

[0143] Threat Level This refers to the level of danger of the current driving environment as assessed by the ADAS system. The value ranges from four predefined levels: "critical," "high," "medium," and "low." It directly determines the base score for priority.

[0144] Timestamp This refers to the moment (usually a time value with millisecond precision) when the ADAS system issues this computing power lending request. It's used to calculate the freshness of the request to ensure that new requests receive a timely response. For example, before the current request is issued, there may be other lending requests that have been issued but not yet responded to; in this case, the lending task will be queued for processing.

[0145] Subsequently, the internal variables and constants of the driver assistance system priority function are defined.

[0146] Base priority: An integer value obtained by looking up a table, reflecting the inherent priority weight corresponding to different threat levels. The mapping relationship is: critical=100, high=80, medium=50, low=20.

[0147] Current time refers to the system time when the arbitrator receives the request and executes the function calculation.

[0148] The time decay factor is a coefficient between 0 and 1. (From the formula...) This is calculated. Its function is that the longer the request wait time (...). The older the request, the smaller the factor, thus reducing its overall priority and preventing outdated requests from occupying resources for a long time.

[0149] 0.001: Decay coefficient, an empirical constant used to control the rate of time decay. The smaller this value, the slower the decay, and the more moderate the impact of time on priority.

[0150] Finally, calculate the final priority of the output.

[0151] Final dynamic priority value The calculation formula is:

[0152] This value takes into account both the severity of the threat (basic priority) and the urgency of the request (time decay). The higher the value, the higher the likelihood of approval in resource arbitration and the more resources may be allocated.

[0153] In some embodiments of this application, to prevent the driver assistance system from overusing the resources of the smart cockpit system and excessive resource lending between the two from affecting the data processing efficiency of the vehicle system, quota management is further implemented for the driver assistance system. A lending quota is set for each ADAS instance: Daily quota: =Driver Assistance System 1000 TOPS·seconds, used to represent the daily quota limit; Maximum loan per transaction: =Driver Assistance System 20 TOPS, used to represent the maximum loan limit per transaction; Excess Penalty: Priority is reduced by 50% after exceeding the limit, used to represent the excess penalty mechanism. This priority policy will be fed back for the judgment and adjustment of resource scheduling methods.

[0154] To ensure the security and real-time performance of data transmission between the intelligent driving assistance system and the intelligent cockpit system, some embodiments of this application further employ a security and timing assurance module. This module is used to ensure the dual core attributes of cross-domain computing tasks when executed between a secure domain (such as ADAS) and a non-secure domain (such as the cockpit): the driving assistance system prevents task code or data from being tampered with, stolen, or leaked; and the driving assistance system ensures that the task is completed before its deadline, meeting automotive-grade deterministic requirements.

[0155] The driver assistance system, acting as the initiator, calculates the task hash: HDriverAssistanceSystem = HAssistanceSystem SHA256(Task Code HAssistanceSystem + HAssistanceSystem Input Data); it then signs the hash using its private key: SigAssistanceSystem = HAssistanceSystem RSA_Encrypt(H, HAssistanceSystem Private Key); Upon receiving the task, the smart cockpit system, acting as the executor, decrypts SigAssistanceSystem using its public key to obtain H'; it then recalculates H_local = HAssistanceSystem SHA256(Received Code HAssistanceSystem + HAssistanceSystem Data); if H' = H_local, the task has not been tampered with and execution is permitted. Otherwise, a security risk is considered, and task execution is refused.

[0156] To ensure the intelligent cockpit system can successfully execute tasks upon receiving them, a timing guarantee algorithm was further designed. This algorithm aims to guarantee the reliable execution of cross-domain computational tasks (such as tasks offloaded from the intelligent driving system to the intelligent cockpit) under strict real-time constraints, meeting the stringent deadline requirements of automotive-grade systems for driver assistance systems. The overall process consists of four stages: feasibility prediction for the driver assistance system → time budget allocation for the driver assistance system → monitoring the execution process of the driver assistance system → dynamic acceleration intervention by the driver assistance system.

[0157] Phase 1: Estimating Execution Time and Assessing Feasibility. When a task is submitted for scheduling, the system first calculates the estimated execution time (estimated_time) under ideal conditions based on the estimated total computational workload (compute_cost) and the currently allocated NPU computing power (npu_power): estimated_time = compute_cost ÷ npu_power. compute_cost represents the total computational workload required by the task, for example, a task requiring 2.1 TOPS; npu_power represents the computing power of the NPU (unit: TOPS, i.e., trillions of operations per second); for example, an NPU with 10 TOPS. Dividing the two gives the required execution time (unit: seconds or milliseconds). During the estimated execution time calculation, it is necessary to ensure unit consistency. For example, if compute_cost is in TOPS (i.e.,...), the unit must be consistent. (number of operations), while the unit of npu_power is TOPS (i.e., ... If the operation is per second, then the unit of estimated_time is seconds.

[0158] If the estimated execution time exceeds the task deadline, the system will immediately determine that the task cannot be completed on time and return "Scheduling Failure" to avoid wasting resources on a task that is destined to time out.

[0159] Phase Two: Allocating Time Budget Proportionally If the estimated execution time of the task meets the time target, the execution time is further divided into three parts through a feasibility check: 60% for actual calculations (such as neural network inference); 20% reserved for data transmission (such as moving input / output data from the ADAS domain to the cockpit domain); and 20% reserved for system overhead (including task initialization, context switching, resource locking, etc.).

[0160] It should be understood that the above indicators are not limited to being set at 60%, 20%, or 20%, and can be set according to the actual application scenario.

[0161] This allocation method reflects the characteristics of real-world in-vehicle computing scenarios, where computing is the primary focus and communication and scheduling are secondary, providing clear time boundaries for each stage.

[0162] Phase 3: Runtime Execution Time Monitoring. After the task begins execution, the system continuously monitors its elapsed time, which is the difference between the task's start time and the current system time. This monitoring is typically implemented using timers or hardware performance counters in a real-time operating system (RTOS), rather than software-based waiting, to reduce scheduling overhead.

[0163] Phase Four: Triggering Dynamic Acceleration Mechanisms. During task execution, once the elapsed time exceeds 80% of the "computation budget" (e.g., if the total driver assistance system deadline is 5ms and the computation budget is 3ms, and execution exceeds 2.4ms), the system will immediately trigger an acceleration intervention mechanism to prevent eventual timeout. Possible acceleration strategies include: Dynamically increasing computing power: increasing the NPU's operating frequency and allocating more computing cores; Reducing computational precision: adjusting computational precision by modifying the computational model, for example, switching the model from FP16 to INT8, sacrificing a small amount of precision within an acceptable range for significant speedup; Progressive output: if the task supports it, returning the currently generated partial results in advance to meet real-time requirements.

[0164] To ensure the smooth execution of tasks offloaded to the intelligent cockpit system, a method for managing the computing resources of the cockpit system is further designed.

[0165] The driver assistance system tasks that can be offloaded are transferred to the computing resource area of ​​the intelligent cockpit system, which can be used for data processing of the intelligent cockpit system and the driver assistance system.

[0166] Based on the intelligent cockpit system's computing power configuration, current load, and predicted load, the available computing power for lending within the cockpit can be calculated. If a designated computing resource area is used for lending computing power, the available computing power can be further calculated for that computing resource area.

[0167] Within the limit of available computing power in the cockpit, computing power is allocated to tasks based on the priority of the driver assistance system tasks that can be offloaded.

[0168] The computing resource area configured in the intelligent cockpit system can be defined as a dynamic resource wall manager. Based on the priority of ADAS requests and the cockpit's own load prediction, it dynamically determines the upper limit of computing power that can be safely lent out and implements coordinated allocation on multi-dimensional resources such as CPU, NPU, memory, and bandwidth.

[0169] Based on the configuration of the intelligent cockpit system, the cockpit resources are modeled, and the intelligent cockpit system is equivalent to a multi-dimensional resource vector. Based on the cabin resource model, the current cabin load and the predicted load over a specified future time period, calculate the amount of cabin resources available for loan.

[0170] For example, multidimensional resource vectors It can be represented as:

[0171] This indicates the number of CPU cores, specifically the number of CPU cores in the cockpit SoC that can be used to handle general computing tasks. These cores typically employ a big.LITTLE architecture, including high-performance cores and energy-efficient cores.

[0172] : Indicates the computing power of the neural network processor, the peak computing power of the neural network processor (NPU) in the cockpit SoC, used to perform AI inference tasks (such as image recognition, speech recognition, etc.).

[0173] Graphics processor computing power refers to the floating-point computing capabilities of the graphics processor (GPU) in the cockpit SoC, used to handle tasks such as graphics rendering and parallel computing.

[0174] Memory capacity: The amount of system memory available to the cockpit SoC for storing program code, data, and intermediate results.

[0175] Memory bandwidth, the peak bandwidth of the cockpit SoC memory system, determines the speed of data read and write.

[0176] In some embodiments of this application, a computing power allocation method is also provided for multi-task computing power requests.

[0177] The intelligent cockpit system receives loan request information, which includes the computing power required for the task and the task priority.

[0178] For high-priority tasks, such as those involving proactive security scenarios: up to 80% of the available computing power can be allocated, but not exceeding the request amount. That is, for medium- and low-priority tasks, a maximum of 50% of the available computing power can be allocated.

[0179] In some embodiments of this application, in order to ensure that the intelligent cockpit system provides a safe and isolated environment for the driver assistance system to run ADAS tasks, the computing resource area of ​​the intelligent cockpit system is configured as a secure and isolated computing container to execute a security management method.

[0180] Secure, isolated computing containers are key execution units deployed in the intelligent cockpit domain to take over computing tasks offloaded from Advanced Driver Assistance Systems (ADAS). Their core function is to provide a hardware-level isolated, secure, reliable, and real-time controllable operating environment for ADAS tasks without interfering with existing cockpit functions (such as instrument clusters, entertainment systems, and HMIs).

[0181] The container runs on the cockpit SoC and uses physical resources such as CPU, NPU, and memory in the cockpit domain, but a dedicated area is designated through a resource wall mechanism.

[0182] In some embodiments of this application, a method for performance monitoring and evaluation of secure isolated computing containers is further provided. By fusing multi-dimensional performance indicators and employing anomaly-based circuit breaker strategies, the operating status of the container is dynamically quantified, providing a basis for resource scheduling, fault recovery, and security decisions. Specifically, the method includes the following steps.

[0183] Set performance testing indicators Assign weights to each indicator Set up a safe isolation system to calculate the health of the container. Evaluation model.

[0184]

[0185] In this embodiment of the application, performance testing indicators This includes: container CPU utilization, memory access latency, NPU utilization, and task completion time, with weights configured as: [0.3, 0.2, 0.3, 0.2]. This indicates that each indicator has been standardized.

[0186] The calculated value reflects the health status of the secure isolated computing container. The smaller the calculated value, the worse the health status, and the larger the calculated value, the better the health status.

[0187] In some embodiments, an error rate metric can also be added as a separate indicator. When the error rate exceeds a set threshold, the container health is assessed. It will be set directly to 0.

[0188] In some embodiments of this application, after receiving the transferred task, the intelligent cockpit system executes the task calculation according to the processing logic shown in Figure 4. The ADAS domain encapsulates the task into an encrypted task package and sends it to the intelligent cockpit via a secure channel (such as TLS / DTLS or a vehicle-specific encryption protocol). The cockpit side decrypts the task package using a pre-shared key or certificate, loads the corresponding neural network model from the local model repository according to the model ID specified in the task, calls the dynamic resource wall manager to create a secure and isolated computing container on the cockpit SoC, starts the inference engine within the isolated container, extracts the output results (such as target box coordinates and confidence scores) after inference is completed, encrypts the result data using a symmetric or asymmetric encryption algorithm negotiated with the ADAS domain, and returns the encrypted result to the ADAS domain via a secure communication channel (such as CANFD+TSN or Ethernet).

[0189] To ensure that the tasks of offloading the driver assistance system to the intelligent cockpit domain operate efficiently and reliably in a safe and isolated container, this application further proposes a collaborative optimization mechanism that integrates performance acceleration and intelligent fault tolerance, covering four core technology modules: model caching, computation graph reconstruction, memory management, and fault recovery.

[0190] This application proposes a preloaded model caching mechanism, which includes the following steps.

[0191] Model loading: Frequently used ADAS models (such as object detection and semantic segmentation networks) are preloaded into the intelligent cockpit system memory to avoid cold start delays during task execution; Model management: A cache update strategy combining Least Recently Used (LRU) and hot-weighted caching is adopted: the least recently used model is eliminated, and models with high relevance to high-threat scenarios and high call frequency are retained first.

[0192] This application proposes a computational graph depth optimization mechanism, which includes the following steps.

[0193] Operator fusion: Merging consecutive small operators (such as Conv+BN+ReLU) into a single kernel reduces the number of intermediate tensor generation and global memory accesses; Quantization acceleration: The intelligent cockpit system supports multi-model coordinated invocation and FP16 / INT8 mixed-precision inference, improving NPU throughput by 2–4 times while maintaining controllable accuracy loss; Pipeline parallelism: The intelligent cockpit system supports scheduling and rearranging of multi-task dependency graphs, enabling multi-task data overlapping loading, computation, and result feedback, maximizing hardware utilization.

[0194] This application proposes a memory efficiency management optimization mechanism, which includes the following steps.

[0195] Memory pool pre-allocation: Fixed-size memory blocks are allocated during container initialization to avoid fragmentation and latency jitter caused by dynamic allocation at runtime; Zero-copy data transfer: Through hardware shared memory, direct access to input / output data between ADAS and the cockpit is achieved, eliminating copy overhead; Computation-transmission overlap: Computation and data transfer are processed in parallel, effectively hiding communication latency.

[0196] Considering that the health of the secure isolated computing container may change, the operation of the secure isolated computing container is adjusted by calculating the number of errors.

[0197] Maintain an error_count counter for each secure isolated container, recording the cumulative number of errors (including failed retries) since the container started.

[0198] error_count: The total number of errors that have occurred in the current safe isolation container (this could be within a time period or since the container started). This count may include tasks that failed after retries. When the error count exceeds 3, the container is considered unhealthy.

[0199] mark_container_unhealthy: Marks the container as unhealthy. This means that the container will no longer be used to perform new ADAS unloading tasks until it is repaired or reinitialized.

[0200] migrate_tasks_to_backup: Migrate currently running tasks in this container to a backup container (or other healthy container) to ensure that tasks can continue to execute and avoid task loss.

[0201] In some embodiments of this application, a performance monitoring and recording module is further configured in the intelligent cockpit system as the "runtime perception hub" of the cross-domain elastic computing system, which is responsible for full-link, multi-granularity, and high-frequency performance monitoring and data recording of the collaborative execution process between the ADAS domain and the cockpit domain.

[0202] The performance monitoring and recording module is responsible for assessing the health and quality of the intelligent cockpit system. It evaluates the system's health score using multi-dimensional indicators.

[0203] The performance monitoring and recording module is configured with performance monitoring metrics, and each metric is assigned a weight. The sum of the products of each metric and its corresponding weight is used as the Quality of Service (QoS) metric.

[0204] QoS = d × Success_Rate + e × 1 / Latency_P99 + f × Resource_Efficiency Success_Rate: The percentage of tasks that are successfully executed, i.e., the percentage of ADAS offloading tasks that are successfully completed in the cockpit domain and return the correct results.

[0205] Latency_P99: The 99th percentile of task execution delay, meaning that 99% of task completion times do not exceed this value, reflecting the real-time performance of the system.

[0206] Resource_Efficiency: Resource utilization efficiency, which is the proportion of resources actually used for effective computation out of the allocated resources, to avoid resource waste.

[0207] QoS metrics are used to comprehensively evaluate the overall performance of a cross-domain computing power lending system, balancing three key dimensions: task success rate, real-time performance, and resource efficiency. Quantitative evaluation can guide system optimization, such as adjusting resource allocation strategies and improving task offloading decisions, to ensure efficient utilization of cockpit computing resources while guaranteeing ADAS safety performance.

[0208] The system health score is a comprehensive, high-level operational status indicator used to evaluate the overall operational quality, stability, and efficiency of the entire cross-domain computing power collaboration system in real time. In this embodiment, it includes: Quality of Service (QoS) indicator: measuring the degree to which the system's core business objectives are achieved. In this system, the core business is "providing computing power support for ADAS safely and promptly." Error rate indicator: measuring the stability and reliability of system operation. It reflects the frequency of various anomalies and failures occurring during the collaboration process.

[0209] Resource balance is an evaluation metric that measures the rationality and sustainability of system resource utilization. Ideally, ADAS receives the necessary computing power while cockpit resources are not excessively squeezed out, with both systems operating within an efficient and balanced load range.

[0210] The system health score (health_score) is calculated using a defined calculation function f.

[0211] In some embodiments of this application, the intelligent cockpit system is further configured with a strategy optimization and parameter tuning module. Based on runtime data collected by the performance monitoring module, it adopts a reinforcement learning-driven online parameter optimization mechanism to dynamically adjust key strategy parameters such as load prediction, resource arbitration, and task scheduling, thereby improving the quality assessment indicator QoS.

[0212] The system constructs a state space based on system monitoring indicator vectors; configures adjustable system parameters; constructs a reward function based on the difference between QoS improvement and resource cost; and updates adjustable parameters daily based on intelligent learning algorithms to improve the performance of the intelligent cockpit system.

[0213] In some embodiments of this application, the system monitoring indicator vector should include indicators that can comprehensively reflect the system's operating status, which can be divided into several categories: performance indicators: task processing latency (including P50, P95, P99), task success rate, task throughput (TOPS / sec), resource utilization (CPU, NPU, GPU, memory, bandwidth), etc.

[0214] Safety metrics include: ADAS task deadline miss rate, delays in core cockpit domain functions (such as instrument refresh delays), and system error rates (such as communication errors and calculation errors).

[0215] Resource metrics: Current idle resources in the cockpit domain (TOPS, memory), current load in the ADAS domain, cross-domain communication bandwidth utilization, etc.

[0216] Environmental indicators: ADAS threat level, weather conditions, road complexity, etc.

[0217] In some embodiments of this application, the calculation of resource costs considers the following parameters: the increase in latency of the cockpit domain's own tasks (e.g., if the instrument rendering latency increases from 16ms to 20ms, the cost is 4ms). The proportion of cockpit domain tasks that are degraded due to insufficient resources (e.g., the navigation map is downgraded from 3D to 2D). The utilization rate of the resource wall manager (the higher the utilization rate, the higher the cost, because the smaller the remaining resource buffer).

[0218] Key adjustable parameters include: load forecasting parameters, arbitration strategy parameters, and resource allocation parameters.

[0219] Load prediction parameters specifically include: threat conversion coefficient k and environmental threat parameter feature weights. i. Thresholds used to evaluate the current real-time load. Adjusting these parameters can improve the load prediction level of the assisted driving system; arbitration strategy parameters specifically include: priority weight: inherent priority weights corresponding to different threat levels; quota limit: dynamically adjusting the daily quota based on historical data analysis of borrowing demand at different times of the day. For example, the quota can be appropriately increased during morning and evening peak hours; adjusting the maximum single borrowing amount based on the success rate of ADAS tasks and cabin resource utilization; safety margin coefficient: adjusting the safety margin coefficient based on actual cabin task experience indicators (such as the number of stutters, response latency, utilization rate, etc.). Indicators are divided into positive and negative indicators. Under the premise of negative indicators, such as if the cabin task stutters, the safety margin coefficient is increased to retain more resources; under the premise of positive indicators, if the cabin resource utilization rate is low, the safety margin coefficient is decreased to increase the resource lending amount.

[0220] Resource allocation parameters include: container resource configuration: including resource allocation for secure isolated computing containers (such as CPU core binding, NPU computing unit partitioning, memory regions, etc.), and specific resource configurations (such as npu_cores, memory_mb, etc.).

[0221] Preloading strategies include preloading model caching strategies, etc.

[0222] Error recovery threshold: Adjust the health evaluation system for the marked container by adjusting error_count and migrate the task.

[0223] System performance can be optimized by maximizing the objective function.

[0224]

[0225] : No. The Quality of Service (QS) metric for each task. It is a value between 0 and 1, representing the level of satisfaction with task completion. The calculation can be based on factors such as task completion time and accuracy. For example, for an ADAS unloading task, It can be defined as the probability that a task is completed within the deadline and the accuracy of the result meets the requirements.

[0226] : No. The priority of each task is determined by its safety criticality and urgency, such as the priority value calculated in the arbitrator.

[0227] Through the above optimization objectives, the overall service quality of the system is maximized while considering task priorities. Specifically, the service quality of high-priority tasks is guaranteed first.

[0228] The technical effects of the computing power lending method between the driving assistance system and the intelligent cockpit system provided in this application, and the independent data processing method of each system in the traditional technology are compared in Table 1.

[0229] Table 1 Comparison of the technical effects of driver assistance systems

[0230] As can be seen, the data processing method provided in this application embodiment can maintain the ADAS perception frame rate at more than 28fps in extreme or sudden scenarios (traditional solutions may drop to 15fps), reduce decision latency (P99) by more than 50%, significantly improve driving safety and system determinism; increase the average computing power utilization of the cockpit SoC from 30%–40% to 60%–70%, and transform long-term idle computing resources into effective computing power; and cope with high-load scenarios without upgrading the ADAS main chip, saving 20%–30% in hardware costs.

[0231] The second embodiment of this application provides a data processing system that relies on a driver assistance system and a smart cockpit system.

[0232] The logical structure of the driver assistance system is shown in Figure 3.

[0233] Multi-sensor module: Used to acquire raw data from sensors such as cameras, radar, lidar, IMU, maps, and V2X to obtain multi-modal environmental information in the vehicle operation scenario.

[0234] Real-time load assessment submodule: used to calculate the current real-time load of the driver assistance system.

[0235] Scene Threat Prediction Submodule: Used to predict the scene threat level of the driver assistance system within a set future time period, and convert the scene threat level into predicted demand load.

[0236] The computing power gap calculation submodule converts the current real-time load and scenario threat level into the sum of the predicted demand load as the predicted load of the driving assistance system. Based on the predicted load of the driving assistance system and the computing power of the driving assistance system, it predicts the computing power gap of the driving assistance system.

[0237] Lending Demand Output Module: Used to output lending demands to the intelligent cockpit system based on the computing power gap of the driver assistance system. It transfers driver assistance system tasks that can be offloaded to the intelligent cockpit system for computation.

[0238] Input module: Used to obtain the calculation results of the intelligent cockpit system and feed back the values ​​to the chip execution of the driver assistance system.

[0239] The scenario threat prediction submodule is further configured to: define a threat load conversion coefficient, and use the product of the scenario threat level and the threat load conversion coefficient as the predicted demand load for scenario threat level conversion.

[0240] The scenario threat prediction submodule is further configured to: set an environmental threat feature vector, which includes multiple environmental threat parameters; and use the sum of the products of each environmental threat parameter and its corresponding weight as the scenario threat level.

[0241] The real-time load assessment submodule is further configured to use two or more of the following indicators as the current real-time load of the driving assistance system: central processing unit utilization, neural network processor utilization, all graphics processor utilization, and memory bandwidth utilization, and the sum of the products of the corresponding weights of the indicators.

[0242] In some embodiments of this application, the data processing system further includes a cross-domain secure communication module. This module is used to achieve secure, reliable, and low-latency communication between the driver assistance system and the arbitration layer of the smart cockpit system, and is responsible for the encapsulation, encryption, and transmission of loan requests.

[0243] In some embodiments of this application, the data processing system further includes a cross-domain arbitration layer module for implementing a standardized cross-domain computing power lending protocol, including connection management, session maintenance, heartbeat detection, and error recovery.

[0244] In some embodiments of this application, the data processing system further includes a resource scheduling arbitrator for realizing dynamic computing resource sharing between the driver assistance system and the intelligent cockpit system.

[0245] In some embodiments of this application, the data processing system further includes a security and timing guaranteeer for encrypting and decrypting cross-domain data.

[0246] In some embodiments of this application, the intelligent cockpit system includes a computing power supply-side module, which uses a resource wall mechanism to delineate a dedicated area for driver assistance system task computing, and executes an intelligent cockpit domain computing power allocation strategy according to task priority to achieve elastic resource allocation.

[0247] In some embodiments of this application, the intelligent cockpit system includes a securely isolated computing container that provides a hardware-level isolated execution environment for ADAS tasks.

[0248] The third embodiment of this application provides a vehicle. Referring to FIG5, the vehicle provided in this embodiment includes a driver assistance system and an intelligent cockpit system. The two are interconnected through a high-speed vehicle communication bus (such as Ethernet TSN or PCIe) and jointly build a vehicle-level elastic computing platform at the software and hardware levels.

[0249] Driving assistance system: Configured on a separate ADAS chip, responsible for performing core autonomous driving tasks such as environmental perception, decision planning, and control execution; it has the ability to predict computing power gaps: it can collect CPU / NPU / GPU / memory bandwidth utilization in real time; predict the complexity of future scenarios based on multi-dimensional environmental threat feature vectors (including weights); estimate future computing power requirements by combining threat-load conversion coefficients; and generate computing power lending requests when the predicted local computing power is insufficient to meet the demand.

[0250] Intelligent cockpit system: Deployed in the cockpit chip system; originally used to run applications such as instrument display, infotainment, and voice interaction; in this embodiment, it further integrates the calculation function of handing over tasks from the driver assistance system.

[0251] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute the data processing method involved in the above embodiments. This computer program can be installed in a vehicle system.

[0252] This application also provides a computer-readable storage medium storing program code that is executed by one or more processors. When the program code runs on the processor, it causes a device including one or more processors to perform the data processing method described in the above embodiments. The processor running this computer-readable storage medium can be mounted in a vehicle system.

[0253] It should be understood that when the modules or units described herein are implemented using software, they can be implemented in whole or in part as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0254] This application provides a chip system including a processor, or the chip system including a memory and a processor, for calling computer programs or computer instructions stored in the memory to cause the processor to execute the data processing method involved in the above embodiments. The chip system can be a single chip or a chip module composed of multiple chips. This chip system can be installed in a vehicle system.

[0255] This application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the data processing method described in the above embodiments. This electronic device can be installed in a vehicle system.

[0256] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, Includes the following steps: Calculate the computing power required by the main chip of the driving assistance system; based on the peak computing power of the main chip and the computing power required by the main chip, determine whether there is a computing power gap in the main chip; If so, the main chip analyzes the driver assistance system tasks that can be unloaded from all driver assistance system tasks; the main chip unloads the unloadable driver assistance system tasks to the intelligent cockpit system to calculate and generate task execution strategies; The main chip receives the task execution strategy fed back by the intelligent cockpit system and executes it.

2. The data processing method according to claim 1, characterized in that, The calculation of the computing power required by the main chip of the driving assistance system includes: calculating the current real-time load of the driving assistance system; predicting the scenario threat level of the driving assistance system in a set future time period, converting the scenario threat level into a predicted demand load, wherein the scenario threat level is used to characterize the impact of driving scenario complexity on intelligent driving strategy; taking the sum of the current real-time load and the predicted demand load as the predicted total load of the driving assistance system; and calculating the required computing power based on the predicted total load.

3. The data processing method according to claim 2, characterized in that, The method for converting the scenario threat level into predicted demand load includes: defining a threat load conversion coefficient, and using the product of the scenario threat level and the threat load conversion coefficient as the predicted demand load for scenario threat level conversion.

4. The data processing method according to claim 2 or 3, characterized in that, The method for predicting the scene threat level of the driving assistance system within a set future time period includes: setting an environmental threat feature vector, which includes multiple environmental threat parameters; and using the sum of the products of each environmental threat parameter and its corresponding weight as the scene threat level.

5. The data processing method according to claim 2, characterized in that, The calculation of the current real-time load of the driving assistance system includes: obtaining the utilization rate of the central processing unit, the neural network processor, the graphics processor, and the memory bandwidth utilization of the driving assistance system; and using the sum of the products of two or more of the central processing unit utilization, the neural network processor utilization, the graphics processor utilization, and the memory bandwidth utilization, and their corresponding weights, as the current real-time load of the driving assistance system.

6. The data processing method according to claim 1, characterized in that, The method for analyzing driver assistance system tasks that can be offloaded includes: setting evaluation metrics for each task, the evaluation metrics including computational intensity metrics, data locality metrics, and time criticality metrics; evaluating whether a task is suitable for offloading based on any one evaluation metric, or the sum of the products of any number of evaluation metrics and their corresponding weights; wherein, computational intensity refers to the computational amount required to measure the task; the data locality metrics are used to characterize whether the data required by the task is already located in cross-domain shared memory or the amount of task data migration overhead; the time criticality metrics are used to characterize the sensitivity of the task to the impact of latency on the driver assistance system.

7. The data processing method according to claim 1, characterized in that, The main chip unloads the unloadable driver assistance system tasks to the intelligent cockpit system for calculating and generating task execution strategies. This also includes: determining the availability of intelligent cockpit system resources; if not, rejecting the unloading request for the driver assistance system tasks to be unloaded; if so, calculating a priority queue of the driver assistance system tasks to be unloaded; analyzing the borrowing records of the driver assistance system tasks to be unloaded in the priority queue, where the borrowing records characterize the impact of the task's past borrowing behavior on other tasks; and passing the driver assistance system tasks with compliant borrowing records and high priority to the intelligent cockpit system for calculation.

8. The data processing method according to claim 1 or 7, characterized in that, The main chip offloads the unloadable driving assistance system tasks to the intelligent cockpit system to calculate and generate task execution strategies. This also includes: transferring the unloadable driving assistance system tasks to the computing resource area of ​​the intelligent cockpit system, where the computing resource area can be used for data processing of both the intelligent cockpit system and the driving assistance system; calculating the available computing power of the cockpit resources based on the intelligent cockpit system's computing power configuration, real-time load, and predicted demand load; and allocating different proportions of computing power to tasks of different priorities within the limited range of available computing power available in the cockpit resources.

9. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores program code that is executed by one or more processors. When the program code is run on a processor, it causes an apparatus including one or more processors to perform the data processing method described in the above embodiments.

10. A vehicle, characterized in that, It is capable of performing the data processing method according to any one of claims 1 to 8.

Images