An automatic driving domain-oriented dynamic computing power scheduling method for a car intelligent chip

Abstract

The application discloses a kind of car intelligent chip dynamic computing power scheduling methods for automatic driving domain, it is related to automatic driving chip scheduling technical field, including collecting automatic driving vehicle multimodal environment perception data and vehicle ontology state data, and is formed synchronous environment state stream after preprocessing alignment, assesses road scene complexity and identifies computing power demand key elements, determines the computing resource category and intensity of each functional subtask;Real-time state snapshot is formed by obtaining chip dispensable computing power, computing unit utilization and thermal state parameters, generate global scheduling instruction set including task mapping, computing power allocation proportion and execution timing and execute, then according to task execution progress and unit load on-line fine tuning to scheduling parameter, form closed-loop control.The method can accurately match the scenario computing power demand, let computing power allocation fit chip real-time running state, realize the dynamic adaptation of heterogeneous computing unit computing power, improve the cooperativity of scheduling and task execution.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving chip scheduling technology, specifically a dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain. Background Technology

[0002] Existing automotive intelligent chip computing power scheduling technologies for the autonomous driving domain typically process multimodal environmental perception data and vehicle body state data separately, without preprocessing or aligning these two types of data, thus failing to form a synchronized and unified environmental state flow. The assessment of road scene complexity lacks complete and consistent data support, making it difficult to accurately identify key scene elements that determine computing power requirements. The types and intensity of computing resources corresponding to each autonomous driving function sub-task cannot be accurately quantified and classified. Chip scheduling only collects scattered operating parameters, failing to integrate total available computing power, real-time utilization of computing units, and thermal state parameters to form a real-time state snapshot. Scheduling instructions only complete basic task allocation, without constructing a global dynamic scheduling instruction set that includes task mapping, computing power allocation ratios, and execution timing.

[0003] Existing computing power scheduling mostly adopts an open-loop execution mode. After the scheduling command is issued, it cannot fine-tune the computing power allocation ratio and execution timing online based on the progress of computing tasks and the actual load of computing units. The computing power configuration of heterogeneous computing units cannot dynamically adapt to changes in scenarios and tasks. The computing power allocation results are difficult to match with the actual operating status of chips and real-time task requirements. The entire scheduling process lacks a closed-loop control mechanism.

[0004] This invention aims to achieve synchronous alignment of multimodal data and vehicle status data, identify key elements of the scenario based on the synchronous data and determine the computing power requirements of sub-tasks, integrate multi-dimensional operating parameters of the chip to form a real-time status snapshot, generate a global scheduling instruction set, and fine-tune the scheduling parameters in real time based on task execution and load conditions to establish a closed-loop dynamic computing power scheduling method. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art;

[0006] To this end, the present invention proposes a dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain, comprising:

[0007] Collect multimodal environmental perception data and vehicle body state data of autonomous vehicles, and preprocess and align the collected multimodal environmental perception data and vehicle body state data to form a synchronous environmental state stream;

[0008] Based on the synchronous environmental state stream, the complexity of road scenarios in the current and predicted future periods is assessed, and key scenario elements that have a decisive impact on computing power requirements are identified.

[0009] Based on the identified key scene elements, determine the types and intensity of computing resources required to execute different autonomous driving function sub-tasks;

[0010] The system obtains the current total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters to form a real-time state snapshot of the chip.

[0011] Based on the required computing resource type, computing intensity, and real-time chip status snapshot, a global dynamic scheduling instruction set is generated, which includes computing task mapping, computing power allocation ratio, and execution timing.

[0012] The system executes a global dynamic scheduling instruction set to allocate and start corresponding computing tasks on multiple heterogeneous computing units of the smart chip, while monitoring the execution progress of the computing tasks and the actual load of the computing units.

[0013] Based on the execution progress of the computing tasks and the actual load of the computing units, the computing power allocation ratio and execution timing in the global dynamic scheduling instruction set are fine-tuned online to form a closed-loop control.

[0014] Furthermore, the collected multimodal environmental perception data and vehicle body state data are preprocessed and aligned to form a synchronized environmental state stream, specifically including:

[0015] It receives raw sensing data from lidar, cameras, and millimeter-wave radar, and performs independent timestamp marking and data integrity verification for each type of raw sensing data.

[0016] For image data from cameras, distortion correction and noise filtering are performed; for point cloud data from lidar, outlier removal and motion distortion compensation are performed; and for data from millimeter-wave radar, clutter suppression is performed.

[0017] For various types of sensory data that have been processed independently, spatial registration and temporal synchronization are performed based on a unified world coordinate system and time reference, so that the information describing the same object is aligned in space and time.

[0018] The vehicle's speed, acceleration, steering angle, and position information are acquired, and then fused with spatiotemporally aligned multimodal perception data.

[0019] The fused data stream is segmented and encapsulated according to a preset time slice length. Each data packet contains all the sensing and state information within a complete time slice, and the final output is a synchronized environmental state stream.

[0020] Furthermore, the method based on synchronized environmental state flow assesses the complexity of road scenarios in the current and predicted future periods, identifying key scenario elements that decisively influence computing power requirements, specifically including:

[0021] Extract the number, type, speed, and trajectory prediction uncertainty measure of dynamic traffic participants from the synchronized environmental state stream;

[0022] Extract the density and type of static traffic facilities, as well as the current state and change patterns of traffic lights, from the synchronized environmental state stream;

[0023] Extract the road's geometric topology, lane line clarity, and current weather and lighting conditions from the synchronized environmental state stream;

[0024] By integrating dynamic traffic participant information, static traffic facility information, and road environment information, the scene complexity score at the current moment is calculated through a scene understanding model, and the scene complexity scores for multiple future time slices are predicted based on the current vehicle motion state.

[0025] The moment slice where the scene complexity score exceeds the preset threshold is identified as the key scene element in the corresponding synchronous environmental state stream that dominates the increase in complexity. The key scene elements include dense pedestrian groups with unpredictable trajectories, complex intersections without lane lines, and reduced visibility under extreme weather conditions.

[0026] Furthermore, based on the identified key scene elements, the determination of the types and intensity of computing resources required to execute different autonomous driving function sub-tasks specifically includes:

[0027] Establish a mapping relationship library between key scene elements and autonomous driving function sub-tasks, wherein the autonomous driving function sub-tasks include target detection, semantic segmentation, target tracking, trajectory prediction, path planning, and behavior decision-making;

[0028] Based on the mapping relationship library, for each key scene element identified in the current and predicted time period, a list of autonomous driving function subtasks that need to be activated or enhanced is matched.

[0029] For each subtask in the list of autonomous driving function subtasks that need to be activated or enhanced, query its computing resource requirement model under different computing accuracy and real-time requirements. The computing resource categories include scalar computing units, vector computing units, matrix computing units, and customized accelerators.

[0030] From the computational resource requirement model obtained from the query, select the computational precision and real-time configuration that matches the security level of the current key scenario elements, thereby determining the type of computational resources and the expected value of computational intensity required for each subtask.

[0031] Furthermore, the acquisition of the current total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters to form a real-time state snapshot of the chip specifically includes:

[0032] Periodically read the real-time operating frequency and utilization rate of the central processing unit core, image processing unit core, and neural network processing unit core of the intelligent chip from the internal performance counters;

[0033] The real-time temperature values ​​of different computing unit areas of the smart chip are periodically read from the internal temperature sensor of the chip.

[0034] Based on the real-time operating frequency, peak computing power and utilization rate of the central processing unit core, image processing unit core, and neural network processing unit core, calculate their current actual available computing power and sum them up to obtain the current total available computing power.

[0035] Record the average utilization rate and utilization trend of each computing unit over the past time window;

[0036] The current available total computing power, the real-time utilization rate, average utilization rate, utilization rate trend, and real-time temperature value are encapsulated into a structured data packet as a real-time snapshot of the chip's status.

[0037] Furthermore, the step of generating a global dynamic scheduling instruction set, which includes computing task mapping, computing power allocation ratio, and execution timing, based on the required computing resource type, computing intensity, and real-time chip status snapshot, specifically includes:

[0038] Based on the required computing resource category, a set of candidate computing units that meet the resource category, have a current utilization rate below the safety threshold, and have a temperature within the normal range are selected from the chip's real-time status snapshot;

[0039] Based on the expected computational intensity and the actual available computing power of each unit in the set of candidate computing units, the computational load of the subtask is distributed to multiple candidate computing units, forming a mapping relationship between computational tasks and physical computing units.

[0040] For each physical computing unit assigned a task, the computing power allocation ratio to be used is calculated based on its assigned load and actual available computing power.

[0041] Considering the data dependencies between subtasks, computation latency requirements, and data communication bandwidth between computing units, a conflict-free execution sequence is planned for all scheduled computing tasks to ensure that dependent tasks are executed in sequence and high-priority tasks obtain resources first.

[0042] The computational task mapping relationship, computing power allocation ratio and execution timing are encoded into a set of machine-executable instructions to form a global dynamic scheduling instruction set.

[0043] Furthermore, the execution of the global dynamic scheduling instruction set allocates and initiates corresponding computing tasks on multiple heterogeneous computing units of the intelligent chip, while monitoring the execution progress of the computing tasks and the actual load of the computing units, specifically including:

[0044] The global dynamic scheduling instruction set is parsed, and the executable code and input data of the subtasks are loaded into the local memory of the specified heterogeneous computing unit according to the computing task mapping relationship therein.

[0045] Based on the computing power allocation ratio in the global dynamic scheduling instruction set, configure the operating frequency, voltage, and number of concurrent threads of the corresponding heterogeneous computing units;

[0046] According to the execution sequence of the global dynamic scheduling instruction set, task start signals are sent to each heterogeneous computing unit in sequence, and the system monitors whether the task is started correctly within the preset latest start time.

[0047] During the execution of the computing task, the performance counters of each computing unit are periodically read to obtain the percentage of the workload that has been completed to the total workload, which is used as the execution progress of the computing task.

[0048] During the execution of computing tasks, the instantaneous utilization rate of each computing unit and the queue length of the queued tasks are collected in real time as the actual load of the computing unit.

[0049] Furthermore, the online fine-tuning of the computing power allocation ratio and execution timing in the global dynamic scheduling instruction set based on the execution progress of the computing task and the actual load of the computing unit, forming a closed-loop control, specifically includes:

[0050] The execution progress of the computation tasks is compared with the preset schedule to identify the computation tasks that are lagging behind and the computation units in which they are located.

[0051] Analyze the actual load of the computing unit where the delayed computing task is located to determine whether the delay is due to excessive load of the computing unit or due to delay of the preceding data-dependent task.

[0052] If the problem is caused by excessive load on the computing unit, the system will search for the same type of computing unit with low utilization in the chip's real-time status snapshot, migrate part of the load of the lagging tasks to the same type of computing unit, and update the computing task mapping and computing power allocation ratio.

[0053] If the delay is caused by the preceding data dependency task, the execution sequence of the global dynamic scheduling instruction set will be dynamically adjusted without violating the core data dependency, and the preceding dependency task will be scheduled first or more computing resources will be allocated to it.

[0054] The instructions for adjusting the computing power allocation ratio and execution timing are sent to the corresponding computing units in real time, replacing some of the content in the original instruction set, thereby realizing closed-loop control based on execution feedback.

[0055] Furthermore, for each subtask in the list of autonomous driving function subtasks that need to be activated or enhanced, the computational resource requirement model under different computational precision and real-time requirements is queried, specifically including:

[0056] An offline performance analysis dataset is pre-established for each autonomous driving function subtask. The offline performance analysis dataset contains records of the execution time and resource consumption of the subtask on various computing units under different input data scales, different computational precisions and different real-time constraints.

[0057] Based on the offline performance analysis dataset, a computing resource demand model is trained. The computing resource demand model takes input data characteristics, computing accuracy requirements, and real-time requirements as inputs, and outputs the theoretical computing intensity and memory usage on different types of computing units.

[0058] When it is necessary to determine the resource requirements of a subtask, the input data features of the current scenario are extracted from the synchronized environment state stream, and combined with the computational precision and real-time requirements corresponding to the security level determined from the mapping relationship library, and used as the input of the computational resource requirement model.

[0059] By running the computational resource requirement model, the expected computational intensity and memory usage of the subtask on the scalar computation unit, vector computation unit, matrix computation unit, and customized accelerator are obtained respectively.

[0060] Furthermore, the step of migrating a portion of the load of the delayed tasks to the same type of computing unit and updating the computing task mapping and computing power allocation ratio specifically includes:

[0061] Identify the independent computational sub-blocks that can be divided in the delayed computational task, and evaluate the data transfer overhead required to migrate the independent computational sub-blocks to the target computational unit of the same type;

[0062] Compare the data transfer overhead with the computational speedup benefits after migration. If the benefits outweigh the overhead, select an appropriate number of computational sub-blocks to migrate.

[0063] The selected computational sub-block and its associated input data are transferred from the local memory of the source computational unit to the local memory of the target computational unit of the same type.

[0064] Modify the computation task mapping relationship in the global dynamic scheduling instruction set and record the mapping relationship between the migrated computation sub-blocks and the original computation tasks on the target computation unit;

[0065] Based on the changes in load on the source and target computing units after migration, the computing power allocation ratio of these two computing units in the new scheduling cycle is recalculated and updated.

[0066] Compared with the prior art, the beneficial effects of the present invention are:

[0067] Multimodal environmental perception data and vehicle body state data are preprocessed and aligned to form a synchronized environmental state stream. Based on the synchronized environmental state stream, the complexity of the road scene in the current and future time periods is assessed, and key scene elements that have a decisive impact on computing power requirements are identified. The types and intensity of computing resources required for different autonomous driving function sub-tasks are divided according to the key scene elements. The synchronous processing of data ensures that the reference basis for scene complexity assessment remains consistent. The identification of key scene elements eliminates the interference of irrelevant parameters on the determination of computing power requirements. The division of computing resource categories and computing intensity directly matches the actual computing power requirements of the scene, ensuring that the determination of computing power requirements is consistent with the changes in the dynamic autonomous driving scene.

[0068] The system collects the total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters to form a real-time snapshot of the chip's status. Combining the computing resource type, computing intensity, and real-time chip status, it generates a global dynamic scheduling instruction set that includes computing task mapping, computing power allocation ratio, and execution timing. After executing the scheduling instructions, the system fine-tunes the computing power allocation ratio and execution timing online based on the computing task execution progress and the actual load of the computing unit. The integration of the chip's multi-dimensional operating status ensures that the generation of scheduling instructions is aligned with the chip's real-time operating conditions. The global scheduling instructions achieve precise matching between heterogeneous computing units and computing tasks. The online fine-tuning operation allows the scheduling parameters to adapt to the real-time changes in task execution. The closed-loop scheduling logic allows the computing power allocation and timing arrangement to dynamically adjust with the chip load and task progress. Attached Figure Description

[0069] Figure 1 This is a flowchart illustrating the steps of a dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain, as described in this invention.

[0070] Figure 2 A flowchart for complexity assessment and key scene element identification;

[0071] Figure 3 Real-time temperature monitoring curves for heterogeneous computing units;

[0072] Figure 4This is a graph showing the trend of throughput variation of the dynamic computing power scheduling system over the scheduling cycle.

[0073] Figure 5 Box plot of real-time utilization of heterogeneous computing units for autonomous driving chips. Detailed Implementation

[0074] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] See Figure 1 This invention provides a dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain, the specific method including:

[0076] Multimodal environmental perception data and vehicle state data of autonomous vehicles are collected, and these collected data are preprocessed and aligned to form a synchronized environmental state stream. Based on this synchronized environmental state stream, the complexity of road scenarios in the current and predicted future periods is assessed, and key scenario elements that have a decisive impact on computing power requirements are identified. Based on the identified key scenario elements, the types and intensity of computing resources required to execute different autonomous driving function sub-tasks are determined. The current total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters are obtained to form a real-time state snapshot of the chip. Based on the required computing resource types, computing intensity, and the chip's real-time state snapshot, a global dynamic scheduling instruction set is generated, which includes computing task mapping, computing power allocation ratio, and execution timing. This global dynamic scheduling instruction set is executed to allocate and start corresponding computing tasks on multiple heterogeneous computing units of the intelligent chip, while monitoring the execution progress of computing tasks and the actual load of computing units. Based on the execution progress of computing tasks and the actual load of computing units, the computing power allocation ratio and execution timing in the global dynamic scheduling instruction set are fine-tuned online to form a closed-loop control.

[0077] In one embodiment of the present invention, raw sensing data from LiDAR, camera, and millimeter-wave radar are received, and each type of raw sensing data is independently timestamped and its data integrity is verified. For camera image data, distortion correction and noise filtering are performed; for LiDAR point cloud data, outlier removal and motion distortion compensation are performed; and for millimeter-wave radar data, clutter suppression is performed. For the independently processed sensing data, spatial registration and temporal synchronization are performed based on a unified world coordinate system and time reference, ensuring that information describing the same object is aligned spatiotemporally. The vehicle's speed, acceleration, steering angle, and position information are acquired, and this information is fused with the spatiotemporally aligned multimodal sensing data. The fused data stream is segmented and encapsulated according to a preset time slice length, with each data packet containing all sensing and state information within a complete time slice, and output as a synchronized environmental state stream.

[0078] See Figure 2 From the synchronized environmental state stream, the system extracts the number, type, speed, and trajectory prediction uncertainty measure of dynamic traffic participants. It also extracts the density and type of static traffic facilities, as well as the current state and changing patterns of traffic lights. Furthermore, it extracts the road's geometric topology, lane line clarity, and current weather and lighting conditions. Integrating dynamic traffic participant information, static traffic facility information, and road environment information, a scene complexity score is calculated for the current moment using a scene understanding model. Based on the current vehicle motion state, the system predicts the scene complexity scores for multiple future time slices. For time slices where the scene complexity score exceeds a preset threshold, the elements in the corresponding synchronized environmental state stream that dominate the increase in complexity are identified as key scene elements. These key scene elements include dense and unpredictable pedestrian groups, complex intersections without lane lines, and reduced visibility under extreme weather conditions.

[0079] In specific implementation, multimodal environmental perception data and vehicle body state data of autonomous vehicles are collected and preprocessed and aligned to form a synchronized environmental state stream. This process receives raw perception data streams from LiDAR, cameras, and millimeter-wave radar. The LiDAR point cloud data, camera image data, and millimeter-wave radar data are independently timestamped and their data integrity is verified. In some embodiments, the accuracy of the timestamps needs to reach the microsecond level to ensure the accuracy of subsequent synchronization. For the image data collected by the camera, the preprocessing performs distortion correction based on camera intrinsic parameters and noise filtering based on filtering algorithms. For the point cloud data collected by LiDAR, the preprocessing performs outlier removal based on statistics and motion distortion compensation based on inertial measurement unit data. For the millimeter-wave radar data, the preprocessing performs clutter suppression based on Doppler information and constant false alarm rate detection.

[0080] In practical implementation, independently processed lidar point cloud data, camera image data, and millimeter-wave radar data are spatially registered and temporally synchronized based on a unified world coordinate system and time reference. Spatial registration transforms data from different sensors into the same coordinate system, while temporal synchronization uses interpolation or extrapolation based on the timestamps of each data point to align information describing the same object in space and time. The vehicle's speed, acceleration, steering angle, and position information are acquired and fused with the spatiotemporally aligned multimodal perception data. The fusion process can be based on Kalman filtering or extended Kalman filtering algorithms to achieve state estimation. The fused data stream is segmented and encapsulated according to a preset time slice length. Each data packet contains all perception and state information within a complete time slice, and the final output is a synchronized environmental state stream.

[0081] The complexity of road scenarios in the current and predicted future periods is assessed based on the synchronous environmental state stream. The number, type, speed, and trajectory prediction uncertainty metric of dynamic traffic participants are extracted from the synchronous environmental state stream. The trajectory prediction uncertainty metric can be calculated using the covariance or entropy value output by the prediction model. The density and type of static traffic facilities, as well as the current state and change patterns of traffic lights, are extracted from the synchronous environmental state stream. The current state and change patterns of traffic lights can be obtained from the vehicle communication unit or visual recognition results. The geometric topology of the road, lane line clarity, and current weather and lighting conditions are extracted from the synchronous environmental state stream. Lane line clarity can be quantified by the gradient intensity of the lane line edges in the image. Weather and lighting conditions can be provided by the ambient light sensor or image feature classifier.

[0082] By integrating dynamic traffic participant information, static traffic facility information, and road environment information, a scene complexity score is calculated at the current moment using a scene understanding model. This scene understanding model can be a trained multilayer perceptron or convolutional neural network, and its input is a feature vector of the aforementioned information. In some embodiments, the scene complexity score... The calculation can be expressed as the following formula:

[0083]

[0084] in: A comprehensive measure representing the density of dynamic traffic participants and the uncertainty of their movement. A metric representing the complexity of static traffic facilities. A measure of the degree of unfavorable road and environmental conditions. , , These are the weight coefficients for the corresponding items, and the scene complexity scores for multiple future time slices are predicted based on the current vehicle motion state. The prediction can be extrapolated based on the vehicle kinematics model and the current synchronized environmental state flow. It can be understood that the number of predicted time slices is consistent with the look-ahead window length of the scheduling system.

[0085] The time slices whose scene complexity scores exceed a preset threshold are identified as key scene elements. The elements that dominate the increase in complexity in the corresponding synchronous environmental state stream are identified as key scene elements. The preset threshold is set according to functional safety requirements and historical operation data. Key scene elements include dense pedestrian groups with uncertain trajectories, complex intersections without lane lines, and reduced visibility under extreme weather conditions. The identification conditions for dense pedestrian groups with uncertain trajectories are that the number of pedestrians exceeds the first density threshold and the average trajectory prediction uncertainty exceeds the first uncertainty threshold. The identification conditions for complex intersections without lane lines are that the lane line detection confidence is lower than the second confidence threshold and the number of path selection branches exceeds the second branch number threshold. The identification conditions for reduced visibility under extreme weather conditions are that the visibility estimate is lower than the third visibility threshold. Optionally, the identified key scene elements and their corresponding time slices will be output together with the scene complexity scores to provide input for subsequent computing power requirement mapping.

[0086] In one embodiment of the present invention, a mapping relationship library between key scene elements and autonomous driving function sub-tasks is established. The autonomous driving function sub-tasks include object detection, semantic segmentation, object tracking, trajectory prediction, path planning, and behavior decision-making. Based on the mapping relationship library, a list of autonomous driving function sub-tasks that need to be activated or enhanced is matched for each key scene element identified in the current and prediction time period. For each sub-task in the list of autonomous driving function sub-tasks that need to be activated or enhanced, its computational resource requirement model under different computational accuracy and real-time requirements is queried. The computational resource categories include scalar computation units, vector computation units, matrix computation units, and customized accelerators.

[0087] An offline performance analysis dataset is pre-established for each autonomous driving function subtask. This dataset contains records of the subtask's execution time and resource consumption on various computing units under different input data scales, computational precision, and real-time constraints. Based on this dataset, a computational resource requirement model is trained. This model takes input data features, computational precision requirements, and real-time requirements as inputs, and outputs the theoretical computational intensity and memory usage on different types of computing units. When determining the resource requirements of a subtask, the input data features of the current scene are extracted from the synchronized environmental state stream. These features, combined with the computational precision and real-time requirements corresponding to the safety level determined from the mapping relation library, are used as inputs to the computational resource requirement model. Running the model yields the expected computational intensity and memory usage for each subtask on scalar computing units, vector computing units, matrix computing units, and customized accelerators. From the retrieved computational resource requirement models, a computational precision and real-time configuration matching the safety level of the current key scene elements is selected to determine the required type of computational resources and the expected computational intensity for each subtask.

[0088] In specific implementation, the types and intensity of computing resources required to execute different autonomous driving function sub-tasks are determined based on the identified key scene elements. This process first establishes a mapping relationship library between key scene elements and autonomous driving function sub-tasks. Autonomous driving function sub-tasks include object detection, semantic segmentation, object tracking, trajectory prediction, path planning, and behavior decision-making. The mapping relationship library is a predefined associated data structure that records a list of autonomous driving function sub-tasks that must be triggered or have their performance improved for each key scene element. In some embodiments, the key scene element "dense and unpredictable pedestrian groups" is associated with the autonomous driving function sub-tasks of object detection, object tracking, trajectory prediction, and behavior decision-making in the mapping relationship library, while the key scene element "complex intersections without lane lines" is associated with semantic segmentation, path planning, and behavior decision-making.

[0089] In practical implementation, based on the mapping relationship library, for each key scene element identified in the current and predicted time period, a list of autonomous driving function sub-tasks that need to be activated or enhanced is matched. The matching process is a direct query operation, using the identified key scene elements as input keys to retrieve the corresponding list of autonomous driving function sub-tasks from the mapping relationship library. It can be understood that a time slice may contain multiple key scene elements at the same time, and the matched list of autonomous driving function sub-tasks is the union of the sub-tasks associated with these elements. For each sub-task in the list of autonomous driving function sub-tasks that need to be activated or enhanced, its computing resource requirement model under different computing precision and real-time requirements is queried. The computing resource categories include scalar computing units, vector computing units, matrix computing units, and customized accelerators.

[0090] The specific implementation of the computational resource requirement model involves pre-establishing an offline performance analysis dataset for each autonomous driving function subtask. This dataset contains execution time and resource consumption records for various computing units under different input data scales, computational precision, and real-time constraints. Resource consumption records include computing unit occupancy, memory bandwidth usage, and power consumption data. Based on this offline performance analysis dataset, a computational resource requirement model is trained. This model takes input data characteristics, computational precision requirements, and real-time requirements as input, and outputs the theoretical computational intensity and memory usage on different types of computing units. In some embodiments… In this model, the computational resource requirement model is constructed using a gradient boosting tree-based regression model. When determining the resource requirements of autonomous driving function subtasks, the input data features of the current scene are extracted from the synchronized environmental state stream. These features include the resolution of the image to be processed, point cloud density, and the number of dynamic targets. Combined with the computational accuracy and real-time requirements corresponding to the safety level determined from the mapping relation library, these features serve as input to the computational resource requirement model. Running the model yields the expected computational intensity and memory usage for the autonomous driving function subtasks on scalar computation units, vector computation units, matrix computation units, and customized accelerators, respectively. The determination of can follow the following relationship:

[0091]

[0092] in: A scalar representing the features of the input data. This represents the required calculation accuracy. Representing real-time requirements, the function and These respectively characterize the mapping relationship between data characteristics and accuracy, and between real-time performance and computational intensity. and It is a weighting factor. It is a basic computational overhead constant.

[0093] From the obtained computing resource requirement model, the computing accuracy and real-time configuration that matches the safety level of the current key scenario elements are selected. The safety level is related to the potential risk level of the key scenario elements. High-risk elements correspond to higher computing accuracy and stricter real-time requirements. This determines the type of computing resources and the expected value of computing intensity required for each autonomous driving function subtask. The computing resource category is determined based on the output of the computing resource requirement model for the expected value of computing intensity of different types of computing units. One or more units whose expected value of computing intensity meets the requirements and whose type of computing unit matches the task characteristics are selected. Optionally, for the object detection subtask, its computing resource category can be determined as matrix computing unit and customized accelerator. The corresponding expected value of computing intensity is quantified in terms of trillions of operations per second. It can be understood that the expected value of computing intensity is a dynamic value that is updated in real time as the characteristics of the input data and the safety level requirements change.

[0094] In one embodiment of the present invention, the real-time operating frequency and utilization rate of the CPU core, image processor core, and neural network processor core of the intelligent chip are periodically read from the chip's internal performance counter. The real-time temperature values ​​of different computing unit regions of the intelligent chip are periodically read from the chip's internal temperature sensor. Based on the real-time operating frequency, peak computing power, and utilization rate of the CPU core, image processor core, and neural network processor core, their respective current actual available computing power is calculated, and the total currently available computing power is obtained by summing them. The average utilization rate and utilization rate trend of each computing unit over the past time window are recorded. The current total available computing power, the real-time utilization rate, average utilization rate, utilization rate trend, and real-time temperature value of each computing unit are encapsulated into a structured data packet as a real-time state snapshot of the chip.

[0095] In practical implementation, the current total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters are obtained to form a real-time state snapshot of the chip. This process periodically reads the real-time operating frequency and occupancy rate of the central processing unit core, image processing unit core, and neural network processing unit core of the intelligent chip from the internal performance counter. The reading operation is triggered at fixed time intervals. Within one reading cycle, the performance counter records the number of active cycles of the central processing unit core, image processing unit core, and neural network processing unit core in the past sampling window. The real-time occupancy rate is obtained by calculating the ratio of the number of active cycles to the total number of cycles. At the same time, the real-time temperature values ​​of different computing unit areas of the intelligent chip are read from the internal temperature sensor. The temperature sensor is usually embedded near the computing unit, and its reading directly reflects the current thermal state of the computing unit.

[0096] Based on the real-time operating frequency, peak computing power, and utilization rate of the central processing unit core, image processing unit core, and neural network processor core, calculate the current actual available computing power of each core, and the actual available computing power of the computing unit. The following relationship can be used for estimation:

[0097]

[0098] in: This represents the computing unit at the reference frequency. The theoretical peak computing power is below This represents the real-time operating frequency read from the performance counter. This represents the real-time utilization rate read from the performance counter, and is summed to obtain the current total available computing power. The total available computing power is the arithmetic sum of the actual available computing power of all online computing units, such as the central processing unit core, the image processing unit core, and the neural network processing unit core. In some embodiments, the performance parameters of the computing units are shown in Table 1.

[0099] Table 1: Example Table of Computing Unit Performance Parameters

[0100] Calculation unit name Peak computing power Reference frequency Current operating frequency Current occupancy rate Central Processing Unit Core A 0.5 2500 2400 0.65 Image Processor Core B 2.0 1000 1100 0.40 Neural Network Processor Core C 10.0 800 900 0.75

[0101] The system records the average utilization rate and utilization trend of each computing unit over a past time window. The average utilization rate is the moving average of the occupancy rates of multiple sampling points over a period of time. The utilization trend can be obtained through linear regression or by calculating the mean difference of the occupancy rates of the most recent sampling points. It is understood that the length of the time window is related to the response speed requirements of the scheduling decision. The current available total computing power, the real-time utilization rate of each computing unit, the average utilization rate, the utilization trend, and the real-time temperature value are encapsulated into a structured data packet as a real-time chip status snapshot. In some embodiments, the real-time chip status snapshot uses a JavaScript object. Organized in a representation or protocol buffer format, its fields include timestamps, total available computing power, and a substructure containing detailed status information of each unit such as the central processing unit core, image processing unit core, and neural network processor core. Optionally, the generation frequency of chip real-time status snapshots is higher than that of the global dynamic scheduling instruction set, which can provide more granular feedback for online fine-tuning. The periodically formed chip real-time status snapshots are stored in shared memory for the scheduling decision module to read in real time. The chip real-time status snapshots provide a complete resource and health status profile of the intelligent chip at the time of decision-making and are one of the key inputs for dynamic load allocation.

[0102] See Figure 3In real-time temperature monitoring of heterogeneous computing units, the acquisition and analysis of thermal state parameters are crucial for achieving dynamic computing power scheduling of intelligent chips. Specifically, real-time temperature data for three types of heterogeneous computing units—CPU, GPU, and NPU—are acquired at a fixed sampling period, forming a temperature sequence curve that changes over time. This curve is then compared with a preset temperature safety threshold (85℃). The curves show that the NPU temperature generally remains within the range of 68℃ to 75℃, exhibiting the largest fluctuation and a peak close to 75℃, reflecting its high computing load and heat generation characteristics in neural network inference tasks. The CPU temperature ranges from 58℃ to 72℃, with the second largest fluctuation, highly correlated with the load changes of scalar computing tasks. The GPU temperature remains stable within the range of 50℃ to 61℃, exhibiting the smallest fluctuation and the most stable thermal state. The real-time temperatures of all three types of computing units do not exceed the 85℃ safety threshold, indicating that the chip's thermal state is within a controllable range under the current scheduling strategy. At the scheduling decision level, the temperature monitoring data will be encapsulated as the core thermal state field of the chip's real-time status snapshot, and together with information such as the utilization rate and available computing power of each computing unit, it will serve as the input for dynamic load allocation: when the temperature of a computing unit continues to approach the safety threshold, the scheduler will trigger a load migration or frequency reduction strategy to migrate some computing tasks to the same type of computing unit with a lower temperature, so as to avoid overheating and frequency reduction or hardware damage, while ensuring the real-time performance and reliability of autonomous driving function subtasks.

[0103] In one embodiment of the present invention, a set of candidate computing units is selected from the chip's real-time status snapshot based on the required computing resource category. These candidate units must meet the resource category requirements, have a current utilization rate below a safety threshold, and have a temperature within the normal range. Based on the expected computing intensity and the actual available computing power of each unit in the candidate computing unit set, the computing load of the subtask is allocated to multiple candidate computing units, forming a mapping relationship between computing tasks and physical computing units. For each physical computing unit assigned a task, the computing power allocation ratio it should occupy is calculated based on its allocated load and actual available computing power. Considering the data dependencies between subtasks, computing latency requirements, and data communication bandwidth between computing units, a conflict-free execution sequence is planned for all scheduled computing tasks, ensuring that dependent tasks are executed sequentially and high-priority tasks receive resources first. The computing task mapping relationship, computing power allocation ratio, and execution sequence are encoded into a set of machine-executable instructions, forming a global dynamic scheduling instruction set.

[0104] In practice, a global dynamic scheduling instruction set containing computing task mapping, computing power allocation ratio, and execution timing is generated based on the required computing resource category, computing intensity, and chip real-time status snapshot. This process first filters out a set of candidate computing units from the chip real-time status snapshot that meet the resource category, have a current utilization rate below the safety threshold, and a temperature within the normal range, based on the required computing resource category. The safety threshold is a preset utilization upper limit used to avoid overloading computing units. The normal temperature range is defined by the chip's thermal design specifications. The filtering operation traverses the status records of all computing units in the chip's real-time status snapshot, checks whether their resource type labels match the required computing resource category, and determines whether their real-time utilization rate and real-time temperature simultaneously meet the constraints. All computing units that meet the constraints are added to the set of candidate computing units.

[0105] Based on the expected computational intensity and the actual available computing power of each unit in the candidate computing unit set, the computational load of the autonomous driving function subtask is allocated to multiple candidate computing units, forming a mapping relationship between computational tasks and physical computing units. The allocation process follows the principles of load balancing and minimizing communication overhead. In some embodiments, an algorithm based on first-fit or best-fit can be used for allocation. For each autonomous driving function subtask to be allocated, units with actual available computing power greater than or equal to the expected computational intensity of the task are selected from the candidate computing unit set. If the computing power of a single unit is insufficient, the computational load of a task is split into multiple sub-loads and allocated to multiple candidate computing units of the same type. After each allocation, the remaining available computing power of the unit is updated. For each physical computing unit of the allocated task, the computing power allocation ratio that the physical computing unit should occupy is calculated based on the load allocated to the physical computing unit and the actual available computing power. The calculation can be expressed as:

[0106]

[0107] in: This represents the computational load allocated to that physical computing unit. This represents the actual available computing power of the physical computing unit. It is a computing power margin coefficient reserved to cope with load fluctuations.

[0108] Considering the data dependencies, computational latency requirements, and data communication bandwidth between computing units among the various autonomous driving function subtasks, a conflict-free execution sequence is planned for all scheduled computational tasks. Data dependencies are defined by a task dependency graph, and computational latency requirements are determined by the safety level of the autonomous driving function. When planning the execution sequence, it is ensured that dependent tasks are executed sequentially, and high-priority tasks receive resources first. In some embodiments, a directed acyclic graph scheduling algorithm or a list scheduling algorithm can be used for sequence planning. It can be understood that the execution sequence determines the start and end times of different computational tasks on the computing units. Optionally, refer to Table 2, which shows a simplified example of computational power allocation and mapping relationships.

[0109] Table 2: Computation Task Mapping and Computational Power Allocation Table

[0110] Autonomous driving function subtasks Allocated computing units Calculate load The computing unit can utilize computing power Computing power allocation ratio Target detection Neural Network Processor Core 1 800 1000 0.85 Semantic segmentation Image Processor Core 1 300 500 0.65 Trajectory prediction Neural Network Processor Core 2 600 750 0.85

[0111] The computational task mapping relationship, computing power allocation ratio, and execution timing are encoded into a set of machine-executable instructions to form a global dynamic scheduling instruction set. The encoding process converts the mapping relationship, ratio parameters, and timing information into binary commands or configuration register values ​​that the intelligent chip scheduler can recognize. Optionally, the global dynamic scheduling instruction set is described using a domain-specific language or intermediate representation. Finally, it is compiled into hardware instructions by the chip driver or runtime library. The global dynamic scheduling instruction set contains the necessary information to control all relevant computing units to work together to complete all scheduling tasks within a scheduling cycle. It can be understood that the generated global dynamic scheduling instruction set will be passed to the execution module for loading and running.

[0112] See Figure 4In the multi-dimensional performance monitoring of the dynamic computing power scheduling system, the trend of system throughput with the scheduling cycle intuitively reflects the adaptive optimization effect of the computing power scheduling strategy. Looking at the curve trend: from scheduling cycles 1 to 5, the system throughput shows a continuous downward trend, gradually decreasing from an initial 30 tasks / second to 20 tasks / second. This stage corresponds to increased scenario complexity and a surge in computing power demand. The system is in a period of load ramping and scheduling strategy adaptation, and the load balancing and task mapping of heterogeneous computing units have not yet reached their optimal state. From scheduling cycles 6 to 10, the system throughput enters a steady recovery phase, gradually climbing from 21 tasks / second to 29 tasks / second, eventually approaching the initial peak level. This stage demonstrates the effectiveness of the closed-loop control mechanism: by monitoring the task execution progress and the actual load of computing units online, the computing power allocation ratio and execution sequence are dynamically fine-tuned, gradually alleviating problems such as computing unit overload and data dependency latency, and achieving optimization of load balancing and resource utilization. The curve's "decline followed by rise" characteristic verifies the robustness and adaptive adjustment capability of the dynamic computing power scheduling method in complex scenarios: initially, the throughput fluctuates due to sudden changes in scenario complexity, but later, it quickly converges to an efficient operating state through closed-loop feedback control, ensuring the real-time performance and stability of the autonomous driving system in dynamic environments.

[0113] In one embodiment of the present invention, a global dynamic scheduling instruction set is parsed, and the executable code and input data of subtasks are loaded into the local memory of designated heterogeneous computing units according to the computing task mapping relationships therein. Based on the computing power allocation ratio in the global dynamic scheduling instruction set, the operating frequency, voltage, and number of concurrent threads of the corresponding heterogeneous computing units are configured. According to the execution sequence in the global dynamic scheduling instruction set, task start signals are sequentially sent to each heterogeneous computing unit, and it is monitored whether the tasks are correctly started within the preset latest start time. During the execution of the computing tasks, the performance counters of each computing unit are periodically read to obtain the percentage of completed workload to the total workload, which is used as the execution progress of the computing tasks. During the execution of the computing tasks, the instantaneous utilization rate and queue length of each computing unit are collected in real time as the actual load of the computing unit.

[0114] The execution progress of computational tasks is compared with a preset schedule to identify lagging computational tasks and their corresponding computing units. The actual load of the computing units containing the lagging tasks is analyzed to determine whether the lag is due to excessive unit load or delays in preceding data-dependent tasks. If it's due to excessive unit load, the chip's real-time status snapshot is used to find similar computing units with lower utilization. Part of the load of the lagging task is migrated to these similar units, and the computational task mapping and power allocation ratio are updated. Independent computational sub-blocks that can be divided within the lagging tasks are identified, and the data transfer overhead required to migrate these sub-blocks to the target similar computing unit is evaluated. The data transfer overhead is compared with the computational acceleration benefits after migration. If the benefits outweigh the overhead, an appropriate number of computational sub-blocks are selected for migration. The selected computational sub-blocks and their associated input data are transferred from the source computing unit's local memory to the target similar computing unit's local memory. The computational task mapping relationship in the global dynamic scheduling instruction set is modified, and the mapping relationship between the migrated computational sub-blocks and the original computational tasks on the target computing unit is recorded. Based on the changes in load on the source and target computing units after migration, the computing power allocation ratio of these two computing units in the new scheduling cycle is recalculated and updated. If the delay is due to a preceding data-dependent task, the execution timing of the global dynamic scheduling instruction set is dynamically adjusted without violating core data dependencies, prioritizing the scheduling of preceding dependent tasks or allocating more computing power resources to them. The instructions for adjusting the computing power allocation ratio and execution timing are sent to the corresponding computing units in real time, replacing parts of the original instruction set, thus achieving closed-loop control based on execution feedback.

[0115] In specific implementation, a global dynamic scheduling instruction set is executed to allocate and start corresponding computing tasks on multiple heterogeneous computing units of the intelligent chip. At the same time, the execution progress of computing tasks and the actual load of computing units are monitored. This process parses the global dynamic scheduling instruction set and, according to the computing task mapping relationship in the global dynamic scheduling instruction set, loads the executable code and input data of the autonomous driving function subtasks into the local memory of the designated heterogeneous computing units. The loading operation is completed through direct memory access or a copy command initiated by the host processor. According to the computing power allocation ratio in the global dynamic scheduling instruction set, the operating frequency, voltage and number of concurrent threads of the corresponding heterogeneous computing units are configured. The configuration is implemented by writing to the specific control registers of the heterogeneous computing units. According to the execution sequence in the global dynamic scheduling instruction set, task start signals are sent to each heterogeneous computing unit in sequence, and it is monitored whether the task is correctly started within the preset latest start time. In some embodiments, the task start signal is a trigger instruction written to a specific address, and monitoring is completed by querying the task status register.

[0116] During the execution of the computing task, the performance counters of each computing unit are periodically read to obtain the percentage of the total workload completed by the autonomous driving function subtasks, which serves as the execution progress of the computing task. The workload can be measured by the number of image frames processed, the number of layers calculated, or the number of loop iterations completed. During the execution of the computing task, the instantaneous utilization rate and the queue length of the queued tasks of each computing unit are collected in real time as the actual load of the computing unit. The instantaneous utilization rate is directly read from the performance counters, while the queue length of the queued tasks is obtained from the length of the pending task list maintained by the scheduler.

[0117] The execution progress of the computation tasks is compared with the preset schedule to identify the computation tasks and their respective computing units that are lagging behind. The schedule is generated based on the execution sequence of the global dynamic scheduling instruction set and the theoretical execution time of the tasks. The actual load of the computing unit where the lagging computation task is located is analyzed to determine whether the delay is due to excessive load of the computing unit or due to the delay of the preceding data-dependent task. The determination is based on checking whether the actual load of the computing unit continuously exceeds the high load threshold and whether its input data buffer is empty or has timed out.

[0118] If the delay is caused by excessive computing unit load, the system will search for similar computing units with low utilization from the chip's real-time status snapshot, migrate part of the load of the delayed task to the same type of computing unit, update the computing task mapping and computing power allocation ratio, identify the independent computing sub-blocks that can be divided in the delayed computing task, and evaluate the data transfer overhead required to migrate the independent computing sub-blocks to the target similar computing unit. Estimate the computational speedup benefits by comparing the data transfer overhead and the computational speedup gains after migration, based on the sub-block data size and the interconnection bandwidth between computing units. Determined by the expected faster execution time of the sub-block on the target computing unit, in some embodiments, the migration decision follows a rule: if Then the migration is executed, where This is a trade-off factor. If the benefits outweigh the costs, an appropriate number of computing sub-blocks are selected for migration. The selected computing sub-blocks and their related input data are transferred from the local memory of the source computing unit to the local memory of the target computing unit of the same type. The computing task mapping relationship in the global dynamic scheduling instruction set is modified. The mapping relationship between the migrated computing sub-blocks and the original computing tasks on the target computing unit is recorded. Based on the changes in the load on the source computing unit and the target computing unit after migration, the computing power allocation ratio of the two computing units in the new scheduling cycle is recalculated and updated.

[0119] If the progress is delayed due to the delay of the preceding data-dependent tasks, the execution sequence of the global dynamic scheduling instruction set will be dynamically adjusted without violating the core data dependencies. Priority will be given to scheduling the preceding dependent tasks, or more computing resources will be allocated to them. It is understood that adjusting the execution sequence may involve swapping the execution order of non-dependent tasks or inserting higher-priority scheduling slots. Allocating more computing resources to the preceding dependent tasks can be achieved by temporarily increasing the working frequency of the computing unit where they are located or by allocating additional parallel threads. Optionally, the instructions for adjusting the computing power allocation ratio and execution sequence are sent to the corresponding computing units in real time, replacing part of the original instruction set to achieve closed-loop control based on execution feedback. The adjustment instructions are injected into the running scheduling process through interrupt or polling. Optionally, after each online fine-tuning, a new subset of instructions containing the latest mapping, ratio and timing information will be generated to cover the scheduling of subsequent time windows.

[0120] See Figure 5 In the real-time computing power monitoring scenario of heterogeneous computing units in autonomous driving chips, the distribution characteristics of the utilization rate of each computing unit can be intuitively presented through box plots. The figure shows the statistical distribution of the utilization rates of four types of heterogeneous computing units—CPU, GPU, NPU, and FPGA accelerators—during the real-time computing power monitoring phase: NPU: The overall utilization rate is the highest, with a median of approximately 70%. The box plots are concentrated in the range of 62% to 78%, with extreme values ​​extending to 52% and 84% respectively. This indicates that as the core of neural network computing, it undertakes the main AI inference load in autonomous driving scenarios, and the load fluctuation is relatively controllable. GPU: The median utilization rate is approximately 57%, with the box plots distributed in the range of 52% to 67%, and extreme values ​​ranging from 40% to 86%. This reflects its load characteristics in graphics rendering and parallel computing tasks, with some peak fluctuations. CPU: The median utilization rate is approximately 52%, with the box plots concentrated in the range of 47% to 64%, and extreme values ​​ranging from 32% to 76%. As a general-purpose control core, the load distribution is relatively balanced, mainly responsible for basic computing tasks such as task scheduling and logic control. FPGA accelerators exhibit the lowest utilization level, with a median of approximately 40%, concentrated in the 33%–44% range, and extreme values ​​ranging from 18% to 52%. This indicates relatively weak current load intensity and significant computational redundancy, making them suitable target units for load migration in dynamic scheduling. The box plot comprehensively depicts the real-time load distribution of heterogeneous computing units in autonomous driving chips, providing a quantitative basis for load balancing, task migration, and resource allocation in subsequent dynamic computing power scheduling strategies.

[0121] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain, characterized in that, Includes the following steps: Collect multimodal environmental perception data and vehicle body state data of autonomous vehicles, and preprocess and align the collected multimodal environmental perception data and vehicle body state data to form a synchronous environmental state stream; Based on the synchronous environmental state stream, the complexity of road scenarios in the current and predicted future periods is assessed, and key scenario elements that have a decisive impact on computing power requirements are identified. Based on the identified key scene elements, determine the types and intensity of computing resources required to execute different autonomous driving function sub-tasks; The system obtains the current total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters to form a real-time state snapshot of the chip. Based on the required computing resource type, computing intensity, and real-time chip status snapshot, a global dynamic scheduling instruction set is generated, which includes computing task mapping, computing power allocation ratio, and execution timing. The system executes a global dynamic scheduling instruction set to allocate and start corresponding computing tasks on multiple heterogeneous computing units of the smart chip, while monitoring the execution progress of the computing tasks and the actual load of the computing units. Based on the execution progress of the computing tasks and the actual load of the computing units, the computing power allocation ratio and execution timing in the global dynamic scheduling instruction set are fine-tuned online to form a closed-loop control.

2. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 1, characterized in that, The collected multimodal environmental perception data and vehicle body state data are preprocessed and aligned to form a synchronized environmental state stream, specifically including: It receives raw sensing data from lidar, cameras, and millimeter-wave radar, and performs independent timestamp marking and data integrity verification for each type of raw sensing data. For image data from cameras, distortion correction and noise filtering are performed; for point cloud data from lidar, outlier removal and motion distortion compensation are performed; and for data from millimeter-wave radar, clutter suppression is performed. For various types of sensory data that have been processed independently, spatial registration and temporal synchronization are performed based on a unified world coordinate system and time reference, so that the information describing the same object is aligned in space and time. The vehicle's speed, acceleration, steering angle, and position information are acquired, and then fused with spatiotemporally aligned multimodal perception data. The fused data stream is segmented and encapsulated according to a preset time slice length. Each data packet contains all the sensing and state information within a complete time slice, and the final output is a synchronized environmental state stream.

3. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 2, characterized in that, The synchronous environmental state stream is used to assess the complexity of road scenarios in the current and predicted future periods, identifying key scenario elements that have a decisive impact on computing power requirements, specifically including: Extract the number, type, speed, and trajectory prediction uncertainty measure of dynamic traffic participants from the synchronized environmental state stream; Extract the density and type of static traffic facilities, as well as the current state and change patterns of traffic lights, from the synchronized environmental state stream; Extract the road's geometric topology, lane line clarity, and current weather and lighting conditions from the synchronized environmental state stream; By integrating dynamic traffic participant information, static traffic facility information, and road environment information, the scene complexity score at the current moment is calculated through a scene understanding model, and the scene complexity scores for multiple future time slices are predicted based on the current vehicle motion state. The moment slice where the scene complexity score exceeds the preset threshold is identified as the key scene element in the corresponding synchronous environmental state stream that dominates the increase in complexity. The key scene elements include dense pedestrian groups with unpredictable trajectories, complex intersections without lane lines, and reduced visibility under extreme weather conditions.

4. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 3, characterized in that, Based on the identified key scene elements, the types and intensity of computing resources required to execute different autonomous driving function sub-tasks are determined, specifically including: Establish a mapping relationship library between key scene elements and autonomous driving function sub-tasks, wherein the autonomous driving function sub-tasks include target detection, semantic segmentation, target tracking, trajectory prediction, path planning, and behavior decision-making; Based on the mapping relationship library, for each key scene element identified in the current and predicted time period, a list of autonomous driving function subtasks that need to be activated or enhanced is matched. For each subtask in the list of autonomous driving function subtasks that need to be activated or enhanced, query its computing resource requirement model under different computing accuracy and real-time requirements. The computing resource categories include scalar computing units, vector computing units, matrix computing units, and customized accelerators. From the computational resource requirement model obtained from the query, select the computational precision and real-time configuration that matches the security level of the current key scenario elements, thereby determining the type of computational resources and the expected value of computational intensity required for each subtask.

5. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 4, characterized in that, The process of acquiring the current total available computing power of the intelligent chip, the real-time utilization rate of each computing unit, and thermal state parameters to form a real-time state snapshot of the chip specifically includes: Periodically read the real-time operating frequency and utilization rate of the central processing unit core, image processing unit core, and neural network processing unit core of the intelligent chip from the internal performance counters; The real-time temperature values ​​of different computing unit areas of the smart chip are periodically read from the internal temperature sensor of the chip. Based on the real-time operating frequency, peak computing power and utilization rate of the central processing unit core, image processing unit core, and neural network processing unit core, calculate their current actual available computing power and sum them up to obtain the current total available computing power. Record the average utilization rate and utilization trend of each computing unit over the past time window; The current available total computing power, the real-time utilization rate, average utilization rate, utilization rate trend, and real-time temperature value are encapsulated into a structured data packet as a real-time snapshot of the chip's status.

6. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 5, characterized in that, The process of generating a global dynamic scheduling instruction set, which includes computing task mapping, computing power allocation ratio, and execution timing, based on the required computing resource type, computing intensity, and real-time chip status snapshot, specifically includes: Based on the required computing resource category, a set of candidate computing units that meet the resource category, have a current utilization rate below the safety threshold, and have a temperature within the normal range are selected from the chip's real-time status snapshot; Based on the expected computational intensity and the actual available computing power of each unit in the set of candidate computing units, the computational load of the subtask is distributed to multiple candidate computing units, forming a mapping relationship between computational tasks and physical computing units. For each physical computing unit assigned a task, the computing power allocation ratio to be used is calculated based on its assigned load and actual available computing power. Considering the data dependencies between subtasks, computation latency requirements, and data communication bandwidth between computing units, a conflict-free execution sequence is planned for all scheduled computing tasks to ensure that dependent tasks are executed in sequence and high-priority tasks obtain resources first. The computational task mapping relationship, computing power allocation ratio and execution timing are encoded into a set of machine-executable instructions to form a global dynamic scheduling instruction set.

7. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 6, characterized in that, The execution of the global dynamic scheduling instruction set allocates and initiates corresponding computing tasks on multiple heterogeneous computing units of the intelligent chip, while monitoring the execution progress of the computing tasks and the actual load of the computing units, specifically including: The global dynamic scheduling instruction set is parsed, and the executable code and input data of the subtasks are loaded into the local memory of the specified heterogeneous computing unit according to the computing task mapping relationship therein. Based on the computing power allocation ratio in the global dynamic scheduling instruction set, configure the operating frequency, voltage, and number of concurrent threads of the corresponding heterogeneous computing units; According to the execution sequence of the global dynamic scheduling instruction set, task start signals are sent to each heterogeneous computing unit in sequence, and the system monitors whether the task is started correctly within the preset latest start time. During the execution of the computing task, the performance counters of each computing unit are periodically read to obtain the percentage of the workload that has been completed to the total workload, which is used as the execution progress of the computing task. During the execution of computing tasks, the instantaneous utilization rate of each computing unit and the queue length of the queued tasks are collected in real time as the actual load of the computing unit.

8. The method for dynamic computing power scheduling of automotive intelligent chips for autonomous driving domain according to claim 7, characterized in that, The process of fine-tuning the computing power allocation ratio and execution timing in the global dynamic scheduling instruction set online based on the execution progress of the computing tasks and the actual load of the computing units to form a closed-loop control specifically includes: The execution progress of the computation tasks is compared with the preset schedule to identify the computation tasks that are lagging behind and the computation units in which they are located. Analyze the actual load of the computing unit where the delayed computing task is located to determine whether the delay is due to excessive load of the computing unit or due to delay of the preceding data-dependent task. If the problem is caused by excessive load on the computing unit, the system will search for the same type of computing unit with low utilization in the chip's real-time status snapshot, migrate part of the load of the lagging tasks to the same type of computing unit, and update the computing task mapping and computing power allocation ratio. If the delay is caused by the preceding data dependency task, the execution sequence of the global dynamic scheduling instruction set will be dynamically adjusted without violating the core data dependency, and the preceding dependency task will be scheduled first or more computing resources will be allocated to it. The instructions for adjusting the computing power allocation ratio and execution timing are sent to the corresponding computing units in real time, replacing some of the content in the original instruction set, thereby realizing closed-loop control based on execution feedback.

9. A dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain according to claim 8, characterized in that, The above refers to querying the computational resource requirement model for each subtask in the list of autonomous driving function subtasks that need to be activated or enhanced, under different computational accuracy and real-time requirements. Specifically, this includes: An offline performance analysis dataset is pre-established for each autonomous driving function subtask. The offline performance analysis dataset contains records of the execution time and resource consumption of the subtask on various computing units under different input data scales, different computational precisions and different real-time constraints. Based on the offline performance analysis dataset, a computing resource demand model is trained. The computing resource demand model takes input data characteristics, computing accuracy requirements, and real-time requirements as inputs, and outputs the theoretical computing intensity and memory usage on different types of computing units. When it is necessary to determine the resource requirements of a subtask, the input data features of the current scenario are extracted from the synchronized environment state stream, and combined with the computational precision and real-time requirements corresponding to the security level determined from the mapping relationship library, and used as the input of the computational resource requirement model. By running the computational resource requirement model, the expected computational intensity and memory usage of the subtask on the scalar computation unit, vector computation unit, matrix computation unit, and customized accelerator are obtained respectively.

10. A dynamic computing power scheduling method for automotive intelligent chips in the autonomous driving domain according to claim 9, characterized in that, The step of migrating a portion of the load from delayed tasks to the same type of computing unit and updating the computing task mapping and computing power allocation ratio specifically includes: Identify the independent computational sub-blocks that can be divided in the delayed computational task, and evaluate the data transfer overhead required to migrate the independent computational sub-blocks to the target computational unit of the same type; Compare the data transfer overhead with the computational speedup benefits after migration. If the benefits outweigh the overhead, select an appropriate number of computational sub-blocks to migrate. The selected computational sub-block and its associated input data are transferred from the local memory of the source computational unit to the local memory of the target computational unit of the same type. Modify the computation task mapping relationship in the global dynamic scheduling instruction set and record the mapping relationship between the migrated computation sub-blocks and the original computation tasks on the target computation unit; Based on the changes in load on the source and target computing units after migration, the computing power allocation ratio of these two computing units in the new scheduling cycle is recalculated and updated.

Images