Artificial intelligence acceleration method and system based on heterogeneous hardware

Abstract

The application provides an artificial intelligence acceleration method and system based on heterogeneous hardware, relates to the technical field of computers, and collects the cache miss rate, instruction throughput and chip power consumption waveform when the heterogeneous hardware executes an artificial intelligence inference task, obtains a performance jitter value through mining processing, generates a cache sensitivity by associating the calculation link corresponding to the artificial intelligence inference task, obtains an access index by extracting waveform features, takes the two as inputs to obtain a mapping strategy through reinforcement learning, dynamically calculates a priority adjustment amount and executes preemptive scheduling after triggering a predetermined condition by monitoring the task execution state, can realize artificial intelligence inference task acceleration, and improves the response efficiency and energy efficiency ratio in a high concurrency scenario.

Description

Technical Field

[0001] This application relates to the technical field of computer science, and in particular to an artificial intelligence acceleration method and system based on heterogeneous hardware. Background Technology

[0002] In scenarios such as cloud computing data centers and intelligent driving edge nodes, artificial intelligence inference tasks often face the need for high concurrency and multi-model mixed execution. Furthermore, heterogeneous hardware clusters have become the mainstream deployment solution due to their advantages in both computing power and energy efficiency. Therefore, how to make heterogeneous hardware with different architectures accurately adapt to various inference tasks and improve resource utilization while ensuring the real-time performance of tasks has become the core problem that artificial intelligence acceleration methods based on heterogeneous hardware urgently need to solve.

[0003] Currently, in existing technologies, the following two types of methods are mostly used to accelerate artificial intelligence inference. One type is to statically allocate tasks based on hardware computing power parameters, such as prioritizing the allocation of computationally intensive tasks to the graphics processing unit (GPU) and memory-intensive tasks to the central processing unit (CPU) with a large cache. The other type is to dynamically adjust the task assignment by monitoring the hardware load rate in real time and using a round-robin or weighted allocation mechanism. Some solutions in these two types of methods will combine the task deadline with preset fixed priority scheduling rules.

[0004] However, the main drawback of existing technologies is that resource scheduling is based solely on superficial indicators such as computing power or load, resulting in insufficient adaptability between hardware resources and task requirements. Furthermore, when task execution causes fluctuations in hardware performance, it is impossible to accurately pinpoint the root cause of the problem and make timely adjustments, making it difficult to balance task execution efficiency with optimal utilization of hardware resources. Summary of the Invention

[0005] The purpose of this application is to provide an artificial intelligence acceleration method and system based on heterogeneous hardware, so as to solve the problem that it is difficult to balance task execution efficiency and optimal utilization of hardware resources in the existing technology.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides an artificial intelligence acceleration method based on heterogeneous hardware, comprising:

[0007] Collect the cache miss rate and instruction throughput of various heterogeneous hardware when performing artificial intelligence inference tasks, as well as the chip power consumption waveform caused by cache miss events;

[0008] The cache miss rate and instruction throughput are mined to obtain a performance jitter value. The performance jitter value is correlated with the computation link corresponding to the artificial intelligence inference task to obtain cache sensitivity. The chip power consumption waveform is subjected to feature extraction processing to obtain an access index.

[0009] Using the cache sensitivity and the access index as state inputs, a reinforcement learning strategy is employed to perform resource mapping, thereby obtaining a mapping strategy between artificial intelligence inference tasks and heterogeneous hardware.

[0010] Based on the mapping strategy, the task execution status on each heterogeneous hardware is monitored. When the task execution status on any of the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index, a dynamic priority scheduling mechanism is adopted to dynamically calculate the priority adjustment amount of the corresponding artificial intelligence inference task according to the real-time execution progress and remaining deadline of the artificial intelligence inference task.

[0011] Based on the aforementioned priority adjustment amount, preemptive task rescheduling is performed to accelerate artificial intelligence inference tasks.

[0012] Optionally, the step of using the cache sensitivity and the access index as state inputs and employing a reinforcement learning strategy to perform resource mapping to obtain a mapping strategy between the artificial intelligence inference task and heterogeneous hardware includes:

[0013] Based on the cache sensitivity and the access index, a state representation vector is constructed;

[0014] Based on the state representation vector, a resource mapping action is generated through the decision network in the reinforcement learning strategy. The resource mapping action is defined as: selecting target execution hardware from a heterogeneous hardware set and assigning an initial execution priority for the AI ​​inference task to be scheduled.

[0015] During task execution, the corresponding comprehensive reward value is calculated based on the matching degree between real-time performance data and the performance jitter value and the access index, respectively.

[0016] Based on the comprehensive reward value, resources are mapped to obtain a mapping strategy.

[0017] Optionally, the step of calculating the corresponding comprehensive reward value based on the matching degree between real-time performance data and the performance jitter value and the access index includes:

[0018] Calculate the task completion reward value based on the number of tasks completed within the preset deadline.

[0019] The system reward value is calculated based on the change in the number of tasks completed by the system per unit time relative to the historical baseline.

[0020] Based on the comparison result of the first matching degree and the first preset threshold, a first penalty value is calculated, and based on the comparison result of the second matching degree and the second preset threshold, a second penalty value is calculated. The first matching degree is the matching degree between the real-time performance data and the performance jitter value, and the second matching degree is the matching degree between the real-time performance data and the access index.

[0021] Calculate the resource penalty value based on the degree of decrease in instruction throughput;

[0022] The task completion reward value, the system reward value, the first penalty value, the second penalty value, and the resource penalty value are weighted and merged to obtain a comprehensive reward value.

[0023] Optionally, the step of dynamically calculating the priority adjustment amount for the corresponding AI inference task based on the real-time execution progress and remaining deadline of the AI ​​inference task using a dynamic priority scheduling mechanism includes:

[0024] Calculate the urgency quantification based on the real-time execution progress, remaining computational load, and remaining deadline;

[0025] Based on the aforementioned urgency quantification value, a priority adjustment amount for the corresponding artificial intelligence inference task is dynamically generated through a dynamic priority scheduling mechanism. A positive priority adjustment amount indicates an increase in task priority, while a negative one indicates a decrease in task priority.

[0026] Optionally, the step of dynamically generating the priority adjustment amount for the corresponding artificial intelligence inference task based on the urgency quantification value through a dynamic priority scheduling mechanism includes:

[0027] Based on the aforementioned urgency quantification, the urgency factor is calculated;

[0028] The compatibility factor is calculated based on the performance matching degree and the degree of exponential deviation of heterogeneous hardware.

[0029] The urgency factor and the fitness factor are normalized to obtain the urgency index and the fitness index.

[0030] By using a dynamic priority scheduling mechanism, the urgency index and the suitability index are comprehensively calculated to obtain the initial priority adjustment amount;

[0031] The initial priority adjustment amount is subjected to range constraint processing to obtain the priority adjustment amount corresponding to the artificial intelligence reasoning task.

[0032] Optionally, the step of mining the cache miss rate and the instruction throughput to obtain the performance jitter value includes:

[0033] The cache miss rate and the instruction throughput are segmented to obtain time series segments;

[0034] The waveform feature recognition method is used to detect the peaks and troughs of the cache miss rate in the time series segment to obtain the target interval;

[0035] A time-series correlation analysis method is used to analyze the instruction throughput and the corresponding cache miss rate in the time series segment to determine the correlation time period when the instruction throughput decreases and the cache miss rate increases. Based on the target interval and the correlation time period, the performance jitter value is determined.

[0036] Optionally, the step of performing preemptive task rescheduling based on the priority adjustment amount to accelerate the artificial intelligence inference task includes:

[0037] Based on the priority adjustment amount, a task queue is constructed, and all artificial intelligence reasoning tasks in the task queue are sorted by priority to obtain a task sequence.

[0038] Based on the task sequence, the target inference task and the corresponding target heterogeneous hardware are determined;

[0039] Perform a preemptive operation on the target heterogeneous hardware to pause the currently running low-priority task and save the corresponding hardware execution state, load the target inference task into the target heterogeneous hardware, and continue to execute the target inference task based on the hardware execution state;

[0040] For low-priority tasks that have been suspended, a priority recalculation strategy is adopted to update the corresponding priority adjustment amount and reinsert them into the task queue, so as to accelerate the artificial intelligence inference task.

[0041] Secondly, this application provides an artificial intelligence acceleration system based on heterogeneous hardware, comprising:

[0042] The acquisition module is used to collect the cache miss rate and instruction throughput generated by various heterogeneous hardware when performing artificial intelligence inference tasks, as well as the chip power consumption waveform caused by cache miss events.

[0043] The association module is used to mine the cache miss rate and the instruction throughput to obtain the performance jitter value, associate the performance jitter value with the calculation link corresponding to the artificial intelligence inference task to obtain the cache sensitivity, and perform feature extraction processing on the chip power consumption waveform to obtain the access index.

[0044] The mapping module is used to take the cache sensitivity and the access index as state inputs, and use a reinforcement learning strategy to perform resource mapping to obtain a mapping strategy between artificial intelligence inference tasks and heterogeneous hardware.

[0045] The computing module is used to monitor the task execution status on each heterogeneous hardware based on the mapping strategy. When the task execution status on any of the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index, the module dynamically calculates the priority adjustment amount of the corresponding artificial intelligence inference task by adopting a dynamic priority scheduling mechanism based on the real-time execution progress and remaining deadline of the artificial intelligence inference task.

[0046] The scheduling module is used to perform preemptive task rescheduling based on the priority adjustment amount, so as to accelerate artificial intelligence inference tasks.

[0047] Thirdly, this application provides an electronic device, comprising:

[0048] Memory, used to store computer programs;

[0049] A processor for executing the computer program to implement the steps of the heterogeneous hardware-based artificial intelligence acceleration method as described in the first aspect above.

[0050] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the artificial intelligence acceleration method based on heterogeneous hardware as described in the first aspect above.

[0051] The technical solution of this application has the following beneficial effects:

[0052] This application provides a foundation for subsequent analysis by collecting key data on hardware operation and task execution; then, through data processing, it generates features to transform the raw data into effective information reflecting the task's cache sensitivity and memory access intensity, supporting accurate subsequent decision-making; subsequently, it achieves precise adaptation between tasks and heterogeneous hardware through reinforcement learning resource mapping to generate the optimal initial resource allocation scheme; then, it promptly detects task execution anomalies by monitoring and calculating priority adjustment amounts to dynamically determine the magnitude and direction of priority adjustments; finally, it rapidly optimizes hardware resource usage through preemptive rescheduling, thereby improving the overall execution efficiency of artificial intelligence inference tasks.

[0053] Furthermore, this application constructs a decision-making basis by integrating multi-dimensional indicators and dynamically optimizes the resource mapping logic with reinforcement learning. This ensures that hardware allocation is both tailored to task characteristics and hardware status, and can be continuously adjusted based on execution feedback. This improves the accuracy and flexibility of resource mapping, and guarantees operational stability and resource utilization. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating an artificial intelligence acceleration method based on heterogeneous hardware provided in this application embodiment;

[0056] Figure 2 A schematic diagram illustrating a specific implementation of an artificial intelligence acceleration method based on heterogeneous hardware provided in this application embodiment;

[0057] Figure 3 A schematic diagram of the structure of an artificial intelligence acceleration system based on heterogeneous hardware provided in an embodiment of this application;

[0058] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] Existing technologies for accelerating artificial intelligence on heterogeneous hardware rely heavily on superficial metrics such as hardware computing power and load rate for task allocation and scheduling. This leads to difficulties in accurately matching hardware resources with task requirements. Furthermore, when task execution causes performance fluctuations, it is impossible to pinpoint the root cause of the problem and optimize it in a timely manner, ultimately affecting task execution efficiency and resource utilization.

[0060] To address this, this application proposes an AI acceleration method based on heterogeneous hardware. This method first collects key operational data during hardware task execution and transforms it into effective information reflecting task caching needs and memory access intensity. Then, it utilizes intelligent strategies to achieve precise matching between tasks and hardware, while simultaneously monitoring task execution status in real time to dynamically adjust task priorities and reschedule resources. This solution precisely solves the shortcomings of insufficient adaptability in existing technologies by establishing a deep correlation between task characteristics and hardware status, and improves the overall operating efficiency and resource utilization of heterogeneous hardware clusters while ensuring task real-time performance.

[0061] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] The core of this application is to provide an artificial intelligence acceleration method based on heterogeneous hardware, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0063] S101. Collect the cache miss rate and instruction throughput generated by various heterogeneous hardware when performing artificial intelligence inference tasks, as well as the chip power consumption waveform caused by cache miss events.

[0064] Among them, cache miss rate refers to the frequency of cache misses when the hardware reads data, instruction throughput refers to the statistical data reflecting the number of task instructions processed by the hardware per unit time, and chip power consumption waveform refers to the instantaneous fluctuation curve of chip power consumption when the hardware reads data and misses the cache.

[0065] In this embodiment, a data acquisition process is triggered synchronously when the task starts and continues throughout the entire execution cycle of the AI ​​inference task on various heterogeneous hardware. Subsequently, a time-series synchronous acquisition technology is used to continuously capture cache access-related states at fixed time intervals. The timing and frequency of each cache miss are recorded, gradually accumulating to form a continuous cache miss rate. At the same time, the total number of instructions successfully processed by the hardware per unit time is counted in real time, and the instruction throughput is obtained by segmenting according to the acquisition cycle. The chip power consumption change at the moment a cache miss event occurs is monitored synchronously, and the complete fluctuation process of power consumption rising from a baseline value to a peak value and then falling back is captured. Each fluctuation is recorded to form a chip power consumption waveform, and finally integrated into time-consistent raw operating data to provide a complete data source for subsequent analysis.

[0066] In this embodiment of the invention, by acquiring core raw data on hardware operation and task execution, the cache access efficiency, instruction processing capability, and power consumption fluctuation patterns are fully reflected, providing high-quality data support for subsequent accurate mining of task cache-sensitive characteristics. By analyzing memory access intensity, intelligent matching of tasks and heterogeneous hardware is achieved, ensuring the accuracy and effectiveness of subsequent decision-making processes and providing a data foundation for the overall acceleration solution.

[0067] S102. Mining the cache miss rate and the instruction throughput to obtain the performance jitter value, associating the performance jitter value with the calculation link corresponding to the artificial intelligence inference task to obtain the cache sensitivity, and performing feature extraction processing on the chip power consumption waveform to obtain the access index.

[0068] In one specific implementation, step S102 includes:

[0069] Step 1021: Segment the cache miss rate and the instruction throughput to obtain time series segments. Use waveform feature recognition method to detect peaks and troughs in the cache miss rate of the time series segments to obtain the target interval.

[0070] Among them, time series segments refer to the segmented data after continuous cache miss rates and instruction throughput are split according to fixed durations, and target intervals refer to the continuous time periods in which peaks of cache miss rates occur.

[0071] For example, the heterogeneous hardware cluster includes three types of devices: D1 type GPU, D2 type high cache CPU, and D3 type accelerator card, which are specifically used to perform intelligent video analysis and reasoning tasks. This task is broken down into three core computing stages: C1 feature extraction, C2 feature fusion, and C3 target recognition. The total computational load of each stage is set to 100, and the total execution deadline is 16 seconds.

[0072] After the cluster starts the task, the hardware monitoring module continuously collects data at a collection period of 100 milliseconds, and obtains complete data for 100 periods by collecting data for 10 seconds. Taking D1 processing C1 as an example, the total number of cache accesses within a certain period is recorded. The number of cache misses is 120. It is 18, and according to the formula Calculate the cache miss rate, where P is the cache miss rate for a single cycle. This represents the number of cache misses within this period. The total number of cache accesses within this cycle is calculated, resulting in a P-value of 0.15 for this cycle. The P-values ​​for 100 cycles are recorded to form the cache miss rate. Subsequently, the number of instructions processed in each cycle is counted, resulting in 15,000 instructions processed over 100 cycles, and then processed according to the formula... Calculate the instruction throughput, where T is the instruction throughput per unit time. Total number of instructions The total acquisition time was calculated, and the instruction throughput T value was found to be 1500 instructions / second. At the same time, 85 cache misses were captured, and the baseline power consumption of each fluctuation was recorded as 38 watts, with peak power consumption ranging from 56 watts to 62 watts, in order to generate the chip power consumption waveform.

[0073] Subsequently, when performing AI inference tasks on the heterogeneous hardware cluster, the cache miss rate and instruction throughput collected by S101 are first split into 10 time series segments according to a 1-second duration; then, peak and trough detection is performed on the cache miss rate of the 5th segment to determine the 0.4-0.7 second period as the target interval.

[0074] Step 1022: Using a time-series correlation analysis method, analyze the instruction throughput and the corresponding cache miss rate in the time series segment to determine the correlation time period when the instruction throughput decreases and the cache miss rate increases, and determine the performance jitter value based on the target interval and the correlation time period.

[0075] Among them, the performance jitter value refers to the characteristic pattern that reflects the fluctuation of hardware execution efficiency caused by local differences in data.

[0076] For example, the instruction throughput of the segment is compared with the cache miss rate in each time period to determine the 0.3-0.8 second period as the associated time period, and then the performance jitter value is extracted by combining the overlapping 0.4-0.7 second period.

[0077] Step 1023: Using graph structure matching technology, the performance jitter value is mapped to the computation graph node corresponding to the artificial intelligence model to establish the correspondence between performance jitter and specific computation links.

[0078] For example, graph structure matching technology can be used to establish a correspondence between performance jitter and the three computational stages C1, C2, and C3 of the task artificial intelligence model.

[0079] Step 1024: Based on the correspondence, determine the jitter intensity of each computing stage and its position distribution in the graph structure to construct a cache sensitivity that reflects the cache sensitivity of the computing stages.

[0080] Among them, cache sensitivity refers to vector data that combines the jitter intensity and location distribution of each computing stage to characterize the degree of dependence of the computing stage on the cache.

[0081] In step 1024, based on this correspondence, the jitter intensity formula is used... Calculate the jitter intensity of each calculation stage, where D represents a certain calculation stage. The intensity of the vibration, This represents the peak cache miss rate corresponding to this calculation stage. This represents the average cache miss rate across all computation stages. The execution time percentage of each computation stage is recorded, along with the location distribution of each computation stage in the computation graph structure. This information is then integrated to construct a cache sensitivity index that reflects the cache sensitivity of each computation stage.

[0082] For example, when calculating jitter intensity, first determine that the peak cache miss rates for C1, C2, and C3 are 0.18, 0.09, and 0.13, respectively, and then calculate the average value. Given that the execution time percentages of each calculation stage are 0.45, 0.3, and 0.25 respectively, and substituting these values ​​into the jitter intensity formula, the jitter intensity of C1 is obtained. C2's vibration intensity The vibration intensity of C3 Based on this, and combined with the position information of each link in the computation graph, cache sensitivities of 0.157, -0.097, -0.00625, 0.45, 0.3, and 0.25 were constructed.

[0083] Step 1025: Using a pulse signal separation method, extract independent pulses triggered by cache miss events from the chip power consumption waveform. Quantize the rising edge steepness, duration, and amplitude of the independent pulses to obtain a pulse waveform feature set. Statistically process the number of independent pulses per unit time to obtain a pulse density index. Combine the pulse waveform feature set and the pulse density index with multiple parameters to obtain an access index.

[0084] Among them, an independent pulse refers to a single power fluctuation signal triggered only by a cache miss event and separated from the chip power consumption waveform; the pulse waveform feature set refers to the feature set obtained by quantizing the rise steepness, duration and amplitude of the independent pulse; the pulse density index refers to the number of times an independent pulse occurs per unit time; and the access index is a comprehensive index that combines the pulse waveform feature set and the pulse density index to reflect the frequency and intensity of memory access.

[0085] In step 1025, the formula can be used. The pulse density index is obtained by counting the number of independent pulses occurring per unit time. This is a pulse density index, where N is the number of independent pulses within the statistical duration. To calculate the duration, a weighted fusion algorithm was used according to the formula. The access index is obtained by fusing the pulse waveform feature set with the pulse density index using multiple parameters; where I is the access index, α and β are weighting coefficients, and α+β=1, and F is the comprehensive quantitative value of the pulse waveform feature set.

[0086] For example, 15 independent pulses are separated from the chip power consumption waveform acquired by S101. These pulses are quantized to obtain an average rise edge steepness of 6 watts / millisecond, an average duration of 10 milliseconds, and an average amplitude of 25 watts. The comprehensive quantization value of the pulse waveform feature set is then calculated with weights of 0.4, 0.3, and 0.3. Then set the statistical duration. seconds, substitute into the formula Calculated pulse density index Finally, the weighting coefficients α=0.65 and β=0.35 were set, and the formula was used to calculate the weighting coefficients. Calculate the access index These cache sensitivity and access indexes will be directly used as state inputs for S103, and used for resource mapping decisions under the reinforcement learning strategy, so as to match the heterogeneous hardware adapted to the three computing stages C1, C2 and C3 respectively.

[0087] This application transforms raw runtime data into feature indicators and accurately captures task cache sensitivity, hardware performance fluctuation patterns, and memory access intensity. It can realize the transformation and upgrading of data from raw collection to effective features, providing accurate and reliable analytical basis for subsequent resource mapping decisions, task execution status monitoring, and dynamic priority adjustment. This will help to efficiently adapt tasks to heterogeneous hardware and lay the foundation for the scientific nature and effectiveness of the overall acceleration solution.

[0088] S103. Using the cache sensitivity and the access index as state inputs, a reinforcement learning strategy is adopted to perform resource mapping, thereby obtaining a mapping strategy between artificial intelligence inference tasks and heterogeneous hardware.

[0089] In one specific implementation, such as Figure 2 As shown, step S103 includes:

[0090] Step 1031: Construct a state representation vector based on the cache sensitivity and the access index.

[0091] Among them, the state representation vector refers to the comprehensive vector data that integrates cache sensitivity, access index, real-time performance indicators of heterogeneous hardware, and system load indicators, and is used to comprehensively describe task characteristics and hardware status.

[0092] In step 1031, real-time performance metrics and system load metrics of each heterogeneous hardware are collected, and these metrics are aligned with cache sensitivity and access index and integrated to construct a state representation vector that comprehensively reflects task characteristics and hardware status.

[0093] For example, when a heterogeneous hardware cluster performs an artificial intelligence inference task, it first collects the real-time computing power utilization, memory usage, and other performance indicators of the three types of heterogeneous hardware D1, D2, and D3 in the cluster, as well as the global system load rate. These are then integrated with the cache sensitivity of 0.157, -0.097, -0.00625, 0.45, 0.3, and 0.25 and the access index of 8.91 obtained from S102 to construct a state representation vector.

[0094] Step 1032: Based on the state representation vector, generate resource mapping actions through the decision network in the reinforcement learning strategy, wherein the resource mapping action is defined as: selecting target execution hardware from the heterogeneous hardware set and assigning initial execution priority for the AI ​​inference task to be scheduled.

[0095] Among them, resource mapping action refers to the specific operation of selecting target execution hardware and allocating initial execution priority for the AI ​​inference task to be scheduled;

[0096] For example, resource mapping actions are generated through reinforcement learning decision networks. Initially, D1, D2, and D3 are selected as target execution hardware for the three computing stages C1, C2, and C3, and the initial execution priorities are set to 3, 2, and 1, respectively.

[0097] Step 1033: During task execution, calculate the corresponding comprehensive reward value based on the matching degree between real-time performance data and the performance jitter value and the access index.

[0098] Among them, the comprehensive reward value refers to the quantitative indicator that integrates the task completion reward value, system reward value, first penalty value, second penalty value and resource penalty value, and is used to evaluate the quality of resource mapping actions;

[0099] Specifically, step 1033 may include the following steps: calculating a task completion reward value based on the number of tasks completed within a preset deadline; calculating a system reward value based on the change in the number of tasks completed by the system per unit time relative to a historical benchmark; calculating a first penalty value based on the comparison result of a first matching degree and a first preset threshold, and calculating a second penalty value based on the comparison result of a second matching degree and a second preset threshold, wherein the first matching degree is the matching degree between the real-time performance data and the performance jitter value, and the second matching degree is the matching degree between the real-time performance data and the access index; calculating a resource penalty value based on the degree of decrease in instruction throughput; and weighting and fusing the task completion reward value, the system reward value, the first penalty value, the second penalty value, and the resource penalty value to obtain a comprehensive reward value.

[0100] Among them, the task completion reward value is a positive indicator calculated based on the number of tasks completed within a preset deadline; the system reward value is a positive indicator calculated based on the change in the number of tasks completed by the system within a unit of time relative to the historical benchmark; the first penalty value is a negative indicator calculated based on the comparison result of the matching degree between real-time performance data and performance jitter value and the first preset threshold; the second penalty value is a negative indicator calculated based on the comparison result of the matching degree between real-time performance data and access index and the second preset threshold; and the resource penalty value is a negative indicator calculated based on the degree of decrease in instruction throughput caused by task concurrency on the target execution hardware.

[0101] For example, set a preset deadline of 10 seconds and a task count of 8. It is 0.5, and according to the formula Calculate the reward value for task completion, where, The reward value for completing the task. The reward coefficient for task completion. The task completion reward value is calculated by substituting the data into the preset deadline for the number of tasks to be completed. =0.5×8=4; then set the number of tasks completed per unit time based on the historical baseline. 15. Number of tasks completed per unit of time. For 18, It is 2, and according to the formula Calculate the system reward value, where, This is the system reward value. This is the throughput reward coefficient. This represents the number of tasks the system completes per unit of time. The system reward value is calculated by substituting the number of tasks completed by the system per unit time, based on historical baseline data. =2×(18-15) / 15=0.4; then set the matching degree between real-time performance data and performance jitter value. 0.3, first preset threshold 0.2 It is 3, and according to the formula Calculate the first penalty value, where, The first penalty value, The first penalty coefficient, To determine the degree of matching between real-time performance data and performance jitter values, The first penalty value is calculated based on the first preset threshold. =3×max(0,0.3-0.2)=0.3; Simultaneously set the matching degree between real-time performance data and access index. 0.25, second preset threshold 0.2 It is 2, and according to the formula Calculate the second penalty value, where, The second penalty value, This is the second penalty coefficient. To determine the matching degree between real-time performance data and access index, The second preset threshold is used as the input data to calculate the second penalty value. =2×max(0, 0.25-0.2)=0.1;

[0102] Next, the system instruction throughput under no-concurrency conditions is set. The system instruction throughput is 2000 messages / second during concurrent operation. 1800 messages / second =1, and according to the formula Calculate the resource penalty value, where, This is the resource penalty value. This is the resource competition penalty coefficient. This represents the system instruction throughput without concurrency. The resource penalty value is calculated by substituting the data into the system instruction throughput under concurrent conditions. =1×(2000-1800) / 2000=0.1; then set the weight. 0.3 0.2 0.2 0.1 It is 0.2, and + + + + =1, according to the formula Calculate the overall reward value, where, For the overall reward value, , , , , By assigning weights to each indicator and substituting the data, the overall reward value is calculated. =0.3×4+0.2×0.4-0.2×0.3-0.1×0.1-0.2×0.1=1.19.

[0103] Step 1034: Perform resource mapping based on the comprehensive reward value to obtain the mapping strategy.

[0104] Among them, the mapping strategy refers to the hardware allocation scheme that summarizes all computing links and the execution scheme that clarifies the correspondence between each computing link of the artificial intelligence inference task and the heterogeneous hardware.

[0105] In step 1034, the adaptability of resource mapping actions is evaluated based on the comprehensive reward value, and suitable heterogeneous hardware is allocated to each computing stage to form a hardware allocation scheme corresponding to each computing stage. All hardware allocation schemes are then summarized and coordinated according to the task execution process to obtain the mapping strategy between artificial intelligence inference tasks and heterogeneous hardware.

[0106] For example, the hardware allocation scheme is adjusted according to the comprehensive reward value, and finally it is determined that C1 is allocated to D1, C2 is allocated to D1, and C3 is allocated to D2. The hardware allocation schemes of all computing links are summarized to obtain the mapping strategy. This mapping strategy will be directly used in S104 to monitor the task execution status of C1, C2, and C3 on each heterogeneous hardware.

[0107] This application achieves precise adaptation of each computational stage of artificial intelligence inference tasks to heterogeneous hardware, and the scientific and reasonable resource allocation scheme provides a clear basis for subsequent task execution status monitoring and dynamic priority scheduling, which can effectively improve hardware resource utilization and task execution stability.

[0108] S104. Based on the mapping strategy, monitor the task execution status on each heterogeneous hardware. When the task execution status on any of the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index, adopt a dynamic priority scheduling mechanism to dynamically calculate the priority adjustment amount of the corresponding artificial intelligence inference task according to the real-time execution progress and remaining deadline of the artificial intelligence inference task.

[0109] In one specific implementation, step S104 includes:

[0110] Step 1041: Based on the mapping strategy, monitor the task execution status on each heterogeneous hardware within a preset period to obtain actual progress data. Analyze the actual progress data with the expected execution progress data to identify abnormal tasks, which are artificial intelligence inference tasks with execution deviations.

[0111] Among them, abnormal tasks refer to artificial intelligence reasoning tasks in which the actual execution progress deviates from the expected execution progress.

[0112] In step 1041, the mapping strategy is used to clarify the correspondence between each artificial intelligence inference task and heterogeneous hardware. Real-time monitoring technology is used to obtain the actual progress data such as the completion ratio and time elapsed of the tasks on each heterogeneous hardware according to a preset cycle. These data are compared with the preset expected execution progress data in time intervals to identify abnormal tasks with execution deviations.

[0113] For example, based on the obtained mapping strategy, the correspondence between C1 and D1, C2 and D1, and C3 and D2 is clearly defined, and the monitoring period is preset to 500 milliseconds. The actual progress data such as the completion ratio of C1 and C2 on D1 and the time elapsed on D2 are obtained in real time. It is expected that C2 will complete 80% of the execution progress within 10 seconds. In fact, it is observed that C2 only completes 65% after 10 seconds of execution. The actual execution progress data is compared with the expected execution progress data for each time period to identify C2 as an abnormal task.

[0114] Step 1042: Extract the cache miss rate and chip power consumption waveform of the heterogeneous hardware, and based on the cache miss rate and chip power consumption waveform of the heterogeneous hardware, determine whether the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index.

[0115] Among them, the predetermined conditions refer to the standards set based on performance jitter value and access index, used to determine whether the task execution status is abnormal.

[0116] In step 1042, the real-time cache miss rate and chip power consumption waveform of the heterogeneous hardware are extracted, and the trend of cache miss rate is compared with the performance jitter value. The access intensity data converted from the chip power consumption waveform is analyzed with the normal range of the access index to determine whether the corresponding predetermined conditions are met.

[0117] For example, extract the real-time cache miss rate and chip power consumption waveform of D1 where C2 is located, and compare the trend of cache miss rate with the performance jitter value. If the two are found to be highly consistent, it is determined that the corresponding predetermined conditions are met. When the predetermined conditions are met, the following step 1043 is executed.

[0118] Step 1043: Calculate the urgency quantification value based on the real-time execution progress, remaining computational load, and remaining deadline.

[0119] Among them, the urgency quantification value refers to the real-time execution progress, remaining computation amount and remaining deadline of the comprehensive task, which is used to characterize the urgency of task execution;

[0120] For example, to calculate the urgency quantification value, given that the real-time execution progress percentage P of C2 is 0.65, the total computational cost is... The remaining computational cost is 100. The deadline is 35. For 5 seconds, and according to the formula Where E is the urgency metric and P is the percentage of the task's real-time execution progress. For the remaining computational load, Total computational load, Substituting the data into the remaining deadline, we get the emergency metric value E, which is 0.0245.

[0121] Step 1044: Based on the urgency quantification value, dynamically generate the priority adjustment amount for the corresponding artificial intelligence inference task through a dynamic priority scheduling mechanism, wherein a positive priority adjustment amount indicates an increase in task priority, and a negative priority indicates a decrease in task priority.

[0122] Specifically, step 1044 may include the following steps: calculating an urgency factor based on the urgency quantification value; calculating an adaptability factor based on the performance matching degree and index deviation degree of heterogeneous hardware; normalizing the urgency factor and the adaptability factor to obtain an urgency index and an adaptability index; comprehensively calculating the urgency index and the adaptability index through a dynamic priority scheduling mechanism to obtain a preliminary priority adjustment amount; and applying range constraint processing to the preliminary priority adjustment amount to obtain the priority adjustment amount for the corresponding artificial intelligence inference task.

[0123] Among them, the urgency factor is an indicator that reflects the urgency of the task, calculated by combining the urgency metric, the remaining deadline, and the proportion of completed work; the suitability factor is an indicator that reflects the suitability of the task with the current hardware, based on the degree of hardware performance matching and the degree of index deviation; the urgency index is the standardized data obtained after normalizing the urgency factor; the suitability index is the standardized data obtained after normalizing the suitability factor; and the preliminary priority adjustment amount is the priority adjustment value calculated by combining the urgency index and the suitability index, without being subject to range constraints.

[0124] Furthermore, when performing range constraint processing, it can be based on the current overall load level of the system and the available resource status of the target heterogeneous hardware. The available resource status refers to the real-time status of various resources of the target heterogeneous hardware that are currently not occupied and can be immediately allocated to artificial intelligence inference tasks. Specifically, it includes the hardware's computing power idle rate, the number of idle processor cores, the idle capacity of memory and cache, and the idle percentage of data read and write bandwidth. The actual availability of these resources directly determines whether the hardware can undertake the task execution requirements after priority adjustment and is the key basis for constraining the amount of priority adjustment.

[0125] For example, according to the formula Calculate the urgency factor, where, E is the urgency factor, and E is the urgency metric. Let P be the remaining deadline, and P be the percentage of real-time execution progress of the abnormal task. Substituting these values ​​into the data yields the urgency factor. Equals 0.014; subsequently, given the known performance matching degree The index deviation is 0.8. When the value is 0.75, according to the formula Calculate the fitness factor, where, For fitness factor, For performance matching degree, To determine the degree of index deviation, substitute the data to obtain the fit factor. It equals 0.775;

[0126] The urgency factor (0.014) and fitness factor (0.775) were then normalized to obtain the urgency index. The fitness index is 0.14. The value is set to 0.78. Then, the weighting coefficients a and b are set to 0.4 and 0.6, respectively, with a plus b equal to 1. The formula is then applied. Calculate the initial priority adjustment amount. This represents the initial priority adjustment amount, where a and b are weighting coefficients. As an indicator of urgency, As an adaptation metric, substituting the data yields the initial priority adjustment amount. It equals 0.524;

[0127] Finally, the system's current overall load rate is 70%, the available resources of the target heterogeneous hardware D1 are sufficient, and the reasonable range of the priority adjustment amount is set to -1 to 1. Then, the initial priority adjustment amount of 0.524 is subjected to range constraint processing to obtain the final priority adjustment amount of 0.52. This priority adjustment amount will be directly used in S105 to perform preemptive task rescheduling on C2 to ensure that C2 completes efficiently.

[0128] This application can promptly identify execution anomalies of tasks on heterogeneous hardware and accurately quantify task urgency and hardware compatibility. The generation of reasonable priority adjustment amounts can provide accurate basis for subsequent preemptive scheduling, thereby ensuring the real-time execution needs of artificial intelligence inference tasks and the efficient utilization of hardware resources.

[0129] S105. Based on the priority adjustment amount, perform preemptive task rescheduling to accelerate the artificial intelligence inference task.

[0130] In one specific implementation, step S105 includes:

[0131] Step 1051: Based on the priority adjustment amount, construct a task queue, and sort all the artificial intelligence reasoning tasks in the task queue by priority to obtain a task sequence.

[0132] Among them, the task queue refers to a data structure built according to the priority adjustment amount of artificial intelligence inference tasks, which is used to coordinate the task scheduling order, and the task sequence refers to the task list formed after prioritizing the tasks in the task queue.

[0133] In step 1051, based on the priority adjustment amount of each artificial intelligence inference task, the identification information of each task is associated with the corresponding adjustment amount, and a task queue is constructed using a linked list structure. Then, the quicksort algorithm is used to sort all tasks in the queue in descending order of adjustment amount, while taking into account the data dependency relationship of the tasks during the sorting process to avoid subsequent execution conflicts caused by sorting. Finally, a task sequence with clear priority hierarchy and no contradiction in execution logic is obtained.

[0134] For example, the heterogeneous hardware cluster uses the priority adjustment amount of 0.52 obtained from S104 for C2, and combines it with the priority adjustment amount of 0.2 for C1 and the priority adjustment amount of 0.1 for C3. The identification information of the three tasks is associated with the corresponding adjustment amount one by one, and a task queue is constructed using a linked list structure. Then, the quicksort algorithm is used to sort the tasks from largest to smallest according to the adjustment amount, while verifying that there are no data dependency conflicts between the tasks, so as to obtain the task sequence C2, C1, and C3.

[0135] Step 1052: Based on the task sequence, determine the target inference task and the corresponding target heterogeneous hardware.

[0136] Among them, the target reasoning task refers to the high-priority task selected from the task sequence that needs to be scheduled first.

[0137] In step 1052, based on the task sequence, the first task in the sequence is extracted as the target inference task by the highest priority selection strategy. Combined with the task and hardware adaptation rules in the mapping strategy, and taking into account the computing power requirements, memory usage requirements and cache sensitivity of the target inference task, the real-time idle resource status of each heterogeneous hardware is compared to select the device with sufficient resources and the highest adaptability as the target heterogeneous hardware.

[0138] For example, based on this sequence, the first C2 is extracted as the target inference task using the highest priority selection strategy. Given that the computing power requirement of C2 is 800 GFLOPS and the memory requirement is 16 GB, the matching rules between C2 and D1 in the mapping strategy are combined to query the real-time idle computing power of D1, which is 950 GFLOPS and the idle memory is 20 GB, so as to determine that D1 is the target heterogeneous hardware.

[0139] Step 1053: Perform a preemptive operation on the target heterogeneous hardware to pause the currently running low-priority task and save the corresponding hardware execution state, load the target inference task into the target heterogeneous hardware, and continue to execute the target inference task based on the hardware execution state.

[0140] Among them, hardware execution status refers to key operational information such as register data, memory read / write locations, and task execution nodes when a low-priority task is paused.

[0141] Subsequently, in step 1053, a preemptive operation is performed on the target heterogeneous hardware to send a pause instruction to the low-priority task through the system kernel-level scheduling interface. After the task execution state is frozen, the hardware execution state of the task is saved through memory snapshot technology. The hardware execution state includes the current value of the registers, memory data read and write pointers, task execution context, and unfinished intermediate calculation results. Then, the execution program of the target inference task, dependent library files, and data to be processed are loaded into the free memory area of ​​the target heterogeneous hardware to call the hardware state recovery driver to reproduce the system running environment before the low-priority task was paused. Based on this, the target inference task can be seamlessly continued to be executed.

[0142] For example, on the target heterogeneous hardware D1, a pause instruction is sent to the currently running low-priority task C1 through the system kernel-level scheduling interface. After the execution state of C1 is frozen, its hardware execution state is saved through memory snapshot technology, including the current value of registers, memory read and write pointers, execution context, and 10 incomplete intermediate calculation results. Then, the executable program of C2, dependent library files, and 35 remaining pending data are loaded into the free memory area of ​​D1 to call the hardware state recovery driver to reproduce the system running environment before C1 was paused, and seamlessly continue the execution of C2.

[0143] Step 1054: For the suspended low-priority tasks, a priority recalculation strategy is adopted to update the corresponding priority adjustment amount and reinsert them into the task queue to accelerate the artificial intelligence inference task.

[0144] The priority recalculation strategy refers to the rule of recalculating the priority adjustment amount of low-priority tasks by combining the remaining computing power of low-priority tasks and changes in system load.

[0145] In step 1054, a priority recalculation strategy is enabled for the suspended low-priority tasks, and data such as their remaining computation, execution time, historical execution efficiency, current overall system load level, and idle rate of various hardware resources are collected. The data is then substituted into a preset calculation model to update the priority adjustment amount. Subsequently, the updated low-priority tasks are inserted into the corresponding positions in the task queue according to the new adjustment amount to maintain the queue sorting rules and complete the entire preemptive task rescheduling process, thereby providing a basis for subsequent continuous scheduling.

[0146] For example, a priority recomputation strategy is enabled for the suspended C1, and its remaining computation time (20), execution time (8 seconds), historical execution efficiency (5 / second), current overall system load rate (72%), idle computing power of D1 (880 GFLOPS), and idle computing power of D2 (750 GFLOPS) are collected. Then, according to the formula... Calculate the updated priority adjustment amount, where, This is the amount of priority adjustment after the update. The original adjustment for C1 was 0.2. The remaining computational load is 20. Let C1 represent the total computational load of 50, and L represent the current system load rate, such as 0.72. D1 has 880 GFLOPS of idle computing power. Given a total computing power of 1000 GFLOPS for D1, substitute the data to obtain the updated priority adjustment amount. The value is equal to 0.019712, and an approximate value of 0.02 is taken as the priority adjustment amount after C1 is updated. Then, the updated C1 is inserted into the task queue with the adjustment amount of 0.02 to obtain a new task sequence C2, C3, C1. This queue will be used for the subsequent continuous monitoring and dynamic scheduling of the execution status of all artificial intelligence inference tasks by the system, which can ensure that resource allocation always adapts to task priority and hardware status.

[0147] This application can prioritize the execution resources of high-urgency tasks and respond quickly to task execution anomalies. It can optimize the allocation of heterogeneous hardware resources through flexible task scheduling to reduce task execution delays and improve the overall execution efficiency and system resource utilization of artificial intelligence inference tasks.

[0148] Figure 3 This is a schematic diagram illustrating a specific implementation of an artificial intelligence acceleration system based on heterogeneous hardware, provided in an embodiment of this application. (Refer to...) Figure 3 The system may include:

[0149] The acquisition module 31 is used to acquire the cache miss rate and instruction throughput generated by various heterogeneous hardware when performing artificial intelligence inference tasks, as well as the chip power consumption waveform caused by cache miss events.

[0150] The association module 32 is used to mine the cache miss rate and the instruction throughput to obtain the performance jitter value, associate the performance jitter value with the calculation link corresponding to the artificial intelligence inference task to obtain the cache sensitivity, and perform feature extraction processing on the chip power consumption waveform to obtain the access index.

[0151] The mapping module 33 is used to take the cache sensitivity and the access index as state inputs, and use a reinforcement learning strategy to perform resource mapping to obtain a mapping strategy between artificial intelligence inference tasks and heterogeneous hardware.

[0152] The calculation module 34 is used to monitor the task execution status on each heterogeneous hardware based on the mapping strategy. When the task execution status on any of the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index, the module dynamically calculates the priority adjustment amount of the corresponding artificial intelligence inference task based on the real-time execution progress and remaining deadline of the artificial intelligence inference task using a dynamic priority scheduling mechanism.

[0153] The scheduling module 35 is used to perform preemptive task rescheduling based on the priority adjustment amount, so as to accelerate the artificial intelligence inference task.

[0154] The AI ​​acceleration system based on heterogeneous hardware in this application is used to implement the aforementioned AI acceleration method based on heterogeneous hardware. Therefore, the specific implementation of the AI ​​acceleration system based on heterogeneous hardware can be found in the embodiment section of the AI ​​acceleration method based on heterogeneous hardware above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0155] like Figure 4 As shown, this application also provides an electronic device, including: a memory 41 for storing a computer program; and a processor 42 for executing the computer program to implement the steps of any of the above-described heterogeneous hardware-based artificial intelligence acceleration methods.

[0156] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described heterogeneous hardware-based artificial intelligence acceleration methods.

[0157] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0158] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the artificial intelligence acceleration method based on heterogeneous hardware described above.

[0159] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0160] The above provides a detailed description of an artificial intelligence acceleration method and system based on heterogeneous hardware provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. An artificial intelligence acceleration method based on heterogeneous hardware, characterized in that, include: Collect the cache miss rate and instruction throughput of various heterogeneous hardware when performing artificial intelligence inference tasks, as well as the chip power consumption waveform caused by cache miss events; The cache miss rate and instruction throughput are mined to obtain a performance jitter value. The performance jitter value is correlated with the computation link corresponding to the artificial intelligence inference task to obtain cache sensitivity. The chip power consumption waveform is subjected to feature extraction processing to obtain an access index. The performance jitter value refers to the characteristic pattern reflecting the fluctuation of hardware execution efficiency caused by data locality differences. The cache sensitivity refers to vector data that combines the jitter intensity and location distribution of each computation link to characterize the degree of dependence of the computation link on the cache. The access index refers to a comprehensive index that combines pulse waveform feature set and pulse density index to reflect the frequency and intensity of memory access. Using the cache sensitivity and the access index as state inputs, a reinforcement learning strategy is employed to perform resource mapping, thereby obtaining a mapping strategy between artificial intelligence inference tasks and heterogeneous hardware. Based on the mapping strategy, the task execution status on each heterogeneous hardware is monitored. When the task execution status on any of the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index, a dynamic priority scheduling mechanism is adopted to dynamically calculate the priority adjustment amount of the corresponding artificial intelligence inference task according to the real-time execution progress and remaining deadline of the artificial intelligence inference task. Based on the aforementioned priority adjustment amount, preemptive task rescheduling is performed to accelerate artificial intelligence inference tasks.

2. The method according to claim 1, characterized in that, The step of using the cache sensitivity and the access index as state inputs and employing a reinforcement learning strategy for resource mapping to obtain a mapping strategy between the artificial intelligence inference task and heterogeneous hardware includes: Based on the cache sensitivity and the access index, a state representation vector is constructed; Based on the state representation vector, a resource mapping action is generated through the decision network in the reinforcement learning strategy. The resource mapping action is defined as: selecting target execution hardware from a heterogeneous hardware set and assigning an initial execution priority for the AI ​​inference task to be scheduled. During task execution, the corresponding comprehensive reward value is calculated based on the matching degree between real-time performance data and the performance jitter value and the access index, respectively. Based on the comprehensive reward value, resources are mapped to obtain a mapping strategy.

3. The method according to claim 2, characterized in that, The calculation of the corresponding comprehensive reward value based on the matching degree between real-time performance data and the performance jitter value and the access index includes: Calculate the task completion reward value based on the number of tasks completed within the preset deadline. The system reward value is calculated based on the change in the number of tasks completed by the system per unit time relative to the historical baseline. Based on the comparison result of the first matching degree and the first preset threshold, a first penalty value is calculated, and based on the comparison result of the second matching degree and the second preset threshold, a second penalty value is calculated. The first matching degree is the matching degree between the real-time performance data and the performance jitter value, and the second matching degree is the matching degree between the real-time performance data and the access index. Calculate the resource penalty value based on the degree of decrease in instruction throughput; The task completion reward value, the system reward value, the first penalty value, the second penalty value, and the resource penalty value are weighted and merged to obtain a comprehensive reward value.

4. The method according to claim 1, characterized in that, The step of dynamically calculating the priority adjustment amount for the corresponding AI inference task based on the real-time execution progress and remaining deadline of the AI ​​inference task using a dynamic priority scheduling mechanism includes: Calculate the urgency quantification based on the real-time execution progress, remaining computational load, and remaining deadline; Based on the aforementioned urgency quantification value, a priority adjustment amount for the corresponding artificial intelligence inference task is dynamically generated through a dynamic priority scheduling mechanism. A positive priority adjustment amount indicates an increase in task priority, while a negative one indicates a decrease in task priority.

5. The method according to claim 4, characterized in that, The step of dynamically generating the priority adjustment amount for the corresponding artificial intelligence inference task based on the urgency quantification value through a dynamic priority scheduling mechanism includes: Based on the aforementioned urgency quantification, the urgency factor is calculated; The compatibility factor is calculated based on the performance matching degree and the degree of exponential deviation of heterogeneous hardware. The urgency factor and the fitness factor are normalized to obtain the urgency index and the fitness index. By using a dynamic priority scheduling mechanism, the urgency index and the suitability index are comprehensively calculated to obtain the initial priority adjustment amount; The initial priority adjustment amount is subjected to range constraint processing to obtain the priority adjustment amount corresponding to the artificial intelligence reasoning task.

6. The method according to claim 1, characterized in that, The process of mining the cache miss rate and the instruction throughput to obtain the performance jitter value includes: The cache miss rate and the instruction throughput are segmented to obtain time series segments; The waveform feature recognition method is used to detect the peaks and troughs of the cache miss rate in the time series segment to obtain the target interval; A time-series correlation analysis method is used to analyze the instruction throughput and the corresponding cache miss rate in the time series segment to determine the correlation time period when the instruction throughput decreases and the cache miss rate increases. Based on the target interval and the correlation time period, the performance jitter value is determined.

7. The method according to claim 1, characterized in that, The step of performing preemptive task rescheduling based on the priority adjustment amount to accelerate artificial intelligence inference tasks includes: Based on the priority adjustment amount, a task queue is constructed, and all artificial intelligence reasoning tasks in the task queue are sorted by priority to obtain a task sequence. Based on the task sequence, the target inference task and the corresponding target heterogeneous hardware are determined; Perform a preemptive operation on the target heterogeneous hardware to pause the currently running low-priority task and save the corresponding hardware execution state, load the target inference task into the target heterogeneous hardware, and continue to execute the target inference task based on the hardware execution state; For low-priority tasks that have been suspended, a priority recalculation strategy is adopted to update the corresponding priority adjustment amount and reinsert them into the task queue, so as to accelerate the artificial intelligence inference task.

8. An artificial intelligence acceleration system based on heterogeneous hardware, characterized in that, include: The acquisition module is used to collect the cache miss rate and instruction throughput generated by various heterogeneous hardware when performing artificial intelligence inference tasks, as well as the chip power consumption waveform caused by cache miss events. The association module is used to mine the cache miss rate and the instruction throughput to obtain a performance jitter value, associate the performance jitter value with the computation link corresponding to the artificial intelligence inference task to obtain cache sensitivity, and perform feature extraction processing on the chip power consumption waveform to obtain an access index. Here, the performance jitter value refers to the characteristic pattern reflecting the fluctuation of hardware execution efficiency caused by data locality differences, the cache sensitivity refers to vector data that combines the jitter intensity and location distribution of each computation link to characterize the degree of dependence of the computation link on the cache, and the access index refers to a comprehensive index that combines pulse waveform feature set and pulse density index to reflect the frequency and intensity of memory access. The mapping module is used to take the cache sensitivity and the access index as state inputs, and use a reinforcement learning strategy to perform resource mapping to obtain a mapping strategy between artificial intelligence inference tasks and heterogeneous hardware. The computing module is used to monitor the task execution status on each heterogeneous hardware based on the mapping strategy. When the task execution status on any of the heterogeneous hardware meets the predetermined conditions corresponding to the performance jitter value or the access index, the module dynamically calculates the priority adjustment amount of the corresponding artificial intelligence inference task by adopting a dynamic priority scheduling mechanism based on the real-time execution progress and remaining deadline of the artificial intelligence inference task. The scheduling module is used to perform preemptive task rescheduling based on the priority adjustment amount, so as to accelerate artificial intelligence inference tasks.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the heterogeneous hardware-based artificial intelligence acceleration method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the artificial intelligence acceleration method based on heterogeneous hardware as described in any one of claims 1 to 7.

Images