A performance prediction and collaborative optimization method and device based on an autonomous heterogeneous platform

Abstract

This invention relates to the field of computer multitasking parallel computing technology, and in particular to a method and apparatus for performance prediction and collaborative optimization based on an autonomous heterogeneous platform. The method includes monitoring the resources of the running autonomous heterogeneous platform to obtain its real-time performance data, wherein computational inference tasks run on different computing units of the autonomous heterogeneous platform; fitting historical performance data within a preset time window to obtain performance peaks; predicting the performance of the real-time performance data to obtain predicted performance indicators; and reallocating the computational inference tasks to different computing units of the autonomous heterogeneous platform for execution based on the predicted performance indicators and the performance peaks. This invention can improve the resource utilization and execution efficiency of autonomous heterogeneous platforms under multitasking loads.

Description

Technical Field

[0001] This invention relates to the field of computer multitasking parallel computing technology, and in particular to a method and apparatus for performance prediction and collaborative optimization based on an autonomous heterogeneous platform. Background Technology

[0002] In the digital economy era, domestically developed heterogeneous platforms have been widely applied in high-performance computing, artificial intelligence, and big data analytics. These platforms typically integrate multiple computing units, including a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a Neural Network Processing Unit (NPU). Each unit possesses different computational characteristics and performance advantages: CPUs offer strong general-purpose computing capabilities, suitable for complex control logic and low-latency serial tasks; GPUs excel at high-parallel computing, suitable for tasks such as image rendering and large-scale matrix operations; and NPUs are optimized for deep learning, suitable for neural network inference and training. With the gradual maturation and large-scale application of domestically developed processors, the scale of domestically developed heterogeneous platforms is rapidly expanding. However, limited by hardware technology maturity, power management, and stability, achieving efficient resource scheduling and performance optimization in high-load and multi-task parallel processing scenarios has become crucial for improving the application efficiency of domestically developed heterogeneous platforms.

[0003] Existing multi-task scheduling methods mainly rely on static scheduling and rule-driven resource allocation strategies, often making decisions based on task priority or hardware load. However, these methods ignore the dynamic changes in the system during actual operation. For example, existing methods usually only schedule according to the current hardware load without considering performance fluctuations and load changes during task execution, resulting in inflexible resource scheduling and easy resource overload or idleness.

[0004] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0005] In response to the above-mentioned deficiencies or improvement needs of existing technologies, this invention proposes a performance prediction and collaborative optimization method and apparatus based on an autonomous heterogeneous platform, which can improve the resource utilization and execution efficiency of the system under multi-task loads.

[0006] The embodiments of the present invention adopt the following technical solutions: In a first aspect, the present invention provides a performance prediction and collaborative optimization method based on an autonomous heterogeneous platform, specifically: monitoring the resources of the autonomous heterogeneous platform in operation and obtaining real-time performance data of the autonomous heterogeneous platform, wherein the computational inference task runs on different computing units of the autonomous heterogeneous platform; The historical performance data within a preset time window is fitted to obtain the performance peak. Performance prediction is performed on real-time performance data to obtain predicted performance metrics; Based on the predicted performance metrics and the peak performance, the computational inference task is redistributed to different computing units of the autonomous heterogeneous platform for execution.

[0007] Preferably, the method further includes: Instrument the target kernel function of the running autonomous heterogeneous platform to obtain the resource status of each computing unit when running tasks; The resource status of each computing unit is converted into performance data to obtain the real-time performance data of the autonomous heterogeneous platform.

[0008] Preferably, the step of fitting historical performance data within a preset time window to obtain the performance peak includes: The historical performance data within a preset time window is preprocessed to obtain the raw performance data. Outlier detection is performed on the original performance data, and outlier performance data is removed from the original performance data to obtain the target performance data; The target performance data is assigned sampling weights based on its importance. The performance data with assigned sampling weights is reconstructed into a smooth performance sequence; Extreme value detection is performed on the performance sequence, and the peak point in the performance sequence is taken as the performance peak.

[0009] Preferably, the formula for outlier detection is as follows: ; in, Let i be the standard score for the i-th performance data point. For the i-th performance data, The mean of the original performance data. denoted as the standard deviation of the original performance data, and n is the total number of original performance data.

[0010] Preferably, the formula for reconstructing a smooth performance sequence is as follows: ; in, This represents the i-th target performance data. This represents the i-th reconstructed data. This represents the sampling weight of the i-th target performance data. Indicates to The difference term, This represents the smoothing intensity parameter, and n represents the total number of target performance data.

[0011] Preferably, the formula for extreme value detection is as follows: ; in, This represents the probability that the performance value belongs to an extreme value. For the input performance value, For position parameters, For scale parameters, For shape parameters.

[0012] Preferably, the method further includes: The computational reasoning task is transformed into an operator computation graph with task operators as nodes; According to the characteristics of the computational inference task and the topological structure of the operator computation graph, the operator computation graph is divided into multiple subgraphs; Based on the predicted performance metrics and the performance peak, each subgraph is redistributed to different computing units on the autonomous heterogeneous platform.

[0013] Preferably, the method further includes: The real-time performance data is standardized to obtain an input sequence that is suitable for the performance prediction model; The input sequence is fed into a trained performance prediction model, which outputs a predicted performance index within a preset future time window. The training process of the performance prediction model employs an adaptive learning rate optimizer to dynamically adjust the learning rate and improve the model's convergence efficiency.

[0014] In a second aspect, the present invention provides a performance prediction and collaborative optimization apparatus based on an autonomous heterogeneous platform, the apparatus comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor to perform the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform as described in the first aspect.

[0015] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions, which are executed by one or more processors to perform the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform as described in the first aspect.

[0016] Compared with existing technologies, the advantages of this invention are as follows: Real-time resource monitoring of the autonomous heterogeneous platform is performed, continuously collecting multi-dimensional resource status data (including performance data such as utilization rate of computing units such as CPU, GPU, and NPU, memory bandwidth, and cache hit rate) and combining historical performance data to fit performance peaks. Furthermore, a deep learning method is introduced to construct a performance prediction model. Based on the aforementioned multi-dimensional resource status, the performance prediction model predicts the performance evolution trend of the autonomous heterogeneous platform within a short-term window. On this basis, according to the characteristics of the computational inference task and the current resource availability and predicted performance of each computing unit on the autonomous heterogeneous platform, the computational inference task is adaptively and dynamically reallocated to the most suitable computing unit. This mechanism significantly improves the overall resource utilization of the autonomous heterogeneous platform while strictly ensuring the task service level and achieves truly task-oriented intelligent scheduling, thus effectively coping with complex heterogeneous computing scenarios such as high concurrency, multimodality, and strong fluctuations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0018] Figure 1 This is a flowchart illustrating a performance prediction and collaborative optimization method based on an autonomous heterogeneous platform provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the method for obtaining real-time performance data provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the method for obtaining performance peaks through fitting, as provided in an embodiment of the present invention. Figure 4 This is a flowchart illustrating the scheduling optimization method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the operator computation graph provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of segmenting the operator computation graph according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the allocation sub-graph provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform provided in an embodiment of the present invention; Figure 9 This is a flowchart illustrating the method for obtaining predicted performance indicators provided in an embodiment of the present invention; Figure 10This is a flowchart of the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform provided in an embodiment of the present invention; Figure 11 A schematic diagram of a performance prediction and collaborative optimization device based on an autonomous heterogeneous platform provided in an embodiment of the present invention; The accompanying figure is labeled as follows: 21: Processor; 22: Memory. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0021] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0022] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0023] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0024] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] Example 1: like Figure 1 As shown, Embodiment 1 of the present invention provides a performance prediction and collaborative optimization method based on an autonomous heterogeneous platform. The method specifically includes the following steps: Step 101: Monitor the resources of the running autonomous heterogeneous platform and obtain the real-time performance data of the autonomous heterogeneous platform, wherein the computational inference task runs on different computing units of the autonomous heterogeneous platform.

[0026] Among them, computational inference tasks refer to target detection tasks, target classification tasks, and target segmentation tasks; different computing units include CPU, GPU, and NPU; performance data includes CPU utilization, cache hit rate, GPU utilization, NPU utilization, input / output (IO), i.e., speed and network speed, etc.

[0027] In one embodiment, running the computational inference task on an autonomous heterogeneous platform essentially means breaking down the computational inference task into multiple subtasks and assigning each subtask to different computing units on the autonomous heterogeneous platform. For example, for an object detection task, the task can be broken down into an image preprocessing task, a backbone network feature extraction task, a multi-scale fusion task, a detection head inference task, and a result packaging and transmission task. The image preprocessing task can be assigned to the CPU, the backbone network feature extraction task to the GPU or NPU, the multi-scale fusion task to the NPU, the detection head inference task to the NPU or GPU, and the result packaging and transmission task to the CPU. Furthermore, for methods to obtain real-time performance data, please refer to [link to relevant documentation]. Figure 2 .

[0028] Step 102: Fit the historical performance data within the preset time window to obtain the performance peak.

[0029] In one embodiment, mathematical modeling and curve fitting are performed on historical performance data (such as CPU utilization, memory usage, network throughput, response latency, etc.) within a preset time window to identify and predict the performance peak that the system or component may reach during that time period. This performance peak serves as the basis for subsequent task allocation. However, a time window that is too short may cause noise interference in the fitting results; a time window that is too long may mask short-term bursts of peak performance. The window length can be reasonably set according to the business cycle, or the user can customize the length of the preset time window (e.g., the last 7 days, monthly peak hours, weekdays 11:00–18:00, etc.). Furthermore, different performance indicators may require different fitting models. Adaptive algorithms can be used, or the user can manually specify the fitting model. For methods to obtain performance peaks, please refer to [link / reference needed]. Figure 3 .

[0030] Step 103: Perform performance prediction on real-time performance data to obtain predicted performance metrics.

[0031] In one embodiment, a performance prediction model based on a deep neural network is constructed. The hidden layers of the performance prediction model employ the ReLU (Linear Rectification Function) activation function to enhance the model's nonlinear expressive power, and the mean squared error is defined as the loss function to measure the deviation between the predicted and actual values. The trained performance prediction model is used to predict the performance of real-time performance data to reflect the performance of the autonomous heterogeneous platform in the future. The performance prediction model can be a Long Short-Term Memory (LSTM) neural network.

[0032] Step 104: Based on the predicted performance index and the performance peak, the computational inference task is redistributed to different computing units of the autonomous heterogeneous platform for execution.

[0033] In one embodiment, the performance peak is a theoretical peak value that enables the autonomous heterogeneous platform to operate efficiently. The peak value represents the level that the autonomous heterogeneous platform can achieve when resources are reasonably allocated and there are no obvious bottlenecks. If the predicted performance index cannot reach the performance peak, it can be considered that there is a performance bottleneck in the system (which may be due to uneven allocation of computational inference tasks leading to resource overload or resource idleness). Therefore, it is necessary to reallocate and optimize the computational inference tasks to improve the overall system performance.

[0034] Reassigning the computational inference task to different computing units of the autonomous heterogeneous platform means reassigning the subtasks that were split in step 101 to different computing units of the autonomous heterogeneous platform for execution.

[0035] In this embodiment, real-time resource monitoring is performed on the autonomous heterogeneous platform, continuously collecting multi-dimensional resource status data (including performance data such as utilization rate of computing units such as CPU, GPU, and NPU, memory bandwidth, and cache hit rate), and fitting performance peaks by combining historical performance data. A deep learning method is introduced to construct a performance prediction model. Based on the aforementioned multi-dimensional resource status, the performance prediction model predicts the performance evolution trend of the autonomous heterogeneous platform within a short-term window. Furthermore, based on the characteristics of the computational inference task and the current resource availability and predicted performance of each computing unit on the autonomous heterogeneous platform, the computational inference task is adaptively and dynamically reallocated to the most suitable computing unit. This mechanism significantly improves the overall resource utilization of the autonomous heterogeneous platform while strictly ensuring task service levels and achieves truly task-oriented intelligent scheduling, thus effectively coping with complex heterogeneous computing scenarios such as high concurrency, multimodality, and strong fluctuations.

[0036] like Figure 2 As shown, Embodiment 1 of the present invention provides a method for obtaining real-time performance data, which specifically includes the following steps: Step 201: Instrument the target kernel function of the running autonomous heterogeneous platform to obtain the resource status of each computing unit when running tasks.

[0037] Kernel functions refer to C functions that implement core functions (initialization, memory allocation, address mapping, task submission, etc.); target kernel functions refer to kernel functions related to the state of the resources to be acquired. In an independent heterogeneous platform, access to system resources (such as memory allocation, I / O submission, task scheduling, etc.) by various computing units (such as CPU, GPU, NPU or self-developed accelerators) must be completed through kernel functions.

[0038] In one embodiment, on an autonomous heterogeneous platform, when a computing unit (such as a GPU) runs, its resource requests (such as memory allocation, address mapping, and context creation) are typically completed by calling specific kernel functions. By deploying extended Berkeley Packet Filter (eBPF) monitoring programs on these specific kernel functions, information such as system call behavior, resource usage, and execution latency related to the computing unit can be captured in real time without modifying the kernel source code or restarting the system. The method for instrumenting kernel functions specifically involves writing and compiling an eBPF monitoring program in user space, loading it into the kernel, and attaching it to selected kernel function execution points. When the kernel runs to these execution points, it automatically triggers the execution of the corresponding eBPF detection program, thereby achieving dynamic instrumentation of specified kernel paths.

[0039] Step 202: Convert the resource status of each computing unit into performance data to obtain the real-time performance data of the autonomous heterogeneous platform.

[0040] In one embodiment, hardware performance counters such as cache hit rate cannot be directly obtained through kernel functions. However, they can be combined with eBPF and user-space tools (such as perf) to achieve coordinated observation of microarchitectural metrics such as cache efficiency. The entire process does not require modification of the kernel function code. Only the eBPF program needs to be compiled and loaded in user space and mounted to the selected kernel function. The collection can be automatically triggered at system runtime. Here, eBPF is responsible for associating process context and time window, and perf is responsible for reading hardware performance events. For example, in the block device I / O path, by inserting eBPF monitoring programs into kernel function 1 and kernel function 2, the start time, data size (number of sectors × 512 bytes), and completion time of each I / O request can be captured, thereby calculating I / O throughput and latency in real time. For self-developed accelerators, if the driver calls kernel function A when submitting a task or calls kernel function B when allocating Direct Memory Access (DMA) memory, instrumentation can be inserted at the entry and return points of kernel function A or kernel function B to record the amount of data transferred and the execution time, thereby deriving key performance indicators such as DMA bandwidth and task throughput.

[0041] like Figure 3 As shown in the figure, this embodiment provides a method for fitting to obtain the performance peak, which specifically includes the following steps: Step 301: Preprocess the historical performance data within the preset time window to obtain the raw performance data.

[0042] Preprocessing includes operations such as noise reduction, missing value imputation, and smoothing to improve the quality and accuracy of performance data.

[0043] Step 302: Perform outlier detection on the original performance data and remove outlier performance data from the original performance data to obtain the target performance data.

[0044] In one embodiment, outlier detection yields a standard score for each performance data point, representing the degree to which each performance data point deviates from the mean. If the performance data... The absolute value of the standard score is greater than the preset score, which indicates performance data. If it is outlier performance data, it will be removed from the original performance data.

[0045] In one embodiment, the formula for outlier detection is as follows: ; in, Let i be the standard score for the i-th performance data point. For the i-th performance data, The mean of the original performance data. denoted as the standard deviation of the original performance data, and n is the total number of original performance data.

[0046] Step 303: Assign sampling weights to the target performance data according to their importance.

[0047] In one embodiment, sampling weights are assigned to target performance data, giving higher weights to key points that significantly impact peak values ​​during reconstruction.

[0048] Step 304: Reconstruct the performance data with sampling weights into a smooth performance sequence.

[0049] In one embodiment, in convex optimization reconstruction, the sampling weights from step 303 are introduced. Adjust the fitting strength of the reconstruction curve to the key points to obtain a performance sequence that is both smooth and retains key features without changing the original data as much as possible.

[0050] In one embodiment, the formula for reconstructing a smooth performance sequence is as follows: ; in, This represents the performance data of the i-th target. This represents the i-th reconstructed data. This represents the sampling weight of the i-th target performance data. Indicates to The difference term, This represents the smoothing strength parameter, which can suppress jitter and make the curve smoother. n represents the total number of target performance data.

[0051] Step 305: Perform extreme value detection on the performance sequence and take the peak point in the performance sequence as the performance peak.

[0052] In one embodiment, the formula for extreme value detection is as follows: ; in, This represents the probability that the performance value belongs to an extreme value. For the input performance value, Here, is a location parameter, representing the central location of the extreme value distribution and the expected value of the corresponding performance peak. This is a scaling parameter used to measure the extent of distribution expansion and control the fluctuation range of performance peaks. This is a shape parameter used to determine the tail characteristics of the distribution.

[0053] like Figure 4 As shown in the figure, this embodiment provides a scheduling optimization method, which specifically includes the following steps: Step 401: Transform the computational reasoning task into an operator computation graph with task operators as nodes.

[0054] In one embodiment, the computational inference task is converted into a raw computation graph using the Baidu Paddle-lite framework. The execution order of operators, data flow paths, and connections between operators in the operator computation graph are analyzed to simplify the graph, remove redundant nodes, and generate an operator computation graph. (See also...) Figure 5 Baidu's Paddle-Lite framework transforms computational inference tasks into a raw computation graph. This raw computation graph includes various types of nodes: operator nodes, other I / O nodes (such as input / output tensor placeholders), and I / O master nodes (representing the main input / output interface of the entire subgraph or model). Traversing this raw computation graph, the framework analyzes the node attributes and functions of each node, clearly distinguishing between operator nodes (i.e., units that execute specific computational logic, such as convolution and pooling) and non-operator nodes (primarily used for data transfer, placeholders, or connections). Based on this distinction, the raw computation graph is further decomposed into two core components: the operator computation graph and the operator I / O information table. The operator computation graph describes the topological dependencies and execution order between operators; the operator I / O information table records metadata such as the names, data types, shapes, and memory layouts of the input and output tensors associated with each operator.

[0055] Step 402: According to the characteristics of the computational inference task and the topological structure of the operator computation graph, the operator computation graph is divided into multiple subgraphs.

[0056] In one embodiment, the computation graph is partitioned according to the characteristics of the computational inference task and the topology of the operator computation graph. During partitioning, parallelism between subgraphs and seriality within subgraphs must be guaranteed. See also... Figure 6 By traversing the operator computation graph structure, operators that support hardware acceleration (such as convolution, matrix multiplication, activation functions, etc.) are identified, and these operators are marked accordingly based on the capabilities of the target backend (e.g., "can be fused", "requires quantization", and "supports Ascend custom instructions"). Operators marked as capable of being efficiently executed by dedicated hardware are aggregated into one or more accelerated subgraphs, and abnormal operators that cannot run efficiently on the computing unit are detected. Based on this, subgraph segmentation is further performed to divide the operator computation graph into several continuous subgraph regions (representing operators that can be efficiently executed by dedicated hardware) or individual operator nodes (representing abnormal operators that cannot run efficiently on the computing unit).

[0057] In this embodiment, the splitting of the computational inference task can be performed only once. That is, the splitting in step 101 can be performed by the methods of steps 401 and 402 above. When the prediction performance index does not reach the performance peak, the dynamic optimization strategy is triggered, and the split computational inference task is redistributed to different computing units of the autonomous heterogeneous platform according to the method of step 403.

[0058] Step 403: Based on the predicted performance index and the performance peak, redistribute each subgraph to different computing units on the autonomous heterogeneous platform.

[0059] In one embodiment, the optimal implementation type is dynamically selected for each operator based on the computing power, memory bandwidth, and supported instruction set of the computing unit. The same logical operator (such as convolution, pooling, or matrix multiplication) may correspond to multiple underlying implementations on different hardware platforms. For example, CPU operators are suitable for general computing scenarios; GPU operators are suitable for handling large-scale tensor operations; while NPU operators are highly optimized kernels customized for dedicated AI chips, enabling high-throughput inference with extremely low power consumption. Paddle-Lite reallocates the subgraph based on the resource usage of each computing unit (including CPU utilization, GPU utilization, NPU utilization, and I / O rate) and the computational requirements of the subtasks. Figure 1 For example, according to the sub Figure 1 The characteristics of the sub-type (such as computationally intensive, high memory access bandwidth requirements, or high parallelism) determine the most suitable sub-type. Figure 1 The efficient computing unit A (e.g., GPU or NPU) is evaluated, its current resource utilization (including performance data such as core utilization, memory usage, power consumption, and temperature) is obtained, and it is determined whether computing unit A can meet the requirements of the sub-processor. Figure 1 The resource requirements corresponding to the subtask. If the remaining resources of computing unit A are sufficient to support the subtask... Figure 1 For efficient execution of the corresponding task, the sub-task will be executed. Figure 1 The corresponding subtask is scheduled to run on computing unit A; if the resource utilization of computing unit A is already close to its limit and cannot meet the subtask's requirements... Figure 1 If the performance requirements are not met, a degradation strategy will be initiated, such as reducing the sub-performance requirements. Figure 1 Execution is migrated to a suboptimal computing unit (such as a CPU or a low-load DSP) until a suitable sub-unit is found. Figure 1 Until the appropriate computational units for the corresponding subtasks are found, the methods described above are used to redistribute all subgraphs corresponding to subtasks onto the computational units of the autonomous heterogeneous platform. For example, sub... Figure 1This is an image recognition sub-model containing a large number of convolutional operations, best suited for running on the NPU. However, if the current NPU's computational unit utilization is detected to be 95% and the remaining GPU memory is less than 10%, making it impossible to guarantee the real-time inference performance of the image recognition sub-model, then the image recognition sub-model will be downgraded and allocated to the CPU for execution. A lightweight inference engine will be enabled to adapt to the CPU architecture, thereby maintaining overall system stability while ensuring task completion. See also... Figure 7 Based on the operator implementation type, operators are assigned to different computing units (CPU operators, GPU operators, and NPU operators) for execution. In this way, the system can maximize the computing power of heterogeneous hardware, ensure efficient collaborative execution of inference tasks across different devices, thereby improving the overall system resource utilization and optimizing the efficiency of multi-task parallel processing.

[0060] See Figure 8 In the Kylin OS environment, the efficient execution of computational inference tasks (such as Task 1, Task 2, and Task 3) on an independent heterogeneous platform supporting domestic CPUs, GPUs, and NPUs relies on the Baidu PaddleLite inference framework. Specifically, the computational graph is obtained through the Baidu PaddleLite inference framework, and then it is segmented into operators. Subsequently, abnormal operators (such as unsupported or degraded operators) are identified based on hardware capabilities, and subgraph allocation is completed (scheduling different subgraphs to the most suitable computational units). That is, the general computing part is handled by the domestic CPU, the graphics acceleration task is delivered to the domestic GPU for execution through the OpenCL interface and calls to the GPU driver, and the AI-specific computing is delivered to the domestic NPU for execution based on the CANN software stack and NPU driver.

[0061] like Figure 9 As shown in the figure, this embodiment provides a method for obtaining a predicted performance index, which specifically includes the following steps: Step 501: Standardize the real-time performance data to obtain an input sequence that fits the performance prediction model.

[0062] The standardization process includes outlier removal, normalization, and time-series processing. The performance prediction model can utilize an LSTM neural network.

[0063] Step 502: Input the input sequence into the trained performance prediction model and output the predicted performance index within a future preset time window. The training process of the performance prediction model adopts an adaptive learning rate optimizer to dynamically adjust the learning rate to improve the convergence efficiency of the model.

[0064] In one embodiment, to construct a high-precision performance prediction model, historical performance data must first be preprocessed according to the standardization method in step 501 to eliminate dimensional differences and improve the stability of model training. The standardized historical performance data is then organized into a time series format to meet the input requirements of the performance prediction model. Furthermore, during training, a dynamic learning rate adjustment strategy is introduced, adaptively reducing the learning rate based on changes in validation loss, thereby accelerating convergence and avoiding getting trapped in local optima. After model training is complete, the input sequence is fed into the trained model to output predicted performance metrics within a preset future time window, providing a basis for subsequent system optimization.

[0065] See Figure 10 eBPF technology instrumentes kernel functions to monitor the performance information of autonomous heterogeneous platforms, collecting system performance data including CPU information (CPU utilization and cache hit rate, etc.), GPU information (2D GPU utilization and 3D GPU utilization), and NPU information (Control CPU usage and NPU usage). The system uses memory, I / O information (disk read rate and disk write rate), and network information (network send rate and network termination rate). On one hand, it trains an LSTM neural network with the collected system performance data to obtain a performance prediction model. On the other hand, it inputs the collected real-time performance data into the trained LSTM neural network to predict future performance data. The collected system performance data is preprocessed, subjected to swarm sampling, convex optimization, and other operations to fit the peak performance. If the predicted performance cannot obtain the peak performance, the automatic tuning module is activated. The automatic tuning module uses the PaddleLite framework to obtain the computation task graph, preprocesses the computation graph according to the operator relationship, segments the computation graph according to the task characteristics and computation graph topology, and allocates tasks and schedules resources according to the computation requirements of sub-tasks and the device resource situation to achieve the tuning effect. In addition, the tuning effect can be fed back to optimize the performance prediction model.

[0066] Example 2: Based on the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform provided in the foregoing embodiments, the present invention also provides an apparatus for implementing the above method based on an autonomous heterogeneous platform, such as... Figure 11 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device for performance prediction and collaborative optimization based on an autonomous heterogeneous platform in this embodiment includes one or more processors 21 and a memory 22. Figure 11 Take a processor 21 as an example.

[0067] Processor 21 and memory 22 can be connected via a bus or other means. Figure 11Taking the example of a connection between China and Israel via a bus.

[0068] The memory 22, as a non-volatile computer-readable storage medium for a performance prediction and collaborative optimization method based on an autonomous heterogeneous platform, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform in the aforementioned embodiments. The processor 21 executes various functional applications and data processing of the performance prediction and collaborative optimization device based on an autonomous heterogeneous platform by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby realizing the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform as described in the aforementioned embodiments.

[0069] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0070] The program instructions / modules are stored in memory 22. When executed by one or more processors 21, they perform the performance prediction and collaborative optimization method based on the autonomous heterogeneous platform described in the foregoing embodiments.

[0071] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 11 One of the processors 21 enables the above one or more processors to execute the performance prediction and collaborative optimization method based on the autonomous heterogeneous platform in the foregoing embodiments.

[0072] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0073] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A performance prediction and collaborative optimization method based on an autonomous heterogeneous platform, characterized in that, include: Resource monitoring is performed on the autonomous heterogeneous platform in operation to obtain real-time performance data of the autonomous heterogeneous platform, wherein computational inference tasks run on different computing units of the autonomous heterogeneous platform; The historical performance data within a preset time window is fitted to obtain the performance peak. Performance prediction is performed on real-time performance data to obtain predicted performance metrics; Based on the predicted performance metrics and the peak performance, the computational inference task is redistributed to different computing units of the autonomous heterogeneous platform for execution.

2. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 1, characterized in that, The method further includes: Instrument the target kernel function of the running autonomous heterogeneous platform to obtain the resource status of each computing unit when running tasks, and convert the resource status of each computing unit into performance data to obtain the real-time performance data of the autonomous heterogeneous platform.

3. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 1, characterized in that, The process of fitting historical performance data within a preset time window to obtain performance peaks includes: The historical performance data within a preset time window is preprocessed to obtain the raw performance data. Outlier detection is performed on the original performance data, and outlier performance data is removed from the original performance data to obtain the target performance data; The target performance data is assigned sampling weights based on its importance. The performance data with assigned sampling weights is reconstructed into a smooth performance sequence; Extreme value detection is performed on the performance sequence, and the peak point in the performance sequence is taken as the performance peak.

4. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 3, characterized in that, The formula for outlier detection is as follows: ； in, Let i be the standard score for the i-th performance data point. For the i-th performance data, The mean of the original performance data. denoted as the standard deviation of the original performance data, and n is the total number of original performance data.

5. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 3, characterized in that, The formula for reconstructing a smooth performance sequence is as follows: ； in, This represents the performance data of the i-th target. This represents the i-th reconstructed data. This represents the sampling weight of the i-th target performance data. Indicates to The difference term, This represents the smoothing intensity parameter, and n represents the total number of target performance data.

6. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 3, characterized in that, The formula for extreme value detection is as follows: ； in, This represents the probability that the performance value belongs to an extreme value. For the input performance value, For position parameters, For scale parameters, For shape parameters.

7. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 1, characterized in that, The method further includes: The computational reasoning task is transformed into an operator computation graph with task operators as nodes; According to the characteristics of the computational inference task and the topological structure of the operator computation graph, the operator computation graph is divided into multiple subgraphs; Based on the predicted performance metrics and the performance peak, each subgraph is redistributed to different computing units on the autonomous heterogeneous platform.

8. The performance prediction and collaborative optimization method based on an autonomous heterogeneous platform according to claim 1, characterized in that, The method further includes: The real-time performance data is standardized to obtain an input sequence that is suitable for the performance prediction model; The input sequence is fed into a trained performance prediction model, which outputs a predicted performance index within a preset future time window. The training process of the performance prediction model employs an adaptive learning rate optimizer to dynamically adjust the learning rate and improve the model's convergence efficiency.

9. A performance prediction and collaborative optimization device based on an autonomous heterogeneous platform, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the performance prediction and collaborative optimization method based on an autonomous heterogeneous platform as described in any one of claims 1-8.

Images