Npu-based storage-computing integrated server optimization method

Abstract

The application discloses an NPU-based memory-computing integrated server optimization method, relates to the technical field of server architecture, and discloses the NPU-based memory-computing integrated server optimization method, which generates a reconstructed computing unit by dynamically adjusting the configuration of an arithmetic logic unit, a multiplication-accumulation computing unit and a cache module, combines heterogeneous hardware performance evaluation and a task migration mechanism, realizes dynamic optimization configuration and load balancing of computing resources, and can improve task execution efficiency and optimize the synergy energy efficiency ratio of heterogeneous hardware.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of server architecture, and particularly relates to an optimization method for a storage-computing integrated server based on an NPU. BACKGROUND

[0002] With the rapid development of artificial intelligence, big data and other technologies, higher and higher requirements are put forward for the computing power and energy efficiency ratio of servers. The storage-computing integrated server, as a new server architecture, integrates storage and computing functions together, reduces the transmission of data between storage and computing units, and thus improves the computing efficiency and energy efficiency ratio to a certain extent. The NPU, as a processor specially designed for neural network computing, has significant advantages in the field of artificial intelligence computing. However, the existing hardware resource configuration strategy of the storage-computing integrated server based on the NPU is rigid, and cannot dynamically adjust the arithmetic logic unit, multiplication accumulation unit and cache module and other key components of the computing unit according to the task characteristics. Moreover, the task allocation algorithm of the existing technology is too simple, and fails to establish a decision model that comprehensively considers the computing complexity, data dependency relationship and hardware performance index, resulting in low synergy efficiency between heterogeneous hardware.

[0003] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide an optimization method for a storage-computing integrated server based on an NPU, aiming to improve the task execution efficiency and optimize the synergy energy efficiency ratio of heterogeneous hardware.

[0005] To achieve the above purpose, the present application provides an optimization method for a storage-computing integrated server based on an NPU, which comprises:

[0006] The configuration instruction generation module receives the computing task type information, and generates hardware resource configuration parameters based on the computing task type information;

[0007] The hardware resource configuration parameters are written into the control register, and the hardware resource configuration parameters in the control register are read by the configuration circuit, so as to dynamically adjust the subunit enable state of the arithmetic logic unit, the parallelism of the multiplication accumulation computing unit and the hierarchical configuration of the cache module, and generate the reconstructed computing unit;

[0008] The task analysis module extracts the computing complexity data, data dependency relationship data and task priority data of the task to be allocated;

[0009] The hardware performance evaluation module collects real-time load data of the NPU, CPU and GPU, and generates heterogeneous hardware performance index data in combination with the pre-stored hardware peak computing power data and power consumption model data;

[0010] The computing complexity data, the data dependency data, the task priority data, and the heterogeneous hardware performance index data are received by a dynamic task allocation module, and a task allocation decision data is generated based on an energy efficiency ratio matching strategy;

[0011] The task allocation decision data is sent to a scheduling module, and a task is scheduled to a target heterogeneous hardware for execution according to the data dependency data, wherein the reconstructed computing unit is called to execute a computing task when the target heterogeneous hardware is an NPU, and a task execution result data is generated;

[0012] Real-time load data of each heterogeneous hardware is monitored by a load balancing module, and a task migration instruction data is generated when a load deviation exceeds a threshold, and the task allocation decision data is adjusted based on the task migration instruction data.

[0013] In an embodiment, the dynamic adjustment of the subunit enablement state of the arithmetic logic unit includes:

[0014] The operation precision requirement data in the hardware resource configuration parameters is parsed, and subunit enablement quantity and data bit width configuration parameters are generated;

[0015] The data bit width of the arithmetic logic unit is set according to the data bit width configuration parameters, a corresponding number of operation subunits are activated based on the subunit enablement quantity, and a reconstructed arithmetic logic unit is generated as a component of the reconstructed computing unit.

[0016] In an embodiment, the dynamic adjustment of the multiply-accumulate computing unit includes:

[0017] The parallelism requirement data in the hardware resource configuration parameters is parsed, and a parallel computing channel quantity configuration parameter of the multiply-accumulate computing unit is generated;

[0018] A corresponding number of multiply-accumulate computing subunits are activated and data transmission paths are reorganized according to the parallel computing channel quantity configuration parameter, and a reconstructed multiply-accumulate computing unit is generated as a component of the reconstructed computing unit.

[0019] In an embodiment, the hierarchical configuration of the cache module includes:

[0020] The cache correlation degree parameter and the data locality feature data in the hardware resource configuration parameters are parsed;

[0021] The address mapping strategy of the secondary cache is set according to the cache correlation degree parameter, and the space ratio of the primary cache and the secondary cache is dynamically allocated based on the data locality feature data, and a hierarchically configured cache module is generated as a component of the reconstructed computing unit.

[0022] In an embodiment, the extraction of the data dependency data includes:

[0023] parsing the task input and output data sets, identifying coincident fields of the preceding task output data sets and the current task input data sets;

[0024] establishing a task interdependence graph according to the coincident fields, and generating data dependence relationship data.

[0025] In an embodiment, the step of generating heterogeneous hardware performance index data by the hardware performance evaluation module includes:

[0026] collecting real-time computing resource occupancy data and memory bandwidth utilization data through monitoring agents deployed on the NPU, CPU and GPU;

[0027] inputting the real-time computing resource occupancy data and the pre-stored hardware peak computing power data into a power consumption model to generate energy efficiency ratio prediction data;

[0028] aggregating the computing resource occupancy data, the memory bandwidth utilization data and the energy efficiency ratio prediction data to generate the heterogeneous hardware performance index data.

[0029] In an embodiment, the step of generating task allocation decision data based on the energy efficiency ratio matching strategy by the dynamic task allocation module includes:

[0030] selecting a candidate hardware set that meets the computing power requirement according to the computing complexity data and the peak computing power data in the heterogeneous hardware performance index data;

[0031] calculating data transmission cost data among the candidate hardware based on the data dependence relationship data, and generating comprehensive evaluation score data in combination with the energy efficiency ratio prediction data;

[0032] selecting the candidate hardware with the highest comprehensive evaluation score data as the target heterogeneous hardware to generate the task allocation decision data.

[0033] In an embodiment, the step of scheduling the task to the target heterogeneous hardware for execution according to the data dependence relationship data includes:

[0034] topologically sorting the task execution order according to the data dependence relationship data to generate task sequence data;

[0035] sending the task to the target heterogeneous hardware according to the task sequence data and the task allocation decision data;

[0036] When the task is completed, writing the task execution result data into a shared storage area and triggering subsequent task allocation based on the data dependence relationship data.

[0037] In an embodiment, the step of calling the reconstructed computing unit to perform a computing task comprises:

[0038] When performing a neural network training task, forward propagation computation is performed by the NPU computing sub-unit in the reconstructed computing unit to generate intermediate result data;

[0039] The intermediate result data is transmitted to the general computing module in the reconstructed computing unit through a high-speed data bus;

[0040] Gradient computation instructions are executed in the general computing module to generate parameter update data, which is fed back to the NPU computing sub-unit to complete back propagation computation and generate task execution result data.

[0041] In an embodiment, the step of monitoring real-time load data of each heterogeneous hardware by the load balancing module, generating task migration instruction data when the load deviation exceeds a threshold, and adjusting task allocation decision data based on the task migration instruction data comprises:

[0042] Periodically collecting resource occupation rate data of each heterogeneous hardware, and calculating load variance data;

[0043] When the load variance data exceeds a threshold, selecting low-priority task data and corresponding data dependency relationship data in a high-load hardware;

[0044] Generating task migration cost data based on the task data and the data dependency relationship data, generating task migration instruction data according to the task migration cost data, and adjusting task allocation decision data based on the task migration instruction data.

[0045] The NPU-based storage-computing integrated server optimization method proposed in the present application generates a reconstructed computing unit by dynamically adjusting the configuration of arithmetic logic units, multiply-accumulate computing units, and cache modules, combines heterogeneous hardware performance evaluation and task migration mechanisms, realizes dynamic optimization configuration and load balancing of computing resources, and can improve task execution efficiency and optimize the synergy energy efficiency ratio of heterogeneous hardware. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0048] Figure 1 A flowchart provided by an embodiment of the NPU-based storage-computing integrated server optimization method of the present application is shown in FIG. 1.

[0049] Figure 2 A detailed flowchart of step S400 in the embodiment of the present application is shown in FIG. 4. Figure 1

[0050] Figure 3 A detailed flowchart of step S500 in the embodiment of the present application is shown in FIG. 5. Figure 1

[0051] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0052] The technical solutions of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0053] It should be understood that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0054] In the prior art, the storage-computing integrated server reduces data transmission delay by integrating storage and computing units, but the NPU computing unit function is fixed and cannot adapt to diversified task requirements. When facing the mixed scene of neural network training, inference and general computing, the existing system adopts the NPU and CPU / GPU collaborative mode with fixed architecture, resulting in insufficient utilization of hardware resources. For example, in a video analysis server, when the NPU performs a target detection task, it cannot dynamically adjust the computing unit configuration and needs to wait for the CPU to complete the preprocessing data, causing task queue blocking. At the same time, there is a lack of real-time load sensing mechanism among heterogeneous hardware, and high-priority tasks may be assigned to overloaded GPUs for execution, causing the overall energy efficiency ratio to decline.

[0055] ​​To solve the above problems, the research and development team observed the core contradiction between the fixed function of the computing unit and the lack of cooperation with heterogeneous hardware in the traditional server architecture. By analyzing the typical data center task log, it was found that the neural network training task had the characteristics of alternating high computing power demand and low load period intermittently, and the existing hardware could not dynamically adjust the computing resources. Further research found that the task allocation strategy did not consider the real-time energy efficiency ratio change of the hardware, resulting in frequent inefficient task migration. Based on this, the computing unit reconstruction capability and dynamic task allocation are combined, and the system-level optimization is realized through dynamic configuration of hardware resources and load-aware scheduling.

[0056] Based on this, the embodiment of the application provides an NPU-based storage-computing integrated server optimization method, referring to Figure 1 , the NPU-based storage-computing integrated server optimization method comprises steps S100-S700, wherein:

[0057] Step S100, receiving computing task type information through a configuration instruction generation module, and generating hardware resource configuration parameters based on the computing task type information;

[0058] Step S200, writing the hardware resource configuration parameters into a control register, reading the hardware resource configuration parameters in the control register through a configuration circuit, dynamically adjusting the subunit enable state of the arithmetic logic unit, the parallelism of the multiply-accumulate computing unit and the hierarchical configuration of the cache module, and generating a reconstructed computing unit;

[0059] Step S300, extracting computing complexity data, data dependency relationship data and task priority data of the to-be-allocated task through a task analysis module;

[0060] Step S400, collecting real-time load data of NPU, CPU and GPU through a hardware performance evaluation module, combining pre-stored hardware peak computing power data and power consumption model data, and generating heterogeneous hardware performance index data;

[0061] Step S500, receiving the computing complexity data, data dependency relationship data, task priority data and heterogeneous hardware performance index data through a dynamic task allocation module, generating task allocation decision data based on an energy efficiency ratio matching strategy;

[0062] Step S600, sending the task allocation decision data to a scheduling module, and scheduling the task to the target heterogeneous hardware for execution according to the data dependency relationship data, wherein when the target heterogeneous hardware is NPU, the reconstructed computing unit is called to execute the computing task, and task execution result data is generated;

[0063] Step S700, monitoring real-time load data of each heterogeneous hardware through the load balancing module, generating task migration instruction data when the load deviation exceeds the threshold, and adjusting the task allocation decision data based on the task migration instruction data.

[0064] In this embodiment, the hardware resource configuration parameter refers to the calculation unit structure parameter determined according to the task type, which can specifically be the bit width, parallelism and other configuration parameters generated after the instruction set parser analyzes the task characteristics, and is used to guide the reconstruction of the calculation unit. The subunit enable state of the arithmetic logic unit refers to the number of activated operation units according to the operation requirement, which can specifically use a gate circuit to control the power supply state of the operation unit to realize dynamic power consumption adjustment. The parallelism adjustment of the multiply-accumulate calculation unit refers to the number of simultaneously running calculation paths, which can specifically be realized by reconstructing the data path connection relationship through a cross switch. The hierarchical configuration of the cache module refers to the capacity allocation strategy of the multi-level cache, which can specifically use a cache partition controller to dynamically divide the storage space and optimize the data locality access. The heterogeneous hardware performance index data refers to the evaluation parameter integrating real-time load and theoretical performance, which can specifically be obtained by using a weighted algorithm to integrate the calculation resource occupation rate, memory bandwidth and energy efficiency ratio prediction value.

[0065] In this embodiment, when the video analysis server receives a mixed task containing target detection and behavior recognition, the configuration instruction generation module identifies that the task contains convolution operation and feature matching operation, and generates configuration parameters requiring 16-bit floating point operation and high parallelism. The control register transmits these parameters to the NPU configuration circuit, activates 8 groups of parallel multiply-accumulate units and configures the two-level cache to use 4-way set associative mapping. The task analysis module parses that the behavior recognition task depends on the output feature map of the target detection, and establishes the task execution sequence constraint. The performance evaluation module detects that the current load rate of the GPU is 85% and the load of the NPU is only 40%, and generates evaluation data containing the energy efficiency ratio index of each hardware. The dynamic allocation module allocates the NPU for execution for the real-time detection task with high priority, uses the reconstructed calculation unit to complete batch convolution calculation, and at the same time schedules the non-real-time behavior recognition task to the idle CPU for processing. When the GPU completes the pre-allocated task, the load balancing module migrates the queued task to the GPU for execution, maintaining the overall load balance of the system.

[0066] In this embodiment, the single NPU can adapt to the task requirements of different precision and parallelism through the dynamic reconstruction mechanism, for example, when processing low-precision inference tasks, some operation units are closed to reduce power consumption. In terms of task allocation, this scheme introduces real-time energy efficiency ratio evaluation, and automatically distributes part of the task to the GPU with better energy efficiency ratio when the NPU load is too high. In addition, the cache hierarchical configuration strategy is more suitable for changes in data access mode than the fixed cache structure, reducing the data transfer delay caused by low cache hit rate. In this way, the application realizes the dynamic matching of the calculation unit structure and the task demand, improves the NPU resource utilization rate in the mixed load scene, and improves the overall energy efficiency ratio of the system through the real-time energy efficiency ratio evaluation and task migration mechanism in the heterogeneous hardware cooperation efficiency aspect. The cache hierarchical configuration strategy reduces the data access delay, effectively supports high-throughput computing tasks, and the load balancing mechanism reduces the standard deviation of hardware resource utilization rate, avoiding performance bottlenecks caused by local overload.

[0067] In a feasible implementation, the dynamic adjustment of the subunit enablement state of the arithmetic logic unit includes: parsing operation precision requirement data in the hardware resource configuration parameter to generate subunit enablement quantity and data bit width configuration parameters; setting the data bit width of the arithmetic logic unit according to the data bit width configuration parameter, activating a corresponding number of operation submodules based on the subunit enablement quantity, and generating the reconstructed arithmetic logic unit as a component of the reconstructed calculation unit.

[0068] In this embodiment, the operation precision requirement data refers to the quantitative index of numerical calculation precision in the task execution process, which can be realized by parsing the floating point operation demand or fixed point operation demand in the calculation task type information. This data is used to determine the numerical range and precision level that the arithmetic logic unit needs to support, thereby guiding the dynamic configuration of the subunit enablement quantity and the data bit width. The subunit enablement quantity refers to the number of operation submodules that need to be activated in the arithmetic logic unit, which can be realized by sending a control signal from the configuration circuit to the power gate circuit of the arithmetic logic unit. This parameter is dynamically adjusted according to the operation precision requirement, for example, only a part of the subunit is enabled to reduce power consumption in the low-precision calculation scene. The data bit width configuration parameter refers to the bit width setting of the data processed by each operation submodule in the arithmetic logic unit, which can be realized by adjusting the data path width register inside the arithmetic logic unit. This parameter ensures that the data processing capacity of the operation unit matches the task precision requirement, for example, the data bit width is set to 32 bits in the 32-bit floating point operation scene.

[0069] In this embodiment, when receiving the calculation task type information, the configuration instruction generation module first parses the operation precision requirement data therein. For example, if the task needs to perform 16-bit fixed-point operation, the configuration parameters corresponding to the subunit activation number of 4 and the data bit width of 16 bits are generated. After the control register writes the above parameters, the configuration circuit reads the parameters and performs the following operations: by adjusting the bit width control register of the arithmetic logic unit, the data path width is set to 16 bits; at the same time, an activation signal is sent to the power gating circuit to start the power supply of the four operation subunits. In this way, the arithmetic logic unit only retains the operation resources matched with the current task precision, and the remaining subunits are in a power-off state. When the reconstructed arithmetic logic unit performs 16-bit fixed-point operation, it can meet the calculation precision requirement and avoid the power consumption waste of redundant hardware resources.

[0070] It can be understood that the traditional arithmetic logic unit adopts a fixed number of operation subunits and a fixed bit width design, which cannot dynamically adjust the hardware resources according to the task requirements. For example, when performing low-precision calculation, all the subunits activated in a fixed manner will cause unnecessary power consumption; and in the high-precision calculation scenario, the fixed bit width may not meet the precision requirement. The present scheme dynamically analyzes the operation precision requirement and real-time configures the subunit activation state and data bit width, so that the arithmetic logic unit can flexibly adapt to calculation tasks of different precision levels, solving the problem of low resource utilization and insufficient energy efficiency ratio caused by the single function of the arithmetic logic unit in the prior art. By dynamically matching the task precision requirement and the hardware resource configuration, the invalid power consumption of redundant hardware resources in low-precision tasks is avoided, and the calculation capability required by high-precision tasks is ensured, so as to significantly improve the utilization efficiency of hardware resources and the system energy efficiency ratio while ensuring the calculation precision.

[0071] In a feasible implementation manner, the dynamic adjustment of the multiply-accumulate calculation unit includes: parsing the parallelism requirement data in the hardware resource configuration parameter, generating a parallel computing path number configuration parameter of the multiply-accumulate calculation unit; activating a corresponding number of multiply-accumulate calculation subunits according to the parallel computing path number configuration parameter and recombining the data transmission path, and generating a reconstructed multiply-accumulate calculation unit as a component of the reconstructed calculation unit.

[0072] In this embodiment, the parallelism requirement data refers to the parallel computing capability index required for task execution, which can be generated by analyzing the task type and the computing scale, for example, a convolution operation task can generate high parallelism requirement data to guide the configuration of the multiply-accumulate calculation unit. The parallel computing path number configuration parameter refers to the parameter for controlling the number of parallel channels in the multiply-accumulate calculation unit, which can be realized by register configuration, for example, the value stored in the register is mapped to the control signal of the number of activated sub-units. The multiply-accumulate calculation sub-unit refers to the basic hardware module that realizes a single multiply-accumulate operation, which can be realized by an independent operation circuit, and each sub-unit has an independent data input and output interface. The data transmission path reorganization refers to adjusting the data distribution and collection path according to the number of activated sub-units, which can be realized by a crossbar circuit, for example, the input data is divided into fragments matching the number of activated sub-units and the corresponding transmission link is established.

[0073] In this embodiment, when receiving the hardware resource configuration parameter containing the parallelism requirement, the parameter is first parsed and the corresponding parallel computing path number configuration parameter is generated. For example, for a matrix multiplication task requiring high throughput, a configuration parameter with a parallelism requirement of 8 can be generated. Then, according to the parameter, a corresponding number of multiply-accumulate calculation sub-units are activated, for example, by enabling the power supply and clock signal of 8 sub-units. At the same time, the data transmission path is reorganized to support 8-way parallel input and output, for example, the input data bus is divided into 8 independent channels, each connected to a sub-unit. The reconstructed multiply-accumulate calculation unit formed in this way can perform computing tasks with a parallelism of 8 times that of a single sub-unit.

[0074] Compared with the prior art, the existing multiply-accumulate calculation unit adopts a fixed number of computing paths, for example, only supporting 4-way parallel computing. When processing low parallelism tasks, part of the sub-units are in idle state, causing resource waste; while facing high parallelism tasks, the computing efficiency cannot be improved due to hardware limitations. The present scheme dynamically adjusts the number of parallel paths, which can accurately match hardware resources according to task requirements, for example, activating only 2 sub-units to reduce power consumption when processing low complexity tasks, and activating all 8 sub-units to improve throughput when processing large-scale calculations, thereby solving the problem that the multiply-accumulate calculation unit cannot adapt to different task parallelism requirements, and realizing the synchronous optimization of hardware resource utilization and computing efficiency. For example, in an image recognition task, the single inference time can be shortened by dynamically improving the parallelism; in a lightweight data processing task, the dynamic power consumption can be reduced by reducing the parallelism.

[0075] In an implementable embodiment, the hierarchical configuration of the cache module comprises: parsing the cache correlation parameter in the hardware resource configuration parameter and the data locality characteristic data; setting the address mapping strategy of the secondary cache according to the cache correlation parameter, dynamically allocating the space ratio of the primary cache and the secondary cache based on the data locality characteristic data, and generating the hierarchical configuration cache module as a component of the reconstructed computing unit.

[0076] In the embodiment, the cache correlation parameter refers to a configurable parameter defining the address mapping mode of the secondary cache, which can be implemented by a set-associative mapping strategy, and the cache conflict probability is controlled by adjusting the correspondence between the cache line and the memory block. The data locality characteristic data refers to a quantitative index reflecting the distribution rule of the task access data space, which can be obtained by analyzing the temporal locality and spatial locality mode of the task input and output data set, and is used to guide the cache space allocation. The address mapping strategy refers to the conversion rule of the memory address to the cache location, which can be implemented by a hash function combined with a tag matching mechanism, and the number of cache sets and ways is changed by adjusting the correlation parameter. The space ratio allocation refers to the capacity division mode of the primary cache and the secondary cache, which can be dynamically set by a programmable register to set the cache partition boundary address, and the partition ratio is adjusted according to the data locality characteristic.

[0077] In the embodiment, when the hardware resource configuration parameter contains the cache correlation parameter, the configuration circuit writes the parameter into the secondary cache controller to control it to adopt a specific set-associative mapping mode. For example, when the correlation parameter is set to 4-way set-associative, each cache set contains 4 cache lines, and the memory address is mapped into the corresponding set by a hash function. At the same time, the data locality characteristic data extracted by the task analysis module is input into the cache controller. If it is detected that the task has high spatial locality, the capacity ratio of the primary cache is increased to improve the hot data hit rate; if it is detected that the temporal locality is dominant, the capacity of the secondary cache is increased to cache more historical data blocks. By dynamically adjusting the space ratio of the two-level cache, the cache structure is adapted to the data access mode of different tasks.

[0078] Compared with the prior art, the traditional cache system adopts a fixed capacity allocation and static address mapping strategy, and cannot adjust the cache structure according to the task characteristics. For example, in the existing scheme, the space ratio of the first-level cache to the second-level cache is fixed in the hardware design stage, resulting in a decrease in cache utilization when processing tasks with different locality characteristics. The scheme dynamically configures the address mapping strategy and the space ratio, so that the cache hierarchy can flexibly adapt to the task requirements. For example, when processing image recognition tasks, the system automatically adopts a high spatial locality optimization configuration, and when processing sequence modeling tasks, the system switches to a time locality optimization mode. Through the above technical scheme, the application can dynamically optimize the cache hierarchy according to the data access characteristics of the computing task, reduce the problem of hit rate reduction caused by unreasonable cache capacity allocation. By adjusting the address mapping strategy and the space ratio in real time, the data access delay is effectively reduced, and the adaptability of the computing unit to diversified tasks is improved, thereby solving the problem of low resource utilization caused by the fixed cache architecture.

[0079] In a feasible implementation, the extracting data dependency relationship data comprises: parsing a task input and output data set, identifying coincident fields of a previous task output data set and a current task input data set; and establishing an inter-task dependency graph according to the coincident fields to generate the data dependency relationship data.

[0080] In this embodiment, the task input and output data set refers to a set of input parameters required for task execution and a set of output results generated after execution, which can be obtained through a task description file or a metadata parsing module, and is used to determine the data transmission relationship between tasks. The coincident field refers to a field name or data structure identifier that exists in the intersection of the previous task output data set and the current task input data set, which can be identified by a field name matching algorithm or a data hash value comparison method, and is used to find the direct dependency relationship between tasks. The dependency graph refers to a topological structure graph reflecting the data transmission relationship between tasks, which can be constructed by a graph database or an adjacency matrix data structure, and is used to guide the arrangement of task scheduling order.

[0081] In this embodiment, before task execution, the input and output data set is parsed to extract the input parameter list and the output result list of each task. By comparing the output result field of the previous task with the input parameter field of the current task, the coincident part with the same field name or data identifier is identified. For example, in a video processing task, if the output of the previous task contains a "video frame sequence" field, and the input parameter of the current task contains a field with the same name, it is determined that there is a data dependency between the two. Subsequently, according to the coincident field information between all tasks, a topological sorting graph of task execution order is constructed to ensure that tasks with dependency relationship are scheduled and executed in the correct order.

[0082] In this embodiment, the data field coincidence relationship is automatically identified, the dynamic dependency graph is accurately established, the task blocking or repeated calculation caused by missing dependency relationship is effectively avoided, the automatic identification and structured expression of the data dependency relationship between tasks are realized, and the task scheduling module can optimize the execution order according to the accurate dependency graph. For example, in an image processing pipeline, by identifying the data transmission field between the feature extraction task and the classification task, the system can automatically schedule the classification task to the idle hardware after the feature extraction task is completed, reduce the task waiting time, and improve the collaborative efficiency of the heterogeneous hardware.

[0083] In a possible implementation, with reference to Figure 2 , step S400 includes steps S410-S430, wherein:

[0084] In step S410, real-time computing resource occupancy data and memory bandwidth utilization data are collected by a monitoring agent deployed on an NPU, a CPU, and a GPU.

[0085] In step S420, the real-time computing resource occupancy data and pre-stored hardware peak computing power data are input into a power consumption model to generate energy efficiency ratio prediction data.

[0086] In step S430, the computing resource occupancy data, the memory bandwidth utilization data, and the energy efficiency ratio prediction data are aggregated to generate heterogeneous hardware performance index data.

[0087] In this embodiment, the monitoring agent refers to a software or firmware module deployed on an NPU, a CPU, and a GPU, which can be implemented by a lightweight daemon process or a hardware performance counter, and is used to collect real-time computing resource occupancy data and memory bandwidth utilization data. The power consumption model refers to a mathematical model for calculating the energy efficiency ratio according to the hardware peak computing power and real-time load, which can be implemented by a polynomial regression model or a machine learning model, and outputs energy efficiency ratio prediction data by inputting the ratio of real-time computing resource occupancy and peak computing power. The heterogeneous hardware performance index data refers to the evaluation result of the comprehensive computing resource occupancy, memory bandwidth utilization, and energy efficiency ratio prediction, which can be implemented by a multi-dimensional vector or a weighted score table, and is used to quantify the current performance state of different hardware.

[0088] In this embodiment, the monitoring agent deployed on the NPU, CPU and GPU continuously collects real-time computing resource occupancy and memory bandwidth utilization data. The real-time computing resource occupancy data and the pre-stored hardware peak computing power data are input into the power consumption model. For example, when the peak computing power of the NPU is 100 TOPS and the current computing resource occupancy is 70%, the power consumption model will calculate the theoretical energy efficiency ratio at this time based on the historical power consumption curve. Subsequently, the computing resource occupancy, memory bandwidth utilization and energy efficiency ratio prediction data are aggregated into multi-dimensional performance indicators, for example, the occupancy, bandwidth utilization and energy efficiency ratio prediction values of the NPU are generated into a comprehensive score according to the preset weight. The performance indicator data provides a decision basis for the dynamic task allocation module, so that the task allocation strategy can be optimized based on the current actual performance state of the hardware rather than fixed parameters.

[0089] Compared with the prior art, the prior art usually only allocates tasks based on the nominal peak performance of the hardware, and cannot reflect the influence of real-time load on energy efficiency. For example, the traditional scheme may only allocate tasks according to the peak computing power of the NPU, but does not consider the case that the current load has caused the actual energy efficiency to decrease. While the present scheme can accurately evaluate the real performance of the heterogeneous hardware at a specific moment by collecting load data in real time and establishing a dynamic power consumption model, thereby avoiding allocating tasks to hardware in a high-load and low-energy efficiency state. Through the above technical scheme, the present application can dynamically evaluate the real-time performance state of the heterogeneous hardware, and optimize the task allocation decision in combination with the energy efficiency ratio prediction. For example, in the case where the NPU is in a high load but the GPU is idle, the system can migrate part of the tasks to the GPU for execution, thereby balancing the load and improving the overall energy efficiency. Thus, the problem of uneven utilization of hardware resources and low synergy efficiency caused by static performance evaluation in the prior art is solved.

[0090] In a feasible implementation manner, referring to Figure 3 , the step S500 includes steps S510-S530, wherein:

[0091] In step S510, a candidate hardware set satisfying the computing power requirement is selected according to the computing complexity data and the peak computing power data in the heterogeneous hardware performance indicator data;

[0092] In step S520, data transmission cost data between the candidate hardware is calculated based on the data dependency relationship data, and comprehensive evaluation score data is generated in combination with the energy efficiency ratio prediction data;

[0093] In step S530, the candidate hardware with the highest comprehensive evaluation score data is selected as the target heterogeneous hardware, and task allocation decision data is generated.

[0094] In this embodiment, the computing complexity data refers to a quantitative evaluation of the amount of computing resources required for the task, which can be realized by analyzing the size of the task instruction set or the number of floating point operations, and is used to determine whether the candidate hardware has sufficient computing power to support task execution. The heterogeneous hardware performance index data refers to a set of information reflecting the real-time performance state of different hardware, which can be realized by aggregating the computing resource occupancy rate, memory bandwidth utilization rate and energy efficiency ratio prediction data, and is used to evaluate the current processing capacity and energy efficiency of the hardware. The data transmission cost data refers to the additional overhead generated by the migration of the task between different hardware, which can be realized by calculating the ratio of the data size to the communication bandwidth between hardware, and is used to measure the impact of task migration on the overall performance of the system. The comprehensive evaluation score data refers to a quantitative evaluation result formed by combining the computing power matching degree, transmission cost and energy efficiency ratio, which can be realized by converting multi-dimensional indicators into a single score value through a weighted algorithm, and is used to objectively compare the comprehensive processing advantages of candidate hardware.

[0095] In this embodiment, the candidate hardware set is first screened, and the screening basis is whether the hardware peak computing power meets the task computing complexity requirement. For example, when the task needs to perform large-scale matrix operation, only NPU and GPU with high parallel computing capability are retained as candidates. Subsequently, the data transmission cost between candidate hardware is quantitatively calculated, for example, if the task needs to obtain input data from the CPU that has completed the pre-task, a higher data migration overhead will be generated when selecting NPU to execute the current task. Energy efficiency ratio prediction data is further introduced, for example, under the same computing power condition, the hardware with lower power consumption is preferentially selected. Finally, the comprehensive evaluation score is generated by fusing the above multi-dimensional indicators, for example, the computing power matching degree is assigned a weight of 40%, the transmission cost is assigned a weight of 30%, and the energy efficiency ratio is assigned a weight of 30%, and the score ranking of each candidate hardware is obtained by weighted summation.

[0096] In this embodiment, by constructing a multi-dimensional evaluation system, the real-time performance state of the hardware, the data migration overhead and the energy efficiency are considered simultaneously when the task is allocated. For example, when the NPU is in a high load state, the system can automatically migrate part of the task to the idle GPU for execution, while evaluating whether the communication delay generated by the data migration is within an acceptable range, avoiding the resource idling or performance bottleneck caused by uneven hardware load in the prior art, and solving the technical defect of single task allocation strategy when heterogeneous hardware cooperatively executes. The dynamic screening mechanism of the candidate hardware set ensures the computing power matching degree of the task and the hardware, the calculation of the data transmission cost avoids inefficient data migration operation, and the introduction of the energy efficiency ratio prediction data optimizes the overall energy efficiency of the system. The application of the multi-dimensional evaluation model makes the task allocation decision meet the real-time performance requirement and also consider the energy efficiency balance of the system, thereby improving the comprehensive utilization rate of the heterogeneous hardware resources.

[0097] In an implementable embodiment, the step of scheduling tasks to target heterogeneous hardware according to data dependency data comprises: topologically sorting task execution order according to data dependency data to generate task sequence data; sending tasks to target heterogeneous hardware according to task sequence data and task allocation decision data; and writing task execution result data to a shared storage area and triggering subsequent task allocation based on data dependency data when the task is completed.

[0098] In this embodiment, the data dependency data refers to a dependency graph between tasks established by analyzing task input and output data sets. Specifically, a field matching algorithm can be used to identify overlapping fields between the output of a previous task and the input of a current task, thereby establishing the execution order constraint relationship between tasks. Topological sorting refers to an algorithm for linearly sequencing task nodes in a directed acyclic graph. Specifically, a depth-first search algorithm can be used to ensure that the task execution order complies with the data dependency relationship. The shared storage area refers to a data storage area commonly accessed by heterogeneous hardware. Specifically, a distributed memory or cache can be used to store task execution result data and achieve cross-hardware data sharing.

[0099] In this embodiment, in the task scheduling process, the task set with data dependency is first topologically sorted to generate a loop-free task execution sequence. For example, when the output data of task A is used as the input data of task B, the topological sorting algorithm will ensure that task A is executed before task B. After the task allocation decision data determines the target execution hardware for each task, the scheduling module sends the tasks in sequence according to the sequence generated by the topological sorting. When the target heterogeneous hardware completes the calculation task, the execution result is written to the shared storage area, and the subsequent task that depends on the result automatically triggers the allocation process by monitoring the data state change of the shared storage area. This mechanism effectively avoids the data waiting or execution conflict problems caused by improper handling of task dependency relationships in traditional scheduling methods.

[0100] Compared with the prior art, the traditional task scheduling method usually uses a simple first-in-first-out queue to process tasks, without fully considering the data dependency between tasks, which can easily lead to idle computing resources or task blocking. For example, when multiple tasks have complex dependency relationships, the traditional method may cause calculation errors or efficiency decline due to incorrect sorting of task execution order. This scheme realizes accurate control of task execution order and real-time sharing of calculation results by introducing a topological sorting algorithm and a shared storage area mechanism.

[0101] By the technical solution, the application effectively solves the problem of task dependency relationship management difficulty in heterogeneous hardware cooperative execution, ensures that tasks with data dependencies are strictly executed in correct order, reduces cross-hardware data transmission delay through a shared storage area mechanism, and improves overall efficiency of task execution. The scheme is particularly suitable for computing task scenarios involving complex data dependency relationships, such as multi-stage computing processes in neural network training, and can achieve seamless connection and efficient use of computing resources.

[0102] In a feasible implementation, the step of invoking the reconstructed computing unit to execute a computing task includes: when a neural network training task is executed, a forward propagation calculation is performed by an NPU computing subunit in the reconstructed computing unit to generate intermediate result data; the intermediate result data is transmitted to a general computing module in the reconstructed computing unit through a high-speed data bus; and parameter update data is generated by performing gradient calculation instructions in the general computing module, and is fed back to the NPU computing subunit to complete a backward propagation calculation to generate task execution result data.

[0103] In this embodiment, the NPU computing subunit in the reconstructed computing unit refers to a special computing module dynamically configured according to task requirements, which can be implemented by a programmable logic array, and can adapt to different computing precision requirements by adjusting the number of operation units and data bit width. The subunit efficiently performs matrix multiplication and activation function calculation in the forward propagation stage. The high-speed data bus refers to a physical channel for realizing low-delay data transmission, which can be implemented by PCIe or a special interconnection protocol, and is used to reduce the transmission time of intermediate results between modules. The general computing module refers to a processor unit supporting multiple instruction sets, which can be implemented by a multi-core CPU or a programmable DSP, and is used to process non-neural network computing tasks such as scalar operations and conditional judgments in gradient calculation. The parameter update data refers to the weight adjustment amount generated by the gradient descent algorithm, which can be stored in floating-point or fixed-point format, and is used to optimize the neural network model parameters in the backward propagation process.

[0104] In this embodiment, in the neural network training task, the forward propagation stage is completed by the NPU computing subunit, which quickly processes input data and generates intermediate results through a parallel computing path. The intermediate results are transmitted to the general computing module through the high-speed data bus to avoid performance bottlenecks caused by data transfer. When the general computing module performs gradient calculation, it uses its flexible instruction set to handle complex branch logic and scalar operations, and generates parameter update data which is fed back to the NPU computing subunit. The backward propagation stage again uses the parallel computing capability of the NPU to update the network weights to form a complete training closed loop. Through the cooperative calculation between modules, the high-efficiency parallel processing capability of the NPU is retained, and the deficiency of the NPU in complex logic operation is made up.

[0105] In some specific implementations, forward propagation computation can employ a mixed-precision mode, such as using 16-bit floating-point operations for convolutional layers and 8-bit fixed-point operations for fully connected layers. Intermediate result data can be compressed before transmission to reduce bus bandwidth usage. The general-purpose computing module can be configured in multi-threaded mode to handle gradient calculation tasks for multiple neural networks simultaneously. Parameter update data can be encrypted using a differential transmission protocol to ensure data security during training.

[0106] In this embodiment, by reconstructing the collaborative mechanism of the internal modules of the computing unit, while maintaining the high computing power of the NPU, the non-parallel computing stages are optimized using general-purpose computing modules. Through high-speed buses and direct interaction between modules, the number of data transfers and memory access latency are reduced. This solves the problem of uneven hardware resource utilization during the forward and backward propagation stages of neural network training. By decomposing the computationally intensive forward propagation and the logically complex backward propagation into different modules for execution, the parallel computing advantages of the NPU are fully utilized, while the general-purpose computing modules handle irregular operations, significantly improving the overall execution efficiency of the training task. Simultaneously, the direct data connection architecture between modules avoids data transfer between traditional heterogeneous hardware, reducing transmission energy consumption and improving energy efficiency.

[0107] In one feasible implementation, the steps of monitoring real-time load data of each heterogeneous hardware through a load balancing module, generating task migration instruction data when the load deviation exceeds a threshold, and adjusting task allocation decision data based on the task migration instruction data include: periodically collecting resource utilization data of each heterogeneous hardware and calculating load variance data; when the load variance data exceeds a threshold, selecting low-priority task data and corresponding data dependency data from high-load hardware; generating task migration cost data based on the task data and data dependency data; generating task migration instruction data based on the task migration cost data; and adjusting task allocation decision data based on the task migration instruction data.

[0108] In this embodiment, load variance data refers to the dispersion of the utilization rate of each hardware resource calculated through statistical methods. Specifically, variance calculation formulas can be used to quantify the real-time resource utilization rate of CPU, GPU, and NPU to characterize the system load balancing status. Low-priority task data refers to task instances marked as interruptible or deferred according to preset task scheduling rules. Specifically, task priority tags can be combined with task execution progress thresholds for filtering to ensure that migration operations do not affect the execution continuity of critical tasks. Task migration cost data refers to the time overhead and resource consumption assessment value required to migrate a task from the current hardware to the target hardware. Specifically, it can be calculated by weighting data transmission rate, target hardware idle computing power, and task restart latency to assess the feasibility of the migration operation.

[0109] In this embodiment, the load balancing module obtains resource occupancy data from the performance monitoring unit of the heterogeneous hardware at fixed time intervals, including the utilization rate of the computing unit, the memory bandwidth occupancy rate and the cache hit rate. Through the variance calculation model, multiple sets of resource occupancy data are processed, and when the calculation result exceeds the preset threshold, it is determined that the system is in an unbalanced load state. At this time, the task migration engine selects low-priority tasks that meet the migration conditions from the high-load hardware according to the task priority queue, and extracts the data dependency graph. The migration cost evaluation unit combines the idle resource capacity of the target hardware and the data transmission path delay to generate a comprehensive cost index of the migration operation. If the migration cost is lower than the preset threshold, a task migration instruction containing the target hardware identifier and the task restart parameter is generated, the task allocation decision data table is dynamically updated, and the load is rebalanced.

[0110] Compared with the prior art, the traditional scheme usually triggers task migration with a static threshold, lacks quantitative evaluation of the overall load state of the system, and is prone to frequent migration or migration lag. While the present scheme dynamically captures the load distribution characteristics through the variance calculation model, and combines multi-dimensional evaluation of task priority and migration cost, it can accurately identify the migration time and migration object while ensuring the execution efficiency of critical tasks, avoiding resource waste caused by invalid migration. Through the above technical scheme, the present application effectively solves the problem of coexistence of resource idling and overloading caused by unbalanced load in the heterogeneous hardware cooperation scene. Through dynamic load monitoring and intelligent migration decision, the present application realizes global optimization of computing resources, reduces task execution delay, and reduces the additional overhead of the system caused by migration operation.

[0111] In the embodiments of the present application, the NPU-based storage-computing integrated server optimization method generates a reconstructed computing unit by dynamically adjusting the configuration of the arithmetic logic unit, the multiply-accumulate computing unit and the cache module, and combines the heterogeneous hardware performance evaluation and the task migration mechanism to realize dynamic optimization of computing resources and load balancing, which can improve the task execution efficiency and optimize the energy efficiency ratio of heterogeneous hardware cooperation.

[0112] It should be understood that various parts of the present application can be realized by hardware, software, firmware or their combination. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0113] The above is merely specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. An optimization method for an in-memory computing server based on an NPU, characterized in that, The method includes: The configuration instruction generation module receives computing task type information and generates hardware resource configuration parameters based on the computing task type information. The hardware resource configuration parameters are written into the control register. The hardware resource configuration parameters in the control register are read through the configuration circuit. The sub-unit enable status of the arithmetic logic unit, the parallelism of the multiplication and accumulation calculation unit, and the hierarchical configuration of the cache module are dynamically adjusted to generate the reconstructed calculation unit. The task analysis module extracts computational complexity data, data dependency data, and task priority data for the tasks to be assigned. The hardware performance evaluation module collects real-time load data of NPU, CPU, and GPU, and combines it with pre-stored hardware peak computing power data and power consumption model data to generate heterogeneous hardware performance index data. The dynamic task allocation module receives computational complexity data, data dependency data, task priority data, and heterogeneous hardware performance index data, and generates task allocation decision data based on the energy efficiency ratio matching strategy. The task allocation decision data is sent to the scheduling module, and the task is scheduled to be executed on the target heterogeneous hardware according to the data dependency relationship data. When the target heterogeneous hardware is an NPU, the reconstructed computing unit is invoked to execute the computing task and generate task execution result data. The load balancing module monitors the real-time load data of each heterogeneous hardware. When the load deviation exceeds the threshold, task migration instruction data is generated, and task allocation decision data is adjusted based on the task migration instruction data.

2. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The activation status of the sub-units of the dynamically adjusted arithmetic logic unit includes: Parse the computational precision requirements data in the hardware resource configuration parameters to generate the sub-unit activation quantity and data bit width configuration parameters; The data bit width of the arithmetic logic unit is set according to the data bit width configuration parameters. The corresponding number of operation sub-units are activated based on the number of sub-units enabled, and the reconstructed arithmetic logic unit is generated as a component of the reconstructed computing unit.

3. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The dynamically adjusted multiplication accumulation calculation unit includes: Parse the parallelism requirement data in the hardware resource configuration parameters to generate the configuration parameters for the number of parallel computing paths of the multiplication and accumulation calculation unit; Activate the corresponding number of multiplication-accumulation sub-units according to the configuration parameters of the number of parallel computing paths, reorganize the data transmission paths, and generate reconstructed multiplication-accumulation units as components of the reconstructed computing units.

4. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The hierarchical configuration of the caching module includes: Analyze cache correlation parameters and data locality characteristic data in hardware resource configuration parameters; The address mapping strategy of the second-level cache is set according to the cache correlation parameter. The space ratio of the first-level cache and the second-level cache is dynamically allocated based on the data locality feature data, and a hierarchical cache module is generated as a component of the reconstructed computing unit.

5. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, Extracting data dependency data includes: Parse the task input and output datasets and identify overlapping fields between the previous task output dataset and the current task input dataset; Based on overlapping fields, a dependency graph between tasks is established, generating data dependency data.

6. The optimization method for an in-memory computing server based on NPU as described in claim 1, characterized in that, The step of collecting real-time load data of NPU, CPU, and GPU through the hardware performance evaluation module, and combining it with pre-stored hardware peak computing power data and power consumption model data to generate heterogeneous hardware performance index data includes: Real-time computing resource utilization and memory bandwidth utilization data are collected by monitoring agents deployed on NPU, CPU, and GPU. The real-time computing resource utilization data and the pre-stored hardware peak computing power data are input into the power consumption model to generate energy efficiency ratio prediction data. Aggregate computing resource utilization data, memory bandwidth utilization data, and energy efficiency ratio prediction data to generate heterogeneous hardware performance index data.

7. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The step of receiving computational complexity data, data dependency data, task priority data, and heterogeneous hardware performance index data through the dynamic task allocation module, and generating task allocation decision data based on the energy efficiency ratio matching strategy, includes: Based on the peak computing power data in the computational complexity data and heterogeneous hardware performance index data, a set of candidate hardware that meets the computing power requirements is selected. Based on the data dependency relationship data, the data transmission cost data between candidate hardware is calculated, and combined with the energy efficiency ratio prediction data, a comprehensive evaluation score data is generated. The candidate hardware with the highest comprehensive evaluation score is selected as the target heterogeneous hardware, and task allocation decision data is generated.

8. The optimization method for an in-memory computing server based on NPU as described in claim 1, characterized in that, The step of scheduling data tasks to target heterogeneous hardware for execution based on data dependencies includes: Based on data dependency data, the task execution order is topologically sorted to generate task sequence data; The task is sent to the target heterogeneous hardware according to the task sequence data and task allocation decision data; When a task is completed, the task execution result data is written to the shared storage area, and subsequent task allocation is triggered based on the data dependency relationship data.

9. The optimization method for an in-memory computing server based on NPU as described in claim 1, characterized in that, The step of calling the reconstructed computing unit to execute the computing task includes: When performing neural network training tasks, forward propagation computation is performed through the NPU computing subunit in the reconstructed computing unit to generate intermediate result data. Intermediate result data is transmitted to the general computing module in the reconstructed computing unit via a high-speed data bus; In the general computing module, gradient calculation instructions are executed to generate parameter update data, which is then sent back to the NPU computing subunit to complete the backpropagation calculation and generate task execution result data.

10. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The steps of monitoring real-time load data of heterogeneous hardware through a load balancing module, generating task migration instruction data when the load deviation exceeds a threshold, and adjusting task allocation decision data based on the task migration instruction data include: Periodically collect resource utilization data of various heterogeneous hardware and calculate load variance data; When the load variance data exceeds the threshold, select the low-priority task data and the corresponding data dependency data from the high-load hardware. Task migration cost data is generated based on task data and data dependency data. Task migration instruction data is generated based on the task migration cost data. Task allocation decision data is adjusted based on the task migration instruction data.

Images