Storage and calculation integrated server optimization method based on NPU

Abstract

The invention discloses a storage and calculation integrated server optimization method based on an NPU, relates to the technical field of server architecture, and discloses the storage and calculation integrated server optimization method based on the NPU by dynamically adjusting the configuration of an arithmetic logic unit, a multiplication and accumulation calculation unit and a cache module to generate a reconstruction calculation unit. By combining heterogeneous hardware performance evaluation and a task migration mechanism, dynamic optimal configuration and load balancing of computing resources are realized, the task execution efficiency can be improved, and the heterogeneous hardware collaborative energy efficiency ratio can be optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of server architecture, and particularly relates to an optimization method of a storage-computing integrated server based on an NPU. BACKGROUND

[0002] With the rapid development of artificial intelligence, big data and other technologies, higher and higher requirements are put forward for the computing power and energy efficiency ratio of servers. The storage-computing integrated server, as a new server architecture, integrates storage and computing functions together, reduces the transmission of data between storage and computing units, and thus improves the computing efficiency and energy efficiency ratio to a certain extent. The NPU, as a processor specially designed for neural network computing, has significant advantages in the field of artificial intelligence computing. However, the existing hardware resource configuration strategy of the storage-computing integrated server based on the NPU is rigid, and cannot dynamically adjust the arithmetic logic unit, multiplication accumulation unit and cache module and other key components of the computing unit according to the task characteristics. Moreover, the task allocation algorithm of the existing technology is too simple, and fails to establish a decision model that comprehensively considers the computing complexity, data dependency relationship and hardware performance index, resulting in low synergy efficiency between heterogeneous hardware.

[0003] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide an optimization method of a storage-computing integrated server based on an NPU, aiming to improve the task execution efficiency and optimize the synergy energy efficiency ratio of heterogeneous hardware.

[0005] To achieve the above purpose, the present application provides an optimization method of a storage-computing integrated server based on an NPU, which comprises: The configuration instruction generation module receives the computing task type information, and generates hardware resource configuration parameters based on the computing task type information; The hardware resource configuration parameters are written into the control register, and the hardware resource configuration parameters in the control register are read by the configuration circuit to dynamically adjust the subunit enable state of the arithmetic logic unit, the parallelism of the multiplication accumulation computing unit and the hierarchical configuration of the cache module, and generate the reconstructed computing unit; The task analysis module extracts the computing complexity data, data dependency relationship data and task priority data of the task to be allocated; The hardware performance evaluation module collects real-time load data of the NPU, CPU and GPU, and generates heterogeneous hardware performance index data in combination with the pre-stored hardware peak computing power data and power consumption model data; The computing complexity data, the data dependency data, the task priority data and the heterogeneous hardware performance index data are received by a dynamic task allocation module, and a task allocation decision data is generated based on an energy efficiency ratio matching strategy; The task allocation decision data is sent to a scheduling module, and a target heterogeneous hardware is scheduled to execute a task according to the data dependency data, wherein the reconstructed computing unit is called to execute a computing task when the target heterogeneous hardware is an NPU, and a task execution result data is generated; Real-time load data of each heterogeneous hardware is monitored by a load balancing module, and a task migration instruction data is generated when a load deviation exceeds a threshold, and the task allocation decision data is adjusted based on the task migration instruction data.

[0006] In an embodiment, the dynamic adjustment of the subunit enablement state of the arithmetic logic unit includes: The operation precision requirement data in the hardware resource configuration parameter is parsed, and the subunit enablement number and the data bit width configuration parameter are generated; The data bit width of the arithmetic logic unit is set according to the data bit width configuration parameter, a corresponding number of operation subunits are activated based on the subunit enablement number, and a reconstructed arithmetic logic unit is generated as a component of the reconstructed computing unit.

[0007] In an embodiment, the dynamic adjustment of the multiply-accumulate computing unit includes: The parallelism requirement data in the hardware resource configuration parameter is parsed, and the parallel computing channel number configuration parameter of the multiply-accumulate computing unit is generated; A corresponding number of multiply-accumulate computing subunits are activated and the data transmission path is reorganized according to the parallel computing channel number configuration parameter, and a reconstructed multiply-accumulate computing unit is generated as a component of the reconstructed computing unit.

[0008] In an embodiment, the hierarchical configuration of the cache module includes: The cache correlation degree parameter and the data locality feature data in the hardware resource configuration parameter are parsed; The address mapping strategy of the secondary cache is set according to the cache correlation degree parameter, and the space ratio of the primary cache and the secondary cache is dynamically allocated based on the data locality feature data, and a hierarchically configured cache module is generated as a component of the reconstructed computing unit.

[0009] In an embodiment, the extraction of the data dependency data includes: The task input and output data set is parsed, and the coincident field of the pre-task output data set and the current task input data set is identified; The inter-task dependency graph is established according to the coincident field, and the data dependency data is generated.

[0010] In an embodiment, the step of collecting real-time load data of the NPU, the CPU and the GPU by the hardware performance evaluation module, combining the pre-stored hardware peak computing power data and the power consumption model data, and generating heterogeneous hardware performance index data includes: Real-time computing resource occupancy rate data and memory bandwidth utilization rate data are collected by monitoring agents deployed on the NPU, the CPU and the GPU; The real-time computing resource occupancy rate data and the pre-stored hardware peak computing power data are input into a power consumption model to generate energy efficiency ratio prediction data; The computing resource occupancy rate data, the memory bandwidth utilization rate data and the energy efficiency ratio prediction data are aggregated to generate heterogeneous hardware performance index data.

[0011] In an embodiment, the step of receiving the computing complexity data, the data dependency relationship data, the task priority data and the heterogeneous hardware performance index data by the dynamic task allocation module, and generating task allocation decision data based on an energy efficiency ratio matching strategy includes: According to the computing complexity data and the peak computing power data in the heterogeneous hardware performance index data, a candidate hardware set satisfying the computing power requirement is selected; Based on the data dependency relationship data, data transmission cost data between the candidate hardware is calculated, and comprehensive evaluation score data is generated in combination with the energy efficiency ratio prediction data; The candidate hardware with the highest comprehensive evaluation score data is selected as the target heterogeneous hardware, and task allocation decision data is generated.

[0012] In an embodiment, the step of scheduling tasks to the target heterogeneous hardware for execution according to the data dependency relationship data includes: The task execution order is topologically sorted according to the data dependency relationship data, and task sequence data is generated; The tasks are sent to the target heterogeneous hardware according to the task sequence data and the task allocation decision data; When the task is completed, the task execution result data is written into a shared storage area and the subsequent task allocation is triggered based on the data dependency relationship data.

[0013] In an embodiment, the step of calling the reconstructed computing unit to execute the computing task includes: When executing a neural network training task, forward propagation calculation is performed by the NPU computing subunit in the reconstructed computing unit to generate intermediate result data; The intermediate result data is transmitted to the general computing module in the reconstructed computing unit through a high-speed data bus; Parameter update data is generated by executing gradient calculation instructions in the general computing module, and is fed back to the NPU computing subunit to complete back propagation calculation, and task execution result data is generated.

[0014] In an embodiment, the step of monitoring real-time load data of each heterogeneous hardware by the load balancing module, generating task migration instruction data when the load deviation exceeds a threshold, and adjusting the task allocation decision data based on the task migration instruction data comprises: periodically collecting resource occupation rate data of each heterogeneous hardware, and calculating load variance data; when the load variance data exceeds a threshold, selecting low-priority task data and corresponding data dependency relationship data in the high-load hardware; generating task migration cost data based on the task data and the data dependency relationship data, generating task migration instruction data according to the task migration cost data, and adjusting the task allocation decision data based on the task migration instruction data.

[0015] The NPU-based storage-computing integrated server optimization method proposed in the present application generates a reconstructed computing unit by dynamically adjusting the configuration of the arithmetic logic unit, the multiply-accumulate computing unit, and the cache module, combines heterogeneous hardware performance evaluation and task migration mechanism, realizes dynamic optimization configuration and load balancing of computing resources, and can improve task execution efficiency and optimize the synergy energy efficiency ratio of heterogeneous hardware. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings from these drawings without creative labor.

[0018] Figure 1 a flowchart provided by an embodiment of the NPU-based storage-computing integrated server optimization method of the present application; Figure 2 a detailed flowchart of step S400 in the present application Figure 1 Figure 3 a detailed flowchart of step S500 in the present application Figure 1

[0019] The purpose implementation, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0020] ​​The technical solutions in the present application will be described clearly and completely in the present application combined with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] It should be understood that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0022] In the prior art, the storage-computing integrated server reduces data transmission delay by integrating storage and computing units, but the NPU computing unit function is fixed and cannot adapt to diversified task requirements. When facing the mixed scene of neural network training, inference and general computing, the existing system adopts the NPU and CPU / GPU collaborative mode with fixed architecture, resulting in insufficient utilization of hardware resources. For example, in a video analysis server, when the NPU performs a target detection task, it cannot dynamically adjust the computing unit configuration and needs to wait for the CPU to complete the preprocessing data, causing task queue blocking. At the same time, there is a lack of real-time load sensing mechanism among heterogeneous hardware, and high-priority tasks may be assigned to overloaded GPUs for execution, causing the overall energy efficiency ratio to decline.

[0023] To solve the above problems, the research and development team observed the core contradiction between the fixed function of the computing unit and the insufficient cooperation of the heterogeneous hardware in the traditional server architecture. By analyzing the typical data center task log, it was found that the neural network training task had the characteristics of alternating high computing power demand and low load period intermittently, and the existing hardware could not dynamically adjust the computing resources. Further research found that the task allocation strategy did not consider the real-time energy efficiency ratio of hardware, resulting in frequent inefficient task migration. Based on this, it is proposed to combine the computing unit reconstruction capability with dynamic task allocation, and to realize system-level optimization through dynamic configuration of hardware resources and load sensing scheduling.

[0024] Based on this, the embodiment of the present application provides an NPU-based storage-computing integrated server optimization method, referring to Figure 1 The NPU-based storage-computing integrated server optimization method includes steps S100-S700, wherein: In step S100, a computing task type information is received by the configuration instruction generation module, and a hardware resource configuration parameter is generated based on the computing task type information; In step S200, the hardware resource configuration parameter is written into a control register, the hardware resource configuration parameter in the control register is read by a configuration circuit, a subunit enable state of an arithmetic logic unit, a parallelism of a multiply-accumulate calculation unit, and a hierarchical configuration of a cache module are dynamically adjusted, and a reconstructed computing unit is generated. In step S300, computing complexity data, data dependency relationship data, and task priority data of a to-be-assigned task are extracted by a task analysis module. In step S400, real-time load data of an NPU, a CPU, and a GPU is collected by a hardware performance evaluation module, and combined with pre-stored hardware peak computing power data and power consumption model data, heterogeneous hardware performance index data is generated. In step S500, the computing complexity data, the data dependency relationship data, the task priority data, and the heterogeneous hardware performance index data are received by a dynamic task allocation module, and task allocation decision data is generated based on an energy efficiency ratio matching strategy. In step S600, the task allocation decision data is sent to a scheduling module, and a task is scheduled to a target heterogeneous hardware for execution according to the data dependency relationship data, wherein the reconstructed computing unit is called to execute a computing task when the target heterogeneous hardware is an NPU, and task execution result data is generated. In step S700, real-time load data of each heterogeneous hardware is monitored by a load balancing module, and task migration instruction data is generated when a load deviation exceeds a threshold value, and the task allocation decision data is adjusted based on the task migration instruction data.

[0025] In this embodiment, the hardware resource configuration parameter refers to a computing unit structure parameter determined according to a task type, which can specifically be a bit width, a parallelism, and the like generated by an instruction set parser after analyzing task characteristics, and is used to guide the reconstruction of the computing unit. The subunit enable state of the arithmetic logic unit refers to the number of activated operation units according to operation requirements, which can specifically be controlled by a gate circuit to control the power supply state of the operation unit, to realize dynamic power consumption adjustment. The parallelism adjustment of the multiply-accumulate calculation unit refers to the number of simultaneously running calculation paths, which can specifically be realized by reconstructing the connection relationship of data paths through a cross switch. The hierarchical configuration of the cache module refers to a capacity allocation strategy of multiple levels of cache, which can specifically be dynamically divided by a cache partition controller to optimize data locality access. The heterogeneous hardware performance index data refers to an evaluation parameter that comprehensively evaluates real-time load and theoretical performance, which can specifically be obtained by a weighting algorithm by integrating a computing resource occupancy rate, a memory bandwidth, and an energy efficiency ratio prediction value.

[0026] In this embodiment, when the video analysis server receives a mixed task containing target detection and behavior recognition, the configuration instruction generation module identifies that the task contains convolution operation and feature matching operation, and generates configuration parameters requiring 16-bit floating point operation and high parallelism. The control register transmits these parameters to the NPU configuration circuit, activates 8 groups of parallel multiply-accumulate units and configures the two-level cache to use 4-way set-associative mapping. The task analysis module parses that the behavior recognition task depends on the output feature map of the target detection, and establishes the task execution sequence constraint. The performance evaluation module detects that the current load rate of the GPU is 85% and the load of the NPU is only 40%, and generates evaluation data containing the energy efficiency ratio indicators of each hardware. The dynamic allocation module allocates the NPU execution for the real-time detection task with high priority, uses the reconstructed calculation unit to complete batch convolution calculation, and at the same time schedules the non-real-time behavior recognition task to the idle CPU for processing. When the GPU completes the pre-allocated task, the load balancing module migrates the queued task to the GPU for execution, maintaining the overall load balance of the system.

[0027] In this embodiment, the dynamic reconstruction mechanism enables a single NPU to adapt to tasks with different precision and parallelism requirements, such as turning off part of the operation units to reduce power consumption when processing low-precision inference tasks. In terms of task allocation, this scheme introduces real-time energy efficiency ratio evaluation, automatically diverting part of the task to the GPU for execution when the NPU load is too high, and the cache hierarchical configuration strategy is more suitable for changes in data access mode than the fixed cache structure, reducing data transfer delays caused by low cache hit rates. In this way, the present application realizes the dynamic matching of calculation unit structure and task demand, improves the NPU resource utilization rate in a mixed load scenario, and improves the overall energy efficiency ratio of the system through real-time energy efficiency ratio evaluation and task migration mechanism in terms of heterogeneous hardware collaboration efficiency. The cache hierarchical configuration strategy reduces the data access delay, effectively supports high-throughput computing tasks, and the load balancing mechanism reduces the standard deviation of hardware resource utilization rate, avoiding performance bottlenecks caused by local overload.

[0028] In a feasible implementation, the dynamic adjustment of the subunit enablement state of the arithmetic logic unit includes: parsing operation precision requirement data in the hardware resource configuration parameter to generate subunit enablement quantity and data bit width configuration parameters; setting the data bit width of the arithmetic logic unit according to the data bit width configuration parameter, activating a corresponding number of operation subunits based on the subunit enablement quantity, and generating a reconstructed arithmetic logic unit as a component of the reconstructed calculation unit.

[0029] In this embodiment, the operation precision requirement data refers to a quantitative index of numerical value calculation precision in the task execution process, which can be specifically realized by analyzing the floating point operation demand or fixed point operation demand in the task type information. The data is used to determine the numerical value range and precision level that the arithmetic logic unit needs to support, thereby guiding the dynamic configuration of the number of sub-units enabled and the data bit width. The number of sub-units enabled refers to the number of operation sub-modules that need to be activated in the arithmetic logic unit, which can be specifically realized by sending a control signal to the power gating circuit of the arithmetic logic unit through the configuration circuit. The parameter is dynamically adjusted according to the operation precision requirement, for example, only part of the sub-units are enabled in the low precision calculation scene to reduce power consumption. The data bit width configuration parameter refers to the bit width setting of the data processed by each operation sub-unit in the arithmetic logic unit, which can be specifically realized by adjusting the data path width register inside the arithmetic logic unit. The parameter ensures that the data processing capacity of the operation unit matches the task precision requirement, for example, the data bit width is set to 32 bits in the 32-bit floating point operation scene.

[0030] In this embodiment, when the calculation task type information is received, the configuration instruction generation module first analyzes the operation precision requirement data therein. For example, if the task needs to perform 16-bit fixed point number operation, the configuration parameters of 4 sub-units enabled and 16-bit data bit width are generated. After the control register writes the above parameters, the configuration circuit reads the parameters and performs the following operations: by adjusting the bit width control register of the arithmetic logic unit, the data path width is set to 16 bits; at the same time, an activation signal is sent to the power gating circuit to start the power supply of the four operation sub-units. In this way, the arithmetic logic unit only retains the operation resources matched with the current task precision, and the remaining sub-units are in the power-off state. The reconstructed arithmetic logic unit can meet the calculation precision requirement and avoid the power consumption waste of redundant hardware resources when performing 16-bit fixed point operation.

[0031] It can be understood that the traditional arithmetic logic unit adopts a fixed number of operation sub-units and a fixed bit width design, which cannot dynamically adjust the hardware resources according to the task demand. For example, when performing low precision calculation, the fixed enabled sub-units will cause unnecessary power consumption; and in the high precision calculation scene, the fixed bit width may not meet the precision requirement. The present scheme dynamically analyzes the operation precision requirement and real-time configures the sub-unit enabled state and data bit width, so that the arithmetic logic unit can flexibly adapt to calculation tasks of different precision levels, solving the problems of low resource utilization and insufficient energy efficiency ratio caused by the single function of the arithmetic logic unit in the prior art. By dynamically matching the task precision requirement and the hardware resource configuration, the invalid power consumption of redundant hardware resources in low precision tasks is avoided, and the calculation capacity required by high precision tasks is ensured, thereby significantly improving the utilization efficiency of hardware resources and the system energy efficiency ratio while ensuring the calculation precision.

[0032] In an implementable embodiment, the dynamic adjustment of the multiply-accumulate calculation unit comprises: parsing parallelism requirement data in the hardware resource configuration parameter to generate a parallel computing path number configuration parameter of the multiply-accumulate calculation unit; and activating a corresponding number of multiply-accumulate calculation subunits and recombining a data transmission path according to the parallel computing path number configuration parameter to generate a reconstructed multiply-accumulate calculation unit as a component of the reconstructed calculation unit.

[0033] In the embodiment, the parallelism requirement data refers to an index of parallel computing capability required by task execution, which can be generated by parsing a task type and a computing scale, for example, a convolution operation task can generate high parallelism requirement data for guiding the configuration of the multiply-accumulate calculation unit. The parallel computing path number configuration parameter refers to a parameter for controlling the number of parallel channels in the multiply-accumulate calculation unit, which can be realized by register configuration, for example, mapping a value stored in a register to a control signal for activating the number of subunits. The multiply-accumulate calculation subunit refers to a basic hardware module for realizing a single multiply-accumulate operation, which can be realized by an independent operation circuit, and each subunit has an independent data input and output interface. The data transmission path recombination refers to adjusting the data distribution and collection path according to the number of activated subunits, which can be realized by a crossbar circuit, for example, dividing input data into fragments matched with the number of activated subunits and establishing corresponding transmission links.

[0034] In the embodiment, when receiving the hardware resource configuration parameter containing the parallelism requirement, the parameter is first parsed to generate a corresponding parallel computing path number configuration parameter. For example, for a matrix multiplication task requiring high throughput, a configuration parameter with a parallelism requirement of 8 can be generated. Subsequently, a corresponding number of multiply-accumulate calculation subunits are activated according to the parameter, for example, by enabling signals to turn on the power supply and clock signals of 8 subunits. At the same time, the data transmission path is recombined into a structure supporting 8-way parallel input and output, for example, dividing the input data bus into 8 independent channels, each connected to a subunit. The reconstructed multiply-accumulate calculation unit formed thereby can execute the computing task with a parallelism of 8 times that of a single subunit.

[0035] Compared with the prior art, the existing multiplication accumulation computing unit adopts a fixed number of computing channels, for example, only supporting 4-way parallel computing. When processing a low parallel degree task, part of the sub-units are in an idle state, causing waste of resources; and when facing a high parallel degree task, the computing efficiency cannot be improved due to hardware limitations. The scheme can accurately match hardware resources according to task requirements by dynamically adjusting the number of parallel channels, for example, activating only 2 sub-units to reduce power consumption when processing a low complexity task, and activating all 8 sub-units to improve throughput when processing large-scale computing, thereby solving the problem that the multiplication accumulation computing unit cannot adapt to different task parallel degree requirements, and realizing the synchronous optimization of hardware resource utilization and computing efficiency. For example, in an image recognition task, the single inference time can be shortened by dynamically improving the parallel degree; in a lightweight data processing task, dynamic power consumption can be reduced by reducing the parallel degree.

[0036] In a feasible implementation, the hierarchical configuration of the cache module includes: parsing the cache correlation degree parameter in the hardware resource configuration parameter and the data locality feature data; setting the address mapping strategy of the second-level cache according to the cache correlation degree parameter, dynamically allocating the space ratio of the first-level cache and the second-level cache based on the data locality feature data, and generating the hierarchical configuration cache module as a component of the reconstructed computing unit.

[0037] In this embodiment, the cache correlation degree parameter refers to a configurable parameter defining the address mapping mode of the second-level cache, which can be implemented by using a set-associative mapping strategy to control the cache conflict probability by adjusting the correspondence between the cache line and the memory block. The data locality feature data refers to a quantitative index reflecting the spatial distribution law of task access data, which can be obtained by analyzing the temporal locality and spatial locality mode of the task input and output data set, and is used to guide the cache space allocation. The address mapping strategy refers to the conversion rule of the memory address to the cache location, which can be implemented by using a hash function combined with a tag matching mechanism, and the number of cache sets and ways can be changed by adjusting the correlation degree parameter. The space ratio allocation refers to the capacity division mode of the first-level cache and the second-level cache, which can be dynamically set by a programmable register to set the cache partition boundary address, and the partition ratio can be adjusted according to the data locality feature.

[0038] In this embodiment, when the hardware resource configuration parameter includes the cache association degree parameter, the configuration circuit writes the parameter into the secondary cache controller to control it to adopt a specific set-associative mapping mode. For example, when the association degree parameter is set to 4-way set-associative, each cache set includes 4 cache lines, and the memory address is mapped into the corresponding set through a hash function. At the same time, the data locality feature data extracted by the task analysis module is input into the cache controller. If it is detected that the task has high spatial locality, the capacity ratio of the primary cache is increased to improve the hot data hit rate; if it is detected that the time locality is dominant, the capacity of the secondary cache is increased to cache more historical data blocks. By dynamically adjusting the space ratio of the two-level cache, the cache structure can adapt to the data access mode of different tasks.

[0039] Compared with the prior art, the traditional cache system adopts a fixed capacity allocation and static address mapping strategy, and cannot adjust the cache structure according to the task characteristics. For example, in the existing scheme, the space ratio of the primary cache to the secondary cache is fixed in the hardware design stage, which leads to a decrease in cache utilization when processing tasks with different locality characteristics. The present scheme dynamically configures the address mapping strategy and the space ratio, so that the cache hierarchy structure can flexibly adapt to the task requirements. For example, when processing an image recognition task, it automatically adopts a high spatial locality optimization configuration, and when processing a sequence modeling task, it switches to a time locality optimization mode. Through the above technical scheme, the present application can dynamically optimize the cache hierarchy structure according to the data access characteristics of the computing task, reduce the problem of hit rate reduction caused by unreasonable cache capacity allocation. By adjusting the address mapping strategy and the space ratio in real time, the data access delay is effectively reduced, and the adaptability of the computing unit to diversified tasks is improved, thereby solving the problem of low resource utilization caused by the fixed cache architecture.

[0040] In a feasible implementation, the extracting data dependency relationship data includes: parsing a task input and output data set, identifying an overlapping field of a previous task output data set and a current task input data set; and establishing an inter-task dependency relationship graph based on the overlapping field to generate the data dependency relationship data.

[0041] In this embodiment, the task input and output data set refers to a set of input parameters required for task execution and a set of output results generated after execution, which can be obtained through a task description file or a metadata analysis module, and is used to determine the data transmission relationship between tasks. The overlapping field refers to a field name or data structure identifier that exists in the intersection of the previous task output data set and the current task input data set, which can be identified by a field name matching algorithm or a data hash value comparison method, and is used to find the direct dependency relationship between tasks. The dependency relationship graph refers to a topological structure graph reflecting the data transmission relationship between tasks, which can be constructed by a graph database or an adjacency matrix data structure, and is used to guide the arrangement of task scheduling order.

[0042] In this embodiment, before task execution, the input and output data set is parsed to extract the input parameter list and output result list of each task. By comparing the output result field of the preceding task with the input parameter field of the current task, the overlapping part with the same field name or data identifier is identified. For example, in a video processing task, if the output of the preceding task contains the "video frame sequence" field, and the input parameter of the current task contains the same field, it is determined that there is a data dependency between the two. Subsequently, according to the overlapping field information between all tasks, a topological sorting graph of task execution order is constructed to ensure that tasks with dependency relationship are scheduled and executed in the correct order.

[0043] In this embodiment, by automatically identifying the data field overlap relationship, the dynamic dependency graph is accurately established, which effectively avoids task blocking or repeated calculation caused by missing dependency relationship, realizes the automatic identification and structured expression of the data dependency relationship between tasks, and enables the task scheduling module to optimize the execution order according to the accurate dependency graph. For example, in an image processing pipeline, by identifying the data transmission field between the feature extraction task and the classification task, the system can automatically schedule the classification task to the idle hardware after the feature extraction task is completed, reducing the task waiting time and improving the collaborative efficiency of heterogeneous hardware.

[0044] In a feasible implementation manner, referring to Figure 2 , step S400 includes steps S410-S430, wherein: In step S410, real-time computing resource occupancy data and memory bandwidth utilization data are collected by a monitoring agent deployed on an NPU, a CPU, and a GPU; In step S420, the real-time computing resource occupancy data and the pre-stored hardware peak computing power data are input into a power consumption model to generate energy efficiency ratio prediction data; In step S430, the computing resource occupancy data, the memory bandwidth utilization data, and the energy efficiency ratio prediction data are aggregated to generate heterogeneous hardware performance index data.

[0045] In this embodiment, the monitoring agent refers to a software or firmware module deployed on an NPU, a CPU, and a GPU, which can be implemented by a lightweight daemon process or a hardware performance counter, and is used to collect real-time computing resource occupancy and memory bandwidth utilization data. The power consumption model refers to a mathematical model for calculating the energy efficiency ratio according to the hardware peak computing power and real-time load, which can be implemented by a polynomial regression model or a machine learning model, and outputs energy efficiency ratio prediction data by inputting the ratio of real-time computing resource occupancy and peak computing power. The heterogeneous hardware performance index data refers to the evaluation result of the comprehensive computing resource occupancy, memory bandwidth utilization, and energy efficiency ratio prediction, which can be implemented by a multi-dimensional vector or a weighted score table, and is used to quantify the current performance state of different hardware.

[0046] In this embodiment, the monitoring agent deployed on the NPU, CPU, and GPU continuously collects real-time computing resource occupancy and memory bandwidth utilization data. The real-time computing resource occupancy data and the pre-stored hardware peak computing power data are input into the power consumption model. For example, when the peak computing power of the NPU is 100 TOPS and the current computing resource occupancy is 70%, the power consumption model will calculate the theoretical energy efficiency ratio at this time based on the historical power consumption curve. Subsequently, the computing resource occupancy, memory bandwidth utilization, and energy efficiency ratio prediction data are aggregated into multi-dimensional performance indicators. For example, the occupancy, bandwidth utilization, and energy efficiency ratio prediction values of the NPU are combined according to the preset weights to generate a comprehensive score. The performance indicator data provides a decision basis for the dynamic task allocation module, enabling the task allocation strategy to be optimized based on the current actual performance state of the hardware rather than fixed parameters.

[0047] Compared with the prior art, the prior art usually only allocates tasks based on the nominal peak performance of the hardware, and cannot reflect the influence of real-time load on energy efficiency. For example, the traditional scheme may only allocate tasks according to the peak computing power of the NPU, but does not consider the case where the current load has caused the actual energy efficiency to decrease. The present scheme, by collecting load data in real time and establishing a dynamic power consumption model, can accurately evaluate the real performance of heterogeneous hardware at a specific moment, thereby avoiding allocating tasks to hardware that is in a high-load low-energy efficiency state. Through the above technical scheme, the present application can dynamically evaluate the real-time performance state of heterogeneous hardware and optimize the task allocation decision in combination with the energy efficiency ratio prediction. For example, in a scenario where the NPU is in a high-load state but the GPU is idle, the system can migrate part of the tasks to the GPU for execution, thereby balancing the load and improving the overall energy efficiency. Thus, the problem of uneven utilization of hardware resources and low synergy efficiency caused by static performance evaluation in the prior art is solved.

[0048] In a feasible implementation, with reference to Figure 3 , step S500 includes steps S510-S530, in which: Step S510, according to the computing complexity data and the peak computing power data in the heterogeneous hardware performance indicator data, filtering a candidate hardware set that meets the computing power requirement; Step S520, calculating the data transmission cost data between the candidate hardware based on the data dependency relationship data, and generating comprehensive evaluation score data in combination with the energy efficiency ratio prediction data; Step S530, selecting the candidate hardware with the highest comprehensive evaluation score data as the target heterogeneous hardware, and generating task allocation decision data.

[0049] In this embodiment, the computing complexity data refers to a quantitative evaluation of the amount of computing resources required for the task, which can be realized by analyzing the size of the task instruction set or the number of floating point operations, and is used to determine whether the candidate hardware has sufficient computing power to support task execution. The heterogeneous hardware performance index data refers to a set of information reflecting the real-time performance state of different hardware, which can be realized by aggregating the computing resource occupancy rate, memory bandwidth utilization rate and energy efficiency ratio prediction data, and is used to evaluate the current processing capacity and energy efficiency of the hardware. The data transmission cost data refers to the additional overhead generated by the migration of the task between different hardware, which can be realized by calculating the ratio of the data size to the communication bandwidth between hardware, and is used to measure the impact of task migration on the overall performance of the system. The comprehensive evaluation score data refers to a quantitative evaluation result formed by combining the computing power matching degree, transmission cost and energy efficiency ratio, which can be realized by converting multi-dimensional indicators into a single score value through a weighted algorithm, and is used to objectively compare the comprehensive processing advantages of candidate hardware.

[0050] In this embodiment, the candidate hardware set is first screened, and the screening basis is whether the hardware peak computing power meets the task computing complexity requirement. For example, when the task needs to perform large-scale matrix operation, only NPU and GPU with high parallel computing capability are retained as candidates. Subsequently, the data transmission cost between candidate hardware is quantitatively calculated, for example, if the task needs to obtain input data from the CPU that has completed the pre-task, a higher data migration overhead will be generated when selecting NPU to execute the current task. Energy efficiency ratio prediction data is further introduced, for example, under the same computing power condition, the hardware with lower power consumption is preferentially selected. Finally, the comprehensive evaluation score is generated by fusing the above multi-dimensional indicators, for example, the computing power matching degree is assigned a weight of 40%, the transmission cost is assigned a weight of 30%, and the energy efficiency ratio is assigned a weight of 30%, and the score ranking of each candidate hardware is obtained by weighted summation.

[0051] In this embodiment, by constructing a multi-dimensional evaluation system, the real-time performance state of the hardware, the data migration overhead and the energy efficiency are considered simultaneously when the task is allocated. For example, when the NPU is in a high load state, the system can automatically migrate part of the task to the idle GPU for execution, while evaluating whether the communication delay generated by the data migration is within an acceptable range, avoiding the resource idling or performance bottleneck caused by uneven hardware load in the prior art, and solving the technical defect of single task allocation strategy when heterogeneous hardware cooperatively executes. The dynamic screening mechanism of the candidate hardware set ensures the computing power matching degree of the task and the hardware, the calculation of the data transmission cost avoids inefficient data migration operation, and the introduction of the energy efficiency ratio prediction data optimizes the overall energy efficiency of the system. The application of the multi-dimensional evaluation model makes the task allocation decision meet the real-time performance requirement and also consider the energy efficiency balance of the system, thereby improving the comprehensive utilization rate of the heterogeneous hardware resources.

[0052] In an implementable embodiment, the step of scheduling tasks to target heterogeneous hardware according to data dependency data comprises: topologically sorting task execution order according to data dependency data to generate task sequence data; sending tasks to target heterogeneous hardware according to task sequence data and task allocation decision data; and writing task execution result data to a shared storage area and triggering subsequent task allocation based on data dependency data when the task is completed.

[0053] In this embodiment, the data dependency data refers to a dependency graph between tasks established by analyzing task input and output data sets. Specifically, a field matching algorithm can be used to identify overlapping fields between the output of a previous task and the input of a current task, thereby establishing the execution order constraint relationship between tasks. Topological sorting refers to an algorithm for linearly arranging task nodes in a directed acyclic graph. Specifically, a depth-first search algorithm can be used to ensure that the task execution order complies with the data dependency relationship. The shared storage area refers to a data storage area commonly accessed by heterogeneous hardware. Specifically, a distributed memory or cache can be used to store task execution result data and achieve cross-hardware data sharing.

[0054] In this embodiment, in the task scheduling process, the task set with data dependency is first topologically sorted to generate a loop-free task execution sequence. For example, when the output data of task A is used as the input data of task B, the topological sorting algorithm will ensure that task A is executed before task B. After the task allocation decision data determines the target execution hardware for each task, the scheduling module sends the tasks in sequence according to the sequence generated by the topological sorting. When the target heterogeneous hardware completes the calculation task, the execution result is written to the shared storage area, and the subsequent task that depends on the result automatically triggers the allocation process by monitoring the data state change of the shared storage area. This mechanism effectively avoids the data waiting or execution conflict problems caused by improper handling of task dependency relationships in traditional scheduling methods.

[0055] Compared with the prior art, the traditional task scheduling method usually uses a simple first-in-first-out queue to handle tasks, without fully considering the data dependency between tasks, which can easily lead to idle computing resources or task blocking. For example, when multiple tasks have complex dependency relationships, the traditional method may cause calculation errors or efficiency decline due to incorrect sorting of task execution order. This scheme realizes accurate control of task execution order and real-time sharing of calculation results by introducing a topological sorting algorithm and a shared storage area mechanism.

[0056] By the technical solution, the application effectively solves the problem of task dependency relationship management difficulty in heterogeneous hardware cooperative execution, ensures that tasks with data dependencies are strictly executed in correct order, reduces cross-hardware data transmission delay through a shared storage area mechanism, and improves overall efficiency of task execution. The scheme is particularly suitable for computing task scenarios involving complex data dependency relationships, such as multi-stage computing processes in neural network training, and can achieve seamless connection and efficient use of computing resources.

[0057] In a feasible implementation, the step of invoking the reconstructed computing unit to execute a computing task includes: when a neural network training task is executed, a forward propagation calculation is performed by an NPU computing subunit in the reconstructed computing unit to generate intermediate result data; the intermediate result data is transmitted to a general computing module in the reconstructed computing unit through a high-speed data bus; and parameter update data is generated by performing gradient calculation instructions in the general computing module, and is fed back to the NPU computing subunit to complete a backward propagation calculation to generate task execution result data.

[0058] In this embodiment, the NPU computing subunit in the reconstructed computing unit refers to a special computing module dynamically configured according to task requirements, which can be implemented by a programmable logic array, and can adapt to different computing precision requirements by adjusting the number of operation units and data bit width. The subunit efficiently performs matrix multiplication and activation function calculation in the forward propagation stage. The high-speed data bus refers to a physical channel for realizing low-delay data transmission, which can be implemented by PCIe or a special interconnection protocol, and is used to reduce the transmission time of intermediate results between modules. The general computing module refers to a processor unit supporting multiple instruction sets, which can be implemented by a multi-core CPU or a programmable DSP, and is used to process non-neural network computing tasks such as scalar operations and conditional judgments in gradient calculation. The parameter update data refers to a weight adjustment amount generated by a gradient descent algorithm, which can be stored in floating-point or fixed-point format, and is used to optimize neural network model parameters in the backward propagation process.

[0059] In this embodiment, in the neural network training task, the forward propagation stage is completed by the NPU computing subunit, which quickly processes input data and generates intermediate results through a parallel computing path. The intermediate results are transmitted to the general computing module through the high-speed data bus to avoid performance bottlenecks caused by data transfer. When the general computing module performs gradient calculation, it uses its flexible instruction set to handle complex branch logic and scalar operations, and generates parameter update data which is fed back to the NPU computing subunit. The backward propagation stage again uses the parallel computing capability of the NPU to update network weights to form a complete training closed loop. Through the cooperative calculation between modules, the high-efficiency parallel processing capability of the NPU is retained, and the deficiency of the NPU in complex logic operation is made up.

[0060] In some specific implementations, forward propagation computation can employ a mixed-precision mode, such as using 16-bit floating-point operations for convolutional layers and 8-bit fixed-point operations for fully connected layers. Intermediate result data can be compressed before transmission to reduce bus bandwidth usage. The general-purpose computing module can be configured in multi-threaded mode to handle gradient calculation tasks for multiple neural networks simultaneously. Parameter update data can be encrypted using a differential transmission protocol to ensure data security during training.

[0061] In this embodiment, by reconstructing the collaborative mechanism of the internal modules of the computing unit, while maintaining the high computing power of the NPU, the non-parallel computing stages are optimized using general-purpose computing modules. Through high-speed buses and direct interaction between modules, the number of data transfers and memory access latency are reduced. This solves the problem of uneven hardware resource utilization during the forward and backward propagation stages of neural network training. By decomposing the computationally intensive forward propagation and the logically complex backward propagation into different modules for execution, the parallel computing advantages of the NPU are fully utilized, while the general-purpose computing modules handle irregular operations, significantly improving the overall execution efficiency of the training task. Simultaneously, the direct data connection architecture between modules avoids data transfer between traditional heterogeneous hardware, reducing transmission energy consumption and improving energy efficiency.

[0062] In one feasible implementation, the steps of monitoring real-time load data of each heterogeneous hardware through a load balancing module, generating task migration instruction data when the load deviation exceeds a threshold, and adjusting task allocation decision data based on the task migration instruction data include: periodically collecting resource utilization data of each heterogeneous hardware and calculating load variance data; when the load variance data exceeds a threshold, selecting low-priority task data and corresponding data dependency data from high-load hardware; generating task migration cost data based on the task data and data dependency data; generating task migration instruction data based on the task migration cost data; and adjusting task allocation decision data based on the task migration instruction data.

[0063] In this embodiment, load variance data refers to the dispersion of the utilization rate of each hardware resource calculated through statistical methods. Specifically, variance calculation formulas can be used to quantify the real-time resource utilization rate of CPU, GPU, and NPU to characterize the system load balancing status. Low-priority task data refers to task instances marked as interruptible or deferred according to preset task scheduling rules. Specifically, task priority tags can be combined with task execution progress thresholds for filtering to ensure that migration operations do not affect the execution continuity of critical tasks. Task migration cost data refers to the time overhead and resource consumption assessment value required to migrate a task from the current hardware to the target hardware. Specifically, it can be calculated by weighting data transmission rate, target hardware idle computing power, and task restart latency to assess the feasibility of the migration operation.

[0064] In this embodiment, the load balancing module obtains resource occupancy data from the performance monitoring unit of the heterogeneous hardware at fixed time intervals, including the utilization rate of the computing unit, the memory bandwidth occupancy rate and the cache hit rate. Through the variance calculation model, multiple sets of resource occupancy data are processed, and when the calculation result exceeds the preset threshold, it is determined that the system is in an unbalanced load state. At this time, the task migration engine selects low-priority tasks that meet the migration conditions from the high-load hardware according to the task priority queue, and extracts the data dependency graph. The migration cost evaluation unit combines the idle resource capacity of the target hardware and the data transmission path delay to generate a comprehensive cost index of the migration operation. If the migration cost is lower than the preset threshold, a task migration instruction containing the target hardware identifier and the task restart parameter is generated, the task allocation decision data table is dynamically updated, and the load is rebalanced.

[0065] Compared with the prior art, the traditional scheme usually triggers task migration with a static threshold, lacks quantitative evaluation of the overall load state of the system, and is prone to frequent migration or migration lag. While the present scheme dynamically captures the load distribution characteristics through the variance calculation model, and combines multi-dimensional evaluation of task priority and migration cost, it can accurately identify the migration time and migration object while ensuring the execution efficiency of critical tasks, avoiding resource waste caused by invalid migration. Through the above technical scheme, the present application effectively solves the problem of coexistence of resource idling and overloading caused by unbalanced load in the heterogeneous hardware cooperation scene. Through dynamic load monitoring and intelligent migration decision, the present application realizes global optimization of computing resources, reduces task execution delay, and reduces the additional overhead of the system caused by migration operation.

[0066] In the embodiments of the present application, the NPU-based storage-computing integrated server optimization method generates a reconstructed computing unit by dynamically adjusting the configuration of the arithmetic logic unit, the multiply-accumulate computing unit and the cache module, and combines the heterogeneous hardware performance evaluation and the task migration mechanism to realize dynamic optimization of computing resources and load balancing, which can improve the task execution efficiency and optimize the energy efficiency ratio of heterogeneous hardware cooperation.

[0067] It should be understood that various parts of the present application can be realized by hardware, software, firmware or their combination. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0068] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any equivalent structural transformation made by using the contents of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. An optimization method for an in-memory computing server based on an NPU, characterized in that, The method includes: The configuration instruction generation module receives computing task type information and generates hardware resource configuration parameters based on the computing task type information. The hardware resource configuration parameters are written into the control register. The hardware resource configuration parameters in the control register are read through the configuration circuit. The sub-unit enable status of the arithmetic logic unit, the parallelism of the multiplication and accumulation calculation unit, and the hierarchical configuration of the cache module are dynamically adjusted to generate the reconstructed calculation unit. The task analysis module extracts computational complexity data, data dependency data, and task priority data for the tasks to be assigned. The hardware performance evaluation module collects real-time load data of NPU, CPU, and GPU, and combines it with pre-stored hardware peak computing power data and power consumption model data to generate heterogeneous hardware performance index data. The dynamic task allocation module receives computational complexity data, data dependency data, task priority data, and heterogeneous hardware performance index data, and generates task allocation decision data based on the energy efficiency ratio matching strategy. The task allocation decision data is sent to the scheduling module, and the task is scheduled to be executed on the target heterogeneous hardware according to the data dependency relationship data. When the target heterogeneous hardware is an NPU, the reconstructed computing unit is invoked to execute the computing task and generate task execution result data. The load balancing module monitors the real-time load data of each heterogeneous hardware. When the load deviation exceeds the threshold, task migration instruction data is generated, and task allocation decision data is adjusted based on the task migration instruction data.

2. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The activation status of the sub-units of the dynamically adjusted arithmetic logic unit includes: Parse the computational precision requirements data in the hardware resource configuration parameters to generate the sub-unit activation quantity and data bit width configuration parameters; The data bit width of the arithmetic logic unit is set according to the data bit width configuration parameters. The corresponding number of operation sub-units are activated based on the number of sub-units enabled, and the reconstructed arithmetic logic unit is generated as a component of the reconstructed computing unit.

3. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The dynamically adjusted multiplication accumulation calculation unit includes: Parse the parallelism requirement data in the hardware resource configuration parameters to generate the configuration parameters for the number of parallel computing paths of the multiplication and accumulation calculation unit; Activate the corresponding number of multiplication-accumulation sub-units according to the configuration parameters of the number of parallel computing paths, reorganize the data transmission paths, and generate reconstructed multiplication-accumulation units as components of the reconstructed computing units.

4. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The hierarchical configuration of the caching module includes: Analyze cache correlation parameters and data locality characteristic data in hardware resource configuration parameters; The address mapping strategy of the second-level cache is set according to the cache correlation parameter. The space ratio of the first-level cache and the second-level cache is dynamically allocated based on the data locality feature data, and a hierarchical cache module is generated as a component of the reconstructed computing unit.

5. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, Extracting data dependency data includes: Parse the task input and output datasets and identify overlapping fields between the previous task output dataset and the current task input dataset; Based on overlapping fields, a dependency graph between tasks is established, generating data dependency data.

6. The optimization method for an in-memory computing server based on NPU as described in claim 1, characterized in that, The step of collecting real-time load data of NPU, CPU, and GPU through the hardware performance evaluation module, and combining it with pre-stored hardware peak computing power data and power consumption model data to generate heterogeneous hardware performance index data includes: Real-time computing resource utilization and memory bandwidth utilization data are collected by monitoring agents deployed on NPU, CPU, and GPU. The real-time computing resource utilization data and the pre-stored hardware peak computing power data are input into the power consumption model to generate energy efficiency ratio prediction data. Aggregate computing resource utilization data, memory bandwidth utilization data, and energy efficiency ratio prediction data to generate heterogeneous hardware performance index data.

7. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The step of receiving computational complexity data, data dependency data, task priority data, and heterogeneous hardware performance index data through the dynamic task allocation module, and generating task allocation decision data based on the energy efficiency ratio matching strategy, includes: Based on the peak computing power data in the computational complexity data and heterogeneous hardware performance index data, a set of candidate hardware that meets the computing power requirements is selected. Based on the data dependency relationship data, the data transmission cost data between candidate hardware is calculated, and combined with the energy efficiency ratio prediction data, a comprehensive evaluation score data is generated. The candidate hardware with the highest comprehensive evaluation score is selected as the target heterogeneous hardware, and task allocation decision data is generated.

8. The optimization method for an in-memory computing server based on NPU as described in claim 1, characterized in that, The step of scheduling data tasks to target heterogeneous hardware for execution based on data dependencies includes: Based on data dependency data, the task execution order is topologically sorted to generate task sequence data; The task is sent to the target heterogeneous hardware according to the task sequence data and task allocation decision data; When a task is completed, the task execution result data is written to the shared storage area, and subsequent task allocation is triggered based on the data dependency relationship data.

9. The optimization method for an in-memory computing server based on NPU as described in claim 1, characterized in that, The step of calling the reconstructed computing unit to execute the computing task includes: When performing neural network training tasks, forward propagation computation is performed through the NPU computing subunit in the reconstructed computing unit to generate intermediate result data. Intermediate result data is transmitted to the general computing module in the reconstructed computing unit via a high-speed data bus; In the general computing module, gradient calculation instructions are executed to generate parameter update data, which is then sent back to the NPU computing subunit to complete the backpropagation calculation and generate task execution result data.

10. The NPU-based in-memory computing server optimization method as described in claim 1, characterized in that, The steps of monitoring real-time load data of heterogeneous hardware through a load balancing module, generating task migration instruction data when the load deviation exceeds a threshold, and adjusting task allocation decision data based on the task migration instruction data include: Periodically collect resource utilization data of various heterogeneous hardware and calculate load variance data; When the load variance data exceeds the threshold, select the low-priority task data and the corresponding data dependency data from the high-load hardware. Task migration cost data is generated based on task data and data dependency data. Task migration instruction data is generated based on the task migration cost data. Task allocation decision data is adjusted based on the task migration instruction data.

Images