Energy efficiency control method and system for dormancy triggering of NPU computing unit

Abstract

The invention discloses an energy efficiency control method and system for dormancy triggering of an NPU computing unit, relates to the technical field of computers, and aims to effectively reduce memory access times and reduce power consumption by optimizing a cache structure of the NPU computing unit and introducing a multi-level cache mechanism, design an intelligent prefetching mechanism and predict and load data possibly needed subsequently in advance. A task scheduling algorithm is optimized; a strategy based on priority and load awareness is adopted; the load of the calculation unit is reasonably distributed; the response speed and the resource utilization rate of the system are improved; load balancing is achieved, the overall performance of the system is further improved, an intelligent sleep triggering mechanism is designed, when the load of the calculation unit is lower than a preset threshold value, a sleep mode is automatically triggered, the static power consumption of the system is reduced, and a complete energy efficiency control strategy is formed.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an energy efficiency control method and system for triggering sleep mode in an NPU computing unit. Background Technology

[0002] With the rapid development of big data technology, energy equipment health monitoring technology has gradually become a key means to ensure the stable operation of energy systems. In recent years, researchers have used big data analysis, machine learning, and other technologies to achieve real-time monitoring and fault prediction of energy equipment operating status. However, existing technologies still have many shortcomings in practical applications.

[0003] Existing energy equipment health monitoring technologies primarily rely on traditional data acquisition and analysis methods. These methods are inefficient when processing large-scale data and struggle to accurately reflect the health status of equipment in real time. Furthermore, current technologies have significant shortcomings in data fusion and multi-source data collaborative analysis, failing to fully leverage the advantages of big data. For example, traditional monitoring systems often only monitor single devices, lacking the comprehensive assessment capability of the entire energy system, making it difficult to detect potential faults in a timely manner under complex operating conditions. These problems limit the further development and application of energy equipment health monitoring technologies. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an energy efficiency control method for NPU computing unit sleep triggering to solve the problems of low data processing efficiency and inability to accurately reflect the health status of equipment in real time in existing energy equipment health monitoring methods, as well as how to achieve multi-source data fusion and collaborative analysis.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides an energy efficiency control method for triggering sleep mode in an NPU computing unit, characterized by the following steps:

[0008] Optimize the cache structure of the NPU computing unit by introducing a multi-level caching mechanism, including L1, L2 and L3 caches, to reduce the number of memory accesses;

[0009] Based on the optimized multi-level cache structure, an intelligent prefetching mechanism is designed to predict and preload data that may be needed later by analyzing task access patterns and historical data.

[0010] By combining the output of the prefetching mechanism, the task scheduling algorithm is optimized, and a priority- and load-aware scheduling strategy is adopted to reasonably allocate the load of computing units.

[0011] Based on the output of the task scheduling algorithm, the load balancing algorithm is improved, the load of the computing unit is monitored in real time, and the task allocation is dynamically adjusted to achieve load balancing.

[0012] Based on the output of the load balancing algorithm, an intelligent sleep triggering mechanism is designed to automatically trigger the sleep mode when the load of the computing unit is lower than a preset threshold.

[0013] By integrating the above-mentioned optimized multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism, a complete energy efficiency control strategy is formed.

[0014] As a preferred embodiment of the energy efficiency control method for NPU computing unit sleep triggering described in this invention, the step of optimizing the cache structure of the NPU computing unit includes:

[0015] The L1 cache is designed as a small-capacity, low-latency cache, located close to the computing unit, and used to store the most frequently accessed data;

[0016] The L2 cache is designed as a cache with a large capacity and moderate latency, used to store data accessed in a moderately frequent manner.

[0017] The L3 cache is designed as a cache with the largest capacity and the highest latency, used to store data that is not accessed so frequently.

[0018] By analyzing task access patterns, the allocation strategy of each level of cache can be dynamically adjusted to further optimize cache utilization efficiency.

[0019] As a preferred embodiment of the energy efficiency control method for triggering NPU computing unit sleep mode according to the present invention, the step of designing the intelligent prefetch mechanism includes:

[0020] Collect current task access patterns and historical data, and analyze the locality and frequency of data access;

[0021] Based on the analysis results, predict the data blocks that may need to be accessed in the future and mark these data blocks as prefetch targets;

[0022] In a multi-level caching structure, the prefetch target data is loaded into the L2 cache first. If the L2 cache space is insufficient, the data is loaded into the L1 or L3 cache according to the access priority.

[0023] Monitor data access in real time and dynamically adjust the prefetching strategy to adapt to changes in task access patterns.

[0024] As a preferred embodiment of the energy efficiency control method for triggering NPU computing unit sleep mode according to the present invention, the steps of optimizing the task scheduling algorithm include:

[0025] Based on task priority and computational complexity, tasks are divided into high-priority tasks and low-priority tasks.

[0026] For high-priority tasks, computing resources are allocated first, and the data required by the task is ensured to be loaded into the cache based on the output of the prefetch mechanism.

[0027] For low-priority tasks, their execution order is dynamically adjusted based on the current load of the computing unit in order to make full use of computing resources;

[0028] Regularly assess the execution status of tasks and the load on computing units, and dynamically adjust task scheduling strategies to achieve load balancing.

[0029] As a preferred embodiment of the energy efficiency control method for triggering NPU computing unit sleep mode according to the present invention, the steps of the improved load balancing algorithm include:

[0030] Real-time monitoring of the load of each NPU computing unit, including the execution status of the current task and the usage of the cache;

[0031] Based on the load, the task allocation is dynamically adjusted, and some tasks on the computing units with high load are migrated to the computing units with low load.

[0032] During task migration, data locality and the output of the prefetching mechanism are considered to ensure that the required data can be quickly obtained after the task migration.

[0033] Regularly evaluate the effectiveness of load balancing and further optimize the load balancing algorithm based on the evaluation results.

[0034] As a preferred embodiment of the energy efficiency control method for NPU computing unit sleep triggering described in this invention, the step of designing the intelligent sleep triggering mechanism includes:

[0035] Set a preset load threshold; when the load on the computing unit falls below this threshold, trigger a sleep mechanism.

[0036] Before triggering hibernation, check the execution status of the current task and the validity of the prefetched data to ensure that the task will not be interrupted due to hibernation;

[0037] Once the hibernation conditions are met, the computing unit is put into hibernation mode and the state information before hibernation is recorded for quick wake-up.

[0038] During hibernation, the system status is continuously monitored, and the computing unit is immediately woken up once a new task arrives or the load increases.

[0039] As a preferred embodiment of the energy efficiency control method for NPU computing unit sleep triggering described in this invention, the steps of integrating and optimizing the multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm, and sleep triggering mechanism include:

[0040] The multi-level caching structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism are integrated into a unified energy efficiency control framework;

[0041] Within the framework, the parameters and strategies of each mechanism are dynamically adjusted according to different application scenarios and task requirements;

[0042] Implement a central controller to coordinate the operation of various mechanisms and ensure they work together to achieve optimal energy efficiency control.

[0043] Regularly assess the energy efficiency of the entire system and further optimize the integration strategy of each mechanism based on the assessment results.

[0044] The beneficial effects of this invention are as follows: By introducing a multi-level caching mechanism and optimizing the cache structure, the number of memory accesses is reduced, and power consumption is lowered. The intelligent prefetching mechanism can predict the data that may be needed later based on task access patterns and historical data and load it in advance, reducing the waiting time of the computing unit after waking up from sleep. The task scheduling algorithm is optimized by adopting a priority-based and load-aware strategy to reasonably allocate the load of the computing unit, improving the system's response speed and resource utilization. The load balancing algorithm is improved by monitoring the load of the computing unit in real time and dynamically adjusting the task allocation to achieve load balancing, further improving the overall performance of the system. An intelligent sleep triggering mechanism is designed to automatically trigger the sleep mode when the load of the computing unit is lower than a preset threshold, reducing the static power consumption of the system. By integrating the above mechanisms, a complete energy efficiency control strategy is formed, realizing the synergistic optimization of hardware and software, and significantly improving the energy efficiency of the NPU system. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of the energy efficiency control method for triggering NPU computing unit hibernation in Example 1. Detailed Implementation

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0048] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0050] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides an energy efficiency control method for triggering sleep mode in an NPU computing unit, characterized by comprising the following steps:

[0051] Optimize the cache structure of the NPU computing unit by introducing a multi-level caching mechanism, including L1, L2 and L3 caches, to reduce the number of memory accesses;

[0052] Based on the optimized multi-level cache structure, an intelligent prefetching mechanism is designed to predict and preload data that may be needed later by analyzing task access patterns and historical data.

[0053] By combining the output of the prefetching mechanism, the task scheduling algorithm is optimized, and a priority- and load-aware scheduling strategy is adopted to reasonably allocate the load of computing units.

[0054] Based on the output of the task scheduling algorithm, the load balancing algorithm is improved, the load of the computing unit is monitored in real time, and the task allocation is dynamically adjusted to achieve load balancing.

[0055] Based on the output of the load balancing algorithm, an intelligent sleep triggering mechanism is designed to automatically trigger the sleep mode when the load of the computing unit is lower than a preset threshold.

[0056] By integrating the above-mentioned optimized multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism, a complete energy efficiency control strategy is formed.

[0057] It should be noted that the cache structure within the NPU computing unit is optimized and modified to establish a three-level cache architecture of L1, L2, and L3. First, an L1 cache is established inside the computing unit chip to store frequently accessed, high-priority data; its capacity is relatively small but its access speed is extremely fast. Next, an L2 cache is deployed in the middle layer outside the chip, between the computing unit and main memory. Its capacity is larger than the L1 cache, primarily used to store the next most frequently accessed data, thus supplementing the L1 cache. Further out, at the higher levels of the entire computing system architecture, an L3 cache is configured with an even larger capacity. This further alleviates the pressure on main memory and provides temporary storage space for data sharing between different tasks, thereby optimizing overall memory access efficiency through a multi-level cache layout.

[0058] By constructing a three-level caching mechanism, the number of accesses to main memory can be effectively reduced. When executing tasks, the computing unit can prioritize retrieving data from the L1 cache, which is closer to the processing core and has a faster access speed. If the L1 cache is not hit, data is then requested from the L2 and L3 caches in sequence. This hierarchical filtering can greatly improve the speed and success rate of data retrieval, reduce the high latency and high power consumption problems caused by frequent access to main memory, and improve the overall operating efficiency and energy efficiency of the NPU computing unit, providing a more efficient data interaction foundation for subsequent task execution.

[0059] Based on the optimized multi-level caching structure described above, we conduct in-depth analysis of the current task's access patterns. This includes patterns such as the order, frequency, and interval of data requests during task execution, and incorporates data mining and modeling based on a large amount of historical data. We use machine learning algorithms, such as Markov chains, to predict the datasets the task might access next, or to anticipate the data requirements based on the data access trajectories of similar historical tasks. Then, through a specific cache control unit, we proactively retrieve the predicted data and load it into the L3 cache according to its importance and access probability priority. The L3 cache then gradually passes the data to the L2 and L1 caches as needed, ensuring that critical data can be quickly accessed when required.

[0060] The intelligent prefetching mechanism can preload data that a task is likely to need later into the cache. This means that when a task actually requests data, a large amount of data already exists in the cache, significantly reducing waiting time and improving the smoothness and continuity of task execution. From an energy perspective, it avoids the extra energy consumption caused by frequent access requests to main memory, while also reducing power consumption fluctuations. This makes the overall energy efficiency of the computing unit more stable and efficient, providing more ample resource space and time window for subsequent task scheduling and load balancing.

[0061] Building upon an intelligent prefetching mechanism and a multi-level caching architecture, the task scheduling algorithm is deeply optimized. First, each task to be executed is sorted according to a predefined priority rule, such as the urgency of the task or the level of real-time requirements. Simultaneously, considering the current load of the computing units, tasks are rationally allocated to different computing cores or threads by real-time monitoring of computing resource occupancy. During resource allocation, the location information of prefetched data is fully considered, ensuring that tasks are executed as close as possible to computing resources with prefetched data to reduce data transmission latency. The task queue is also dynamically adjusted; if a task is found to be blocked due to inaccurate data prefetching or other reasons, it can be rescheduled promptly to ensure the efficient progress of the overall task.

[0062] Priority- and load-aware scheduling strategies ensure that high-priority tasks receive computing resources first, meeting the timely processing needs of real-time tasks and improving system response speed. Furthermore, they rationally allocate computing resources, avoiding excessive concentration or waste, and maintaining a balanced and efficient utilization of computing unit load. Simultaneously, optimizing task allocation by combining prefetch data location further reduces data transmission overhead, comprehensively improving the parallel processing capability and task execution efficiency of the NPU computing unit, and providing a more reliable scheduling guarantee for maintaining load balancing.

[0063] Based on the task scheduling algorithm, real-time load data is collected for each computing core, thread, and overall architecture within the computing unit. This includes key metrics such as CPU utilization, memory usage, and cache hit rate. A dynamic load assessment model is established, and warnings are issued to computing units with excessive load based on set load balancing thresholds. Subsequently, improved load balancing algorithms, such as a dynamic load allocation strategy based on ant colony optimization, are used to adjust the distribution ratio of tasks across different computing units in real time, migrating some tasks from high-load units to relatively low-load units. Simultaneously, based on the storage status of multi-level caches, data caching and transmission paths are adjusted collaboratively to ensure that tasks can quickly obtain the required data after migration, maintaining stable system operation.

[0064] Real-time load monitoring and dynamic task allocation effectively prevent individual computing units from experiencing performance bottlenecks or even overheating damage due to excessive load, extending equipment lifespan. From an overall performance perspective, load balancing ensures that all computing units work collaboratively and maximize resource utilization, reducing task waiting and blocking, and improving the throughput and energy efficiency of the entire computing system. Co-optimization with the caching architecture also avoids data access chaos caused by task migration, ensuring the continuity and stability of system operation.

[0065] Based on the output data of the load balancing algorithm, a preset threshold for the load of computing units is set. This threshold comprehensively considers factors such as the power consumption limit of the computing unit, performance requirements, and the minimum execution frequency of the task. The load of the computing units is monitored in real time. When the load falls below the threshold for a continuous period, the intelligent sleep trigger mechanism automatically sends a sleep command to the corresponding computing unit through the system's internal power management module. The sleep command gradually shuts down unnecessary functional modules of the unit, reduces the clock frequency and supply voltage, and puts it into a low-power sleep state. Simultaneously, critical data within the sleep unit is protected, such as storing intermediate data of unfinished tasks in the L3 cache or a specific area of ​​main memory to ensure rapid resumption of execution upon subsequent wake-up.

[0066] The intelligent sleep trigger mechanism can reduce power consumption in a timely manner when the computing unit load is low, avoiding unnecessary energy waste. This is especially beneficial when computing tasks are intermittent or fluctuating, significantly improving energy efficiency. From a system stability perspective, rationally controlling the sleep and wake-up of each unit avoids hardware wear and data loss risks caused by frequent power-on and power-off, ensuring long-term stable system operation. Combined with a multi-level caching data protection strategy, it can also accelerate the system's transition from sleep to normal operation, improving the user experience.

[0067] The optimized multi-level caching structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm, and sleep triggering mechanism are deeply integrated to build a unified energy efficiency control framework. Within this framework, modules interact and share information through standardized interfaces. For example, the intelligent prefetching mechanism provides real-time feedback on the location and size of prefetched data to the task scheduling algorithm for resource allocation; the load balancing algorithm informs the cache management module of the adjusted load distribution so that cache allocation strategies can be dynamically adjusted. Simultaneously, a series of collaborative working rules are established, such as activating modules sequentially during sleep / wake-up, restoring cached data first, and then gradually restoring computing resources, ensuring the entire system operates efficiently and in a coordinated manner.

[0068] The integrated energy efficiency control strategy enables complementary advantages among various optimization modules. Multi-level caching and intelligent prefetching provide efficient data support for task execution, while reasonable task scheduling and load balancing ensure full utilization of computing resources. The intelligent sleep mechanism effectively saves energy during idle periods. From a system-wide perspective, this integrated strategy allows the NPU computing unit to meet high-performance computing demands while keeping energy consumption within a reasonable range, improving the system's cost-effectiveness and competitiveness. It can be widely applied in various scenarios such as high-performance computing and artificial intelligence inference, contributing to energy conservation and emission reduction goals.

[0069] Specifically, the steps for optimizing the cache structure of the NPU computing unit include:

[0070] The L1 cache is designed as a small-capacity, low-latency cache, located close to the computing unit, and used to store the most frequently accessed data;

[0071] The L2 cache is designed as a cache with a large capacity and moderate latency, used to store data accessed in a moderately frequent manner.

[0072] The L3 cache is designed as a cache with the largest capacity and the highest latency, used to store data that is not accessed so frequently.

[0073] By analyzing task access patterns, the allocation strategy of each level of cache can be dynamically adjusted to further optimize cache utilization efficiency.

[0074] It should be noted that in optimizing the NPU compute unit cache structure, the L1 cache was designed first. Specifically, the L1 cache was constructed as a storage module with small capacity and low latency, and deployed closely near the compute units. This layout allows the L1 cache to respond to data access requests from the compute units at extremely high speeds, thus providing the compute units with the most frequently accessed data and ensuring efficient data interaction. Since the L1 cache capacity is relatively limited, an L2 cache was subsequently set up between the L1 cache and main memory to further expand the storage scale and balance access latency. This L2 cache was designed with a large capacity and moderate latency, primarily responsible for storing data accessed at medium frequency, effectively supplementing the storage capacity of the L1 cache and improving the flexibility and adaptability of the entire cache system. In addition, an L3 cache was configured at an outermost layer, with maximum capacity but correspondingly higher latency, mainly used to store data with relatively low access frequency, providing broader storage space support for the entire cache system. During actual operation, the system dynamically adjusts the allocation strategy of each cache level according to the task access patterns. Specifically, the system continuously monitors the data access behavior of currently running tasks, including key information such as data request frequency, access order, and data block size. For example, when it detects that a certain type of task frequently accesses data in a specific region during execution and exhibits continuous access characteristics, the system will, according to preset allocation rules, appropriately increase the capacity allocation of the corresponding cache region or adjust the storage location of the data in the cache to bring it closer to the computing unit, thereby accelerating access speed. Conversely, if it finds that certain data has not been accessed for a long time, it will correspondingly reduce the space occupied by these regions in the cache, and the freed-up cache space will be reallocated to tasks or data with higher access demands. At the same time, it will also combine factors such as task priority and computing unit load to finely control cache allocation, ensuring that high-priority tasks can obtain sufficient cache resources first, guaranteeing the efficient execution of critical tasks.

[0075] By designing the L1 cache to be small in capacity, low in latency, and close to the computing unit, extremely fast access to the most frequently accessed data is achieved. Because the L1 cache is adjacent to the computing unit, the data transmission path is short, and access latency is significantly reduced. This allows the computing unit to quickly obtain the required data when performing high-frequency data request tasks, reducing waiting time and improving the computing unit's operational efficiency, thus laying the foundation for efficient startup of the entire computing process. The L2 cache, with its larger capacity and moderate latency, can store data accessed at medium frequency, effectively expanding the storage scale of the cache system while avoiding excessive latency caused by excessive capacity. This allows data accessed between high and low frequencies to be accessed promptly, reducing the demand on main memory and further optimizing data access efficiency. The large capacity of the L3 cache provides ample data storage backup support for the entire cache system, capable of storing less frequently accessed but somewhat related data, thereby reducing the overall dependence on main memory and mitigating the high latency and high power consumption issues associated with main memory access. In summary, this optimized three-level caching structure ensures high efficiency and low latency in data access across multiple levels, improving the overall performance and energy efficiency of the NPU computing unit and providing strong support for its stable and efficient operation in complex computing tasks. Furthermore, by real-time monitoring and analysis of task access patterns and dynamically adjusting the allocation strategies of each level of cache accordingly, the potential of cache resources is fully explored, enabling each level of cache to more accurately match the access needs of different tasks. This not only improves cache utilization efficiency and avoids resource waste, but also allows for timely responses to dynamic changes in tasks, rationally allocating cache space and further optimizing overall data access efficiency. Ultimately, this significantly improves the overall performance and energy efficiency of the NPU computing unit in handling complex and variable task scenarios, enhancing its competitiveness and applicability in practical applications.

[0076] Specifically, the steps for designing the intelligent prefetching mechanism include:

[0077] Collect current task access patterns and historical data, and analyze the locality and frequency of data access;

[0078] Based on the analysis results, predict the data blocks that may need to be accessed in the future and mark these data blocks as prefetch targets;

[0079] In a multi-level caching structure, the prefetch target data is loaded into the L2 cache first. If the L2 cache space is insufficient, the data is loaded into the L1 or L3 cache according to the access priority.

[0080] Monitor data access in real time and dynamically adjust the prefetching strategy to adapt to changes in task access patterns.

[0081] It should be noted that in the initial stage of designing the intelligent prefetching mechanism, the system deeply and comprehensively collects access patterns and historical data for the currently running task. Specifically, by setting monitoring points at the data interaction interfaces between the computing unit and the cache, and between the cache and main memory, every data access request during task execution is captured in real time, including key information such as the accessed data address, access timestamp, and number of accesses, and this data is stored in a dedicated analysis buffer. Simultaneously, historical data records similar to and related to the current task type are retrieved from the historical task database. These records cover the access behavior characteristics of past tasks at different stages and under different load conditions. Based on this, a locality analysis algorithm is used, combining the principles of temporal and spatial locality, to jointly analyze the collected current access pattern data and historical data, calculate the access frequency distribution of data blocks, identify which data blocks are frequently accessed in the recent and long term, and the distribution patterns of these data blocks in memory space, thereby accurately locating hot data areas with high locality and high access frequency.

[0082] By collecting and analyzing current task access patterns and historical data, we can accurately grasp the behavioral characteristics of data access. In-depth analysis of locality and access frequency helps identify which data blocks have a high probability of being accessed during task execution, providing crucial information for subsequent prefetching decisions. This step constructs a cognitive model of data access behavior, enabling the system to understand the popularity and spatial distribution characteristics of data. This lays a solid data foundation for the implementation of intelligent prefetching mechanisms, thereby improving the accuracy of prefetching, making prefetching operations targeted and avoiding blind operations, and laying the foundation for efficient data prefetching in the future.

[0083] After analyzing the locality and frequency of data access, the system enters the prediction phase. Based on the previously constructed data access behavior cognitive model, various prediction algorithms, such as Markov chain prediction models and machine learning-based time series prediction algorithms, are used to predict the data blocks that the task may access subsequently. For example, the Markov chain model can predict the most likely data block to be accessed next based on the current data access state transition probability matrix; the time series prediction algorithm combines the temporal characteristics of historical access data to estimate the data access demand at a specific future time point. The candidate data blocks obtained from these prediction algorithms are comprehensively evaluated to select the set of data blocks with the highest access probability. Subsequently, these selected data blocks are marked, and special prefetch identifiers are added to them in the data management structure to distinguish them from other ordinary data blocks, forming a clear prefetch target list. This list will serve as the direct basis for subsequent data prefetching operations.

[0084] The steps of predicting data blocks and marking prefetching targets based on the analysis results realize the transformation from recognizing data access behavior to prefetching decision-making. The application of predictive algorithms can anticipate the potential data needs of a task, identifying data blocks most likely to be accessed as prefetching targets in advance. This allows the system to proactively and forward-lookingly prefetch data, rather than passively waiting for data requests. This process optimizes the timing and selection of data prefetching targets, reducing delays caused by waiting for data during task execution, improving the continuity and smoothness of task execution, and ultimately enhancing the overall system performance and response speed.

[0085] Once the target data for prefetching is determined, the system begins the data prefetching operation. First, according to the preset cache allocation priority rules, the target data is prioritized for loading into the L2 cache. Specifically, the system checks the current occupancy and available space of the L2 cache, and calculates the amount of prefetched data that can be accommodated in the L2 cache, taking into account the size and number of data blocks. For prefetched data blocks that can fit into the L2 cache, the cache controller issues data transfer instructions to sequentially load these data blocks from main memory or other storage locations into the designated area of ​​the L2 cache, and updates the cache directory and status information to ensure that the location and status of the data blocks in the cache can be accurately tracked by the system. However, if there is insufficient L2 cache space, the system will activate a cache selection strategy based on access priority. The access priority is determined by a comprehensive evaluation of multiple factors, including the access frequency of the data block, access timeliness, and the task's dependence on the data block. For high-priority data blocks that play a crucial role in the task execution process and require frequent access, the system will prioritize loading them into the L1 cache, which is closer to the computing unit and has a faster access speed. Even if the L1 cache capacity is relatively small, it will still allocate L1 cache space reasonably to these critical data blocks to meet the fast access requirements of high-priority data. For data blocks with relatively low access priority, they will be placed in the L3 cache, which has a larger capacity but slightly higher latency, in order to balance the utilization efficiency of cache space and the timeliness of data access.

[0086] By loading data prefetched according to priority within a multi-level caching structure, fine-grained management of cache resources is achieved. Prioritizing loading target data into the L2 cache fully leverages its relatively large capacity and moderate latency, meeting task data access needs in most cases, reducing main memory accesses, and lowering access latency and power consumption. When L2 cache space is insufficient, data is flexibly loaded into the L1 or L3 cache based on access priority, further optimizing cache space utilization efficiency. This ensures high-priority data receives faster access, while low-priority data is not discarded due to cache space limitations, thus improving overall data prefetching efficiency. This process rationally allocates cache resources based on data importance and access characteristics, aiming to improve data prefetching success rate, ensuring timely and efficient data retrieval during task execution, thereby enhancing overall system performance and energy efficiency.

[0087] To enable the intelligent prefetching mechanism to adapt to dynamic changes in task access patterns, a real-time monitoring module is implemented during the prefetching operation. This module continuously collects real-time data access information by establishing efficient communication channels between computing units, caches at all levels, and main memory. This includes key indicators such as changes in the access frequency of current data blocks, adjustments to the access order, the generation of new data requests, and cache hit rates. Simultaneously, it is deeply integrated with the task scheduling system to obtain dynamic adjustment information on task priorities and changes in computing unit load. Based on this real-time data, the system triggers a dynamic adjustment process for the prefetching strategy at a preset monitoring frequency (e.g., evaluation every millisecond or every second). During the adjustment process, the current prefetching target list is first re-evaluated, removing data blocks with reduced access probability or no longer needed, while new prefetching target data blocks are added based on predictions of new access patterns. Furthermore, the data distribution in the cache is optimized. For example, if the access frequency of a certain data in the L2 cache suddenly increases significantly, reaching the access priority threshold of the L1 cache, it is migrated to the L1 cache; conversely, if the data access frequency decreases, it is migrated to the L3 cache or cache space is released. The entire dynamic adjustment process is achieved through the collaborative work of the cache management unit and the prefetch control unit, ensuring that the prefetch strategy can closely follow the evolution of task access patterns and always maintain high efficiency and adaptability.

[0088] The steps of real-time monitoring of data access and dynamic adjustment of the prefetching strategy endow the intelligent prefetching mechanism with powerful flexibility and adaptability. By capturing changes in task access patterns in real time, the prefetching target list can be updated promptly, ensuring that the prefetched data always matches the actual needs of the task. This avoids outdated or redundant prefetched data and reduces cache space waste and data transfer overhead caused by invalid prefetching. Dynamically adjusting the data distribution in the cache further optimizes the utilization efficiency of each level of cache, making data storage more in line with the real-time access characteristics of the task, thereby improving the overall effectiveness of data prefetching and the system's response speed. The purpose of this process is to maintain a tight coupling between the prefetching mechanism and the dynamic behavior of the task, ensuring that the system can still operate efficiently and stably when facing complex and ever-changing computing tasks, maximizing the advantages of the intelligent prefetching mechanism, and improving the performance and energy efficiency of the entire NPU computing unit.

[0089] Specifically, the steps of optimizing the task scheduling algorithm include:

[0090] Based on task priority and computational complexity, tasks are divided into high-priority tasks and low-priority tasks.

[0091] For high-priority tasks, computing resources are allocated first, and the data required by the task is ensured to be loaded into the cache based on the output of the prefetch mechanism.

[0092] For low-priority tasks, their execution order is dynamically adjusted based on the current load of the computing unit in order to make full use of computing resources;

[0093] Regularly assess the execution status of tasks and the load on computing units, and dynamically adjust task scheduling strategies to achieve load balancing.

[0094] It should be noted that when optimizing the task scheduling algorithm of the NPU computing unit, tasks are first precisely classified. By comprehensively evaluating task priority and computational complexity, tasks are divided into high-priority and low-priority tasks. For high-priority tasks, the system prioritizes allocating computing resources to them. Simultaneously, based on the data loading status information provided by the prefetching mechanism, it ensures that the data required by these tasks is pre-loaded into the cache, so that the tasks can quickly obtain the necessary data and execute efficiently once started. For low-priority tasks, a dynamic adjustment strategy is adopted based on the current load of the computing unit. This includes real-time monitoring of the computing resource occupancy status, analyzing the waiting time and resource requirements of low-priority tasks in the task queue, and adjusting their execution order accordingly. The system will execute tasks that can complete faster under the current load and have better overall resource utilization in advance, thereby improving resource utilization efficiency. Furthermore, the system will periodically evaluate the execution status of tasks, including task progress, remaining computation, data preparation status, and changes in the load of the computing unit. Based on these evaluation results, the task scheduling strategy is dynamically adjusted. For example, when the load is too high, the execution of low-priority tasks is delayed, and when the load is low, some low-priority tasks are executed earlier to achieve load balancing of the entire computing unit.

[0095] By prioritizing tasks and allocating resources and adjusting execution order accordingly, efficient management of computing resources is achieved. High-priority tasks can quickly acquire resources and start rapidly using prefetched data, reducing waiting time and ensuring the timeliness and efficiency of critical tasks. This is crucial for applications with high real-time requirements, such as real-time image processing and online inference, ensuring priority for these tasks. Dynamic adjustment of the execution order of low-priority tasks fully utilizes idle periods of computing resources, avoiding resource waste and improving overall resource utilization. Regularly evaluating and dynamically adjusting task scheduling strategies allows for flexible responses to load changes based on actual operating conditions, effectively preventing local overload or uneven resource allocation and achieving global load balancing. This not only improves the stability and reliability of the entire computing unit but also extends the lifespan of the device, reduces energy consumption, and ultimately enables the entire NPU computing unit to maintain a highly efficient and stable operating state when facing complex and ever-changing task scenarios, significantly improving its performance and energy efficiency in practical applications.

[0096] Specifically, the steps of the improved load balancing algorithm include:

[0097] Real-time monitoring of the load of each NPU computing unit, including the execution status of the current task and the usage of the cache;

[0098] Based on the load, the task allocation is dynamically adjusted, and some tasks on the computing units with high load are migrated to the computing units with low load.

[0099] During task migration, data locality and the output of the prefetching mechanism are considered to ensure that the required data can be quickly obtained after the task migration.

[0100] Regularly evaluate the effectiveness of load balancing and further optimize the load balancing algorithm based on the evaluation results.

[0101] It should be noted that in improving the load balancing algorithm, a comprehensive monitoring system was first established to track the load status of each NPU computing unit in real time, covering various aspects such as task execution progress, stage, cache usage, and frequency. Specifically, dedicated monitoring modules are deployed in each computing unit. These modules collect key performance indicators such as CPU utilization, memory usage ratio, and cache hit rate at extremely high frequencies and transmit this data to the central load monitoring system in real time. Simultaneously, details such as the waiting time, estimated execution time, and resource requirements of each task in the current task queue are recorded to comprehensively understand the task execution status. Furthermore, the monitoring system performs detailed analysis of cache usage, including the storage space usage and data read / write frequency of each level of cache (L1, L2, L3), thus providing a detailed data foundation for subsequent load balancing decisions. The monitoring process employs a multi-threaded parallel processing mechanism to ensure accurate and timely acquisition of load information without affecting the normal operation of the computing units.

[0102] Such detailed and real-time load monitoring allows for precise understanding of the operational status of each computing unit and the dynamic changes in tasks and cache. This provides highly timely and accurate data support for subsequent dynamic adjustments to task allocation, ensuring the scientific and rational nature of load balancing decisions. Its function is to promptly identify computing units with excessive loads and those with idle resources, providing clear directional guidance for task migration, thereby laying a solid foundation for achieving efficient load balancing overall.

[0103] Based on load data acquired through real-time monitoring, a pre-defined load balancing algorithm dynamically adjusts task allocation. First, the load of each computing unit is quantitatively assessed, calculating its current load value and comparing it to the system's pre-defined load balancing threshold. When the load value of a computing unit exceeds the threshold, it is determined that the unit is overloaded and task migration is required. At this point, the algorithm selects suitable tasks for migration from the task queue of the overloaded unit. The selection criteria comprehensively consider factors such as task priority, execution progress, required resource type and quantity, ensuring that high-priority and easily migrated tasks are prioritized. Next, the most suitable target unit to receive the migrated tasks is selected from the computing units with lower loads. Selection criteria include the target unit's current load value, available resources, and data correlation with the original task, ensuring efficient execution after task migration. After determining the migration task and its target unit, the selected task is retrieved from the task queue of the source unit through the task scheduling module inside the system, and its relevant data and execution status information are transmitted to the target unit. At the same time, the execution order of the migration task is reasonably arranged in the task queue of the target unit to ensure that it can be quickly integrated into the computing process of the new unit. The entire migration process adopts an efficient concurrent data transmission mechanism to reduce the impact of migration on system performance.

[0104] This dynamic task allocation and adjustment mechanism can respond promptly to changes in the load of each computing unit, rationally distributing tasks from overloaded units to other units with idle resources. This avoids localized performance bottlenecks caused by uneven task allocation, fully utilizes the overall system's computing resources, and improves the throughput of the computing unit cluster. Simultaneously, by comprehensively considering task characteristics and unit resource selection principles, it ensures that tasks can quickly adapt to the new environment and execute efficiently after migration, reducing the additional overhead caused by migration. This effectively improves the overall system's operating efficiency and performance, ensuring smoothness and stability in multi-task parallel processing scenarios.

[0105] When migrating tasks, the locality of data and the output of the prefetching mechanism are fully considered to optimize the migration process and ensure efficient data access after migration. Specifically, the historical data access patterns of the migration task and the data content and storage location prefetched by the current prefetching mechanism for the task are analyzed first. For tasks with strong data locality, i.e., tasks tend to access specific regions or related data blocks during execution, these localized data and their surrounding potentially accessed related data are migrated to the target computing unit first. These data are then rationally pre-allocated and laid out in the target unit's cache to ensure that the task can quickly retrieve the required data from the cache when executing in the new unit, reducing the number of accesses to main memory. Simultaneously, based on the output of the prefetching mechanism, prefetched but not yet accessed data is also migrated to the corresponding cache location in the target unit, so that the task can seamlessly utilize this prefetched data during subsequent execution, maintaining execution continuity and efficiency. Furthermore, a data synchronization mechanism is established between the source and target units during the migration process. When the task discovers incomplete or inconsistent data in the cache during execution in the target unit, it can promptly retrieve the latest data from the source unit, ensuring data accuracy and consistency.

[0106] By integrating the output of data locality and prefetching mechanisms into the task migration process, the migrated tasks can quickly adapt to the new data environment in the target computing unit, significantly reducing access latency and cache misses caused by data location changes. This improves the execution efficiency of tasks in the new unit, avoids high latency and high power consumption issues caused by frequent main memory access, and reduces task blocking while waiting for data. This ensures the smoothness and stability of parallel task processing throughout the system, further enhancing overall system performance and energy efficiency.

[0107] To continuously optimize the load balancing algorithm, a periodic evaluation mechanism was established. First, a reasonable evaluation cycle was set, comprehensively considering factors such as the dynamic frequency of system task changes, load fluctuation characteristics, and tolerance for evaluation overhead to ensure the timeliness and effectiveness of the evaluation. At the end of each evaluation cycle, the evaluation module collects all load-related data since the last evaluation, including load change curves for each computing unit, task migration records, execution times of tasks before and after migration, changes in cache hit rate, and other multi-dimensional information. Then, a pre-defined evaluation index system was used to quantitatively analyze the load balancing effect. Evaluation indicators include load balancing degree (statistical indicators such as the standard deviation of load values ​​for all units), average task waiting time, task execution success rate, and system throughput change rate. By comparing with historical evaluation results and pre-defined target performance indicators, it was determined whether the current load balancing algorithm's effect meets the system performance requirements. If the evaluation reveals that the load balancing effect is not as expected, such as a continuous deterioration in load balancing or excessively long task waiting times, the causes will be further analyzed. These could include unreasonable task selection strategies, the need to adjust the criteria for selecting target units, or defects in the data migration mechanism. Based on this, targeted optimizations and improvements will be made to the corresponding parts of the load balancing algorithm, such as adjusting task priority weights, optimizing the target unit load prediction model, and improving the data migration synchronization mechanism. The optimized algorithm will be used in subsequent load balancing operations, and its effectiveness will be evaluated again in the next evaluation cycle, forming a closed-loop control process of continuous improvement.

[0108] By regularly conducting quantitative evaluation and analysis of load balancing performance, potential problems and shortcomings in the algorithm can be identified in a timely manner, providing a clear direction and basis for continuous improvement. This enables the load balancing algorithm to continuously adapt to changes in system operating conditions and task characteristics, gradually improving its accuracy and effectiveness, and ensuring the system maintains a high-efficiency, high-performance operating state in the long term. Ultimately, this achieves the goal of continuously improving the overall performance and energy efficiency of the NPU computing unit cluster, enhancing the system's applicability and competitiveness in complex and ever-changing computing task scenarios.

[0109] Specifically, the steps for designing the intelligent sleep trigger mechanism include:

[0110] Set a preset load threshold; when the load on the computing unit falls below this threshold, trigger a sleep mechanism.

[0111] Before triggering hibernation, check the execution status of the current task and the validity of the prefetched data to ensure that the task will not be interrupted due to hibernation;

[0112] Once the hibernation conditions are met, the computing unit is put into hibernation mode and the state information before hibernation is recorded for quick wake-up.

[0113] During hibernation, the system status is continuously monitored, and the computing unit is immediately woken up once a new task arrives or the load increases.

[0114] It should be noted that when designing the intelligent sleep trigger mechanism, a reasonable preset load threshold must first be set. This threshold is not arbitrarily determined, but rather comprehensively considers factors such as the hardware performance of the computing unit, power consumption limitations, and the statistical characteristics of historical task loads. Specifically, the R&D team conducted in-depth analysis of past task execution data, including the load distribution of the computing unit under different task types, task duration, and execution frequency, and used clustering and regression analysis methods from machine learning algorithms to build a load threshold prediction model. This model can dynamically generate an optimal load threshold based on different application scenarios and task combinations. During the operation of the computing unit, the system monitors its load in real time, including several key indicators such as CPU utilization, memory bandwidth utilization, and cache hit rate. Once these comprehensive load indicators remain below the preset threshold for a period of time, the system will determine that the current computing unit is in an inefficient operating state, at which point the conditions for triggering the sleep mechanism are met, and it prepares to enter a sleep state. Meanwhile, the triggering mechanism also has a fault tolerance mechanism for delayed triggering. That is, after the hibernation conditions are met, the hibernation operation is not executed immediately, but a short delay is performed to observe and confirm that the load is indeed at a stable low level. This avoids frequent hibernation and wake-up operations caused by short-term load fluctuations and ensures the stability of system operation.

[0115] By setting a preset load threshold and triggering a hibernation mechanism accordingly, refined management of the computing unit's operating status is achieved. This scientifically set threshold accurately identifies inefficient operating phases of the computing unit, enabling timely activation of power-saving modes without affecting normal task execution. It effectively reduces overall power consumption and energy waste, with significant energy-saving effects, especially in application scenarios where computing tasks are intermittent or highly fluctuating. Simultaneously, this threshold-based judgment mechanism provides a clear and reliable basis for subsequent hibernation operations, ensuring the orderly initiation of the entire hibernation process and laying the foundation for system energy-saving optimization.

[0116] Before triggering the hibernation mechanism, the system performs a crucial check to ensure tasks are not interrupted by hibernation. Specifically, the system first checks all currently running tasks one by one, examining the execution status of each task, including its process position, whether it is waiting for data, and whether it is performing critical operations. For each task, the system calls its status monitoring interface to read the current task execution flag and status code to determine if the task is in an interruptible state. For example, if a task is currently performing a file read / write operation and has not yet completed, the system identifies it as uninterruptible, temporarily suspends the hibernation operation, and marks the task as a priority task until it completes its current operation or reaches an interruptible point. Simultaneously, the system also performs a comprehensive check on the validity of the prefetched data. Since the prefetched data is based on predictions of historical access patterns, there may be inaccurate predictions or outdated data. Therefore, the system compares the timestamp and version number of the prefetched data to check if it matches the actual needs of the current task. If the system finds that the prefetched data does not match the requirements of the current stage of the task, it will immediately start the data update process, retrieve the required data from main memory or a higher-level cache, and store it in a dedicated area of ​​the L3 cache. This ensures that the task can quickly retrieve accurate data after waking from sleep, and avoids task execution failure or performance degradation due to data errors.

[0117] This step provides dual assurance for task continuity and data accuracy. By rigorously checking the task execution status, it ensures that the hibernation operation is only performed when the task is in a safe, interruptible state, thus preventing abnormal interruptions due to hibernation and guaranteeing the integrity of task execution and the reliability of the system. Simultaneously, the validity verification and updating of pre-fetched data ensures that the task can quickly obtain accurate and timely data after waking from hibernation, maintaining the continuity and efficiency of task execution. This not only improves the user experience but also enhances the system's stability and fault tolerance when handling complex and ever-changing task scenarios.

[0118] Once the hibernation conditions are met and task and data checks are passed, the computing unit will be placed into hibernation mode, while the system meticulously records the state information prior to hibernation. During hibernation, the system first sends a series of pre-hibernation instructions to the computing unit, gradually reducing its operating frequency and supply voltage, and shutting down unnecessary functional modules, such as dynamic refresh circuits for some caches and data bus drive circuits, to reduce silent power consumption. This process is not instantaneous but proceeds in stages according to a pre-designed hibernation procedure, ensuring a smooth transition to a low-power state for the computing unit. Simultaneously, the system records various key state information prior to hibernation, including the computing unit's register values, the program counter's location, valid data blocks in each level of cache and their storage locations, and the task queue's status. This state information is stored in specially designed non-volatile memory areas, such as static random access memory (SRAM) integrated within the computing chip or in a backup battery-powered register set. To ensure data integrity, a cyclic redundancy check (CRC) algorithm is used to verify the data during storage, and the checksum is recorded. Furthermore, the system records the timestamp of the hibernation event and the reason for hibernation for subsequent performance analysis and troubleshooting. For example, if hibernation is triggered due to prolonged low load, it will be marked as "low load hibernation"; if it is triggered by the completion of a specific task, it will be marked as "task completed hibernation".

[0119] By smoothly placing the computing units into a hibernation state and recording detailed status information, effective energy conservation and rapid task recovery capabilities are achieved. Gradually reducing the operating frequency and supply voltage avoids potential hardware damage caused by excessive instantaneous power consumption fluctuations, extending the lifespan of the computing units. The comprehensive recording of pre-hibernation status information provides a solid foundation for subsequent rapid wake-up. Once the system is awakened, this status information guides the computing units to quickly restore their pre-hibernation operating state, reducing task recovery time and improving system response speed and overall performance. Simultaneously, the recorded hibernation reason and timestamp information provide valuable data support for system optimization and fault diagnosis, contributing to further improvements to the hibernation-wake-up mechanism.

[0120] While the computing unit is in sleep mode, the system does not completely stop working but continuously monitors its status to respond promptly to new task requests or load changes. Specifically, the system keeps some low-power monitoring modules active. These modules are primarily responsible for listening to specific wake-up signals on the system bus, such as interrupt requests from external devices, periodic wake-up events triggered by timers, and data packets received from the network interface. Simultaneously, the system tracks the load trends of the computing unit in real time, periodically sampling the resource usage of the computing unit through a lightweight load monitoring algorithm, including CPU idle time and remaining memory capacity. Once a new task arrives or an increasing load trend is detected—for example, when new data is transmitted from external sensors requiring processing, or when a high-priority task is submitted to the task queue—the system immediately initiates the wake-up process. During the wake-up process, a wake-up command is first sent to the computing unit, gradually restoring its power supply voltage and operating frequency to normal operating levels. Subsequently, based on the state information recorded before sleep mode, the system reloads key data such as register values ​​and program counter positions into the corresponding locations on the computing unit and restores valid data blocks from the cache. Simultaneously, the system verifies the integrity of the recovered data. If data is found to be damaged or lost during hibernation, a data recovery mechanism is automatically triggered to retrieve the data from main memory or other backup locations. After data recovery is complete, the computing unit resumes execution of tasks from the point of interruption before hibernation. The entire wake-up process is designed to be as fast and seamless as possible to minimize the impact on system performance.

[0121] This step enables rapid response to dynamic system changes, ensuring that computing units can still handle new task demands or load changes in a timely manner during sleep periods. Continuous monitoring of system status prevents service quality degradation caused by missed task deadlines or response delays, guaranteeing system real-time performance and availability. The ability to quickly wake up computing units further enhances the system's flexibility and adaptability, allowing it to achieve a good balance between energy saving and efficient response. Especially in application scenarios with uncertain task loads and high real-time requirements, this mechanism can significantly improve overall system performance and user experience. Simultaneously, data integrity verification and recovery mechanisms ensure the accuracy and reliability of task execution, further solidifying system stability.

[0122] Specifically, the steps of the integrated and optimized multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm, and sleep triggering mechanism include:

[0123] The multi-level caching structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism are integrated into a unified energy efficiency control framework;

[0124] Within the framework, the parameters and strategies of each mechanism are dynamically adjusted according to different application scenarios and task requirements;

[0125] Implement a central controller to coordinate the operation of various mechanisms and ensure they work together to achieve optimal energy efficiency control.

[0126] Regularly assess the energy efficiency of the entire system and further optimize the integration strategy of each mechanism based on the assessment results.

[0127] It should be noted that this invention integrates a multi-level caching structure, an intelligent prefetching mechanism, a task scheduling algorithm, a load balancing algorithm, and a sleep triggering mechanism into a unified energy efficiency control framework. Specifically, the first step is to construct a highly integrated and scalable framework architecture capable of accommodating and coordinating the operation of each mechanism. Within the framework, each mechanism is assigned an independent functional module, and standardized interfaces are established to ensure seamless integration and information sharing. For example, the multi-level caching structure serves as the core data storage module of the framework; the intelligent prefetching mechanism predicts and preloads data in real time based on the data flow characteristics of the caching structure; the task scheduling algorithm rationally allocates tasks based on the cache status and the location of prefetched data; the load balancing algorithm dynamically adjusts resource allocation based on task scheduling; and the sleep triggering mechanism issues sleep commands based on the overall load. Through unified coding standards and data interaction protocols, these modules are tightly coupled and integrated into a complete system. In actual operation, the framework initializes the parameters of each module, such as cache size, prefetch threshold, and scheduling priority, while reserving necessary system resources, such as memory space and computing cores, for the operation of these mechanisms, ensuring stable operation of the framework and maximizing the advantages of each mechanism.

[0128] This integration process enables the collaborative operation of various mechanisms, breaking down the data silos and resource waste problems caused by the independent operation of these mechanisms in the past. The combination of a multi-level caching structure and an intelligent prefetching mechanism can pre-deploy data to appropriate locations, improving data access efficiency; the linkage between task scheduling algorithms and load balancing algorithms ensures reasonable task allocation and efficient resource utilization; and the sleep-triggered mechanism can save energy in a timely manner when resources are idle. This deep integration improves the overall system's synergy and resource utilization, provides a unified platform for dynamically adjusting parameters according to application scenarios, and enhances the system's adaptability and flexibility in diverse tasks.

[0129] Based on a unified energy efficiency control framework, the parameters and strategies of each mechanism are dynamically adjusted according to different application scenarios and task requirements. Specifically, the framework monitors the system's operating status in real time, including key indicators such as task type, data access mode, and computing unit load. For example, when the application scenario is detected to switch from low-intensity single-task processing to high-intensity multi-task parallel processing, the system will increase the allocation ratio of L1 and L2 caches in the multi-level cache according to preset adjustment rules to meet high-frequency data access requirements; at the same time, the intelligent prefetching mechanism will increase the prefetching threshold to load more potentially needed data in advance. The task scheduling algorithm will reduce the priority of tasks with lower real-time requirements, prioritizing the execution of critical tasks; the load balancing algorithm will increase the granularity of resource allocation to quickly respond to resource requests from multiple tasks; and the load threshold of the sleep triggering mechanism will also be increased accordingly to avoid frequent sleep affecting task execution. These adjustments are implemented through the feedback control loop within the framework. The system will periodically collect operating data, match it with preset application scenario models, and then trigger corresponding parameter updates and strategy switching operations to ensure that each mechanism is always in optimal operating condition.

[0130] The system's ability to dynamically adjust parameters and strategies allows it to flexibly adapt to changing application scenarios and task requirements. By monitoring operational status in real time and responding quickly, the system can accurately match resource allocation with task characteristics, avoiding excessive or insufficient resource consumption. In multi-tasking scenarios, adjusting parameters such as caching, prefetching, and scheduling can improve task execution efficiency and system throughput; in lightweight task scenarios, parameter scaling can reduce power consumption and resource consumption. This flexibility not only enhances the system's versatility and practicality but also strengthens its market competitiveness, enabling it to meet the diverse needs of more users in different scenarios.

[0131] To achieve efficient collaboration among various mechanisms, this invention designs and implements a central controller within a unified energy efficiency control framework. This central controller first needs comprehensive monitoring capabilities to acquire real-time operational status data for each mechanism, such as cache hit rate, prefetch accuracy, and task queue length. Based on this data, the central controller uses built-in coordination algorithms, such as weighted task allocation algorithms or priority-based resource scheduling algorithms, to uniformly direct the operation of each mechanism. For example, when it detects that a computing unit is overloaded and its cache hit rate is declining, the central controller simultaneously instructs the task scheduling algorithm to reduce the allocation of non-critical tasks to that unit, instructs the load balancing algorithm to allocate resources from other units, and instructs the intelligent prefetch mechanism to adjust the data prefetching order, prioritizing the urgently needed data for that unit. In actual development, the central controller's algorithms undergo multiple simulation tests and parameter optimizations to ensure the accuracy and timeliness of its decisions. Simultaneously, the central controller also possesses anomaly handling capabilities; when a mechanism malfunctions or malfunctions, it can quickly activate backup strategies, such as temporarily increasing resource allocation to other mechanisms, to ensure stable system operation.

[0132] The introduction of a central controller enables the orderly coordination of various mechanisms, avoiding conflicts and resource competition between them. By uniformly coordinating the operation of each mechanism, the system can maintain stable and efficient operation under complex conditions. For example, in multi-tasking, high-load scenarios, the central controller's rapid decision-making and resource allocation capabilities can effectively prevent system overload and performance degradation; in the event of sudden tasks or abnormal situations, it can ensure task continuity and system reliability. This synergistic effect further improves the overall performance and energy efficiency of the system, providing users with higher-quality and more stable computing services.

[0133] This invention emphasizes regularly evaluating the energy efficiency of the entire system and optimizing the integration strategy of each mechanism based on the evaluation results. Specifically, a comprehensive evaluation index system is first established, covering key indicators such as energy efficiency ratio, task execution success rate, response time, and resource utilization. Then, these index data are periodically collected during system operation to form an evaluation dataset. Data analysis tools and machine learning algorithms are used to mine and model the data, identifying bottlenecks and potential optimization points in system operation. For example, if the evaluation finds a significant decrease in energy efficiency ratio under a specific task type, tracing the data reveals that it is due to insufficient prefetch accuracy of the intelligent prefetch mechanism, leading to frequent cache misses. Based on this, optimization strategies can focus on improving the parameters of the prefetch algorithm or introducing a new prefetch model. The optimized strategy is simulated and verified in a test environment to evaluate its effectiveness, and then the optimized solution is pushed to the production environment through the framework's update mechanism. The entire evaluation and optimization process forms a closed loop, ensuring that the system can continuously improve and maintain optimal operating conditions.

[0134] The steps of regularly evaluating and optimizing integration strategies provide a scientific basis and driving force for continuous system improvement. Through quantitative evaluation indicators and data-driven optimization methods, the system can accurately pinpoint problems and implement targeted improvements, avoiding the risks and resource waste associated with blind optimization. This self-evolutionary capability enables the system to continuously adapt to new technological developments and changing user needs during long-term operation, consistently maintaining high performance and energy efficiency, extending system lifespan and market competitiveness, and creating greater value for users.

[0135] This embodiment also provides an energy efficiency control system for NPU computing unit sleep triggering, including:

[0136] Cache optimization module: Optimizes the cache structure of the NPU computing unit and introduces a multi-level caching mechanism, including L1, L2 and L3 caches, to reduce the number of memory accesses.

[0137] Prefetch design module: Based on the optimized multi-level cache structure, an intelligent prefetch mechanism is designed to predict and preload data that may be needed later by analyzing task access patterns and historical data.

[0138] Scheduling optimization module: Combines the output of the prefetching mechanism to optimize the task scheduling algorithm, adopts a priority- and load-aware scheduling strategy, and rationally allocates the load of computing units.

[0139] Load balancing module: Based on the output of the task scheduling algorithm, improve the load balancing algorithm, monitor the load of the computing unit in real time, dynamically adjust the task allocation, and achieve load balancing.

[0140] Sleep Trigger Module: Based on the output of the load balancing algorithm, an intelligent sleep trigger mechanism is designed to automatically trigger the sleep mode when the load of the computing unit is lower than a preset threshold.

[0141] Strategy Integration Module: Integrates the above-mentioned optimized multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism to form a complete energy efficiency control strategy.

[0142] In summary, this invention achieves the following: by introducing a multi-level caching mechanism to optimize the cache structure, reducing memory access frequency and power consumption; by using an intelligent prefetching mechanism to predict and preload potentially needed data based on task access patterns and historical data, reducing the waiting time of computing units after waking from sleep; by optimizing the task scheduling algorithm and adopting a priority- and load-aware strategy to rationally allocate the load of computing units, improving system response speed and resource utilization; by improving the load balancing algorithm to monitor the load of computing units in real time and dynamically adjust task allocation, achieving load balancing and further improving the overall performance of the system; and by designing an intelligent sleep trigger mechanism that automatically triggers sleep mode when the load of computing units falls below a preset threshold, reducing the static power consumption of the system. Finally, by integrating these mechanisms, a complete energy efficiency control strategy is formed, achieving synergistic optimization at both the hardware and software levels, significantly improving the energy efficiency of the NPU system.

[0143] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for energy efficiency control of NPU computing unit sleep triggering, characterized in that, Includes the following steps: Optimize the cache structure of the NPU computing unit by introducing a multi-level caching mechanism, including L1, L2 and L3 caches, to reduce the number of memory accesses; Based on the optimized multi-level cache structure, an intelligent prefetching mechanism is designed to predict and preload data that may be needed later by analyzing task access patterns and historical data. By combining the output of the prefetching mechanism, the task scheduling algorithm is optimized, and a priority- and load-aware scheduling strategy is adopted to reasonably allocate the load of computing units. Based on the output of the task scheduling algorithm, the load balancing algorithm is improved, the load of the computing unit is monitored in real time, and the task allocation is dynamically adjusted to achieve load balancing. Based on the output of the load balancing algorithm, an intelligent sleep triggering mechanism is designed to automatically trigger the sleep mode when the load of the computing unit is lower than a preset threshold. By integrating the above-mentioned optimized multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism, a complete energy efficiency control strategy is formed.

2. The energy efficiency control method for NPU computing unit sleep triggering as described in claim 1, characterized in that: The steps for optimizing the cache structure of the NPU computing unit include: The L1 cache is designed as a small-capacity, low-latency cache, located close to the computing unit, and used to store the most frequently accessed data; The L2 cache is designed as a cache with a large capacity and moderate latency, used to store data accessed in a moderately frequent manner. The L3 cache is designed as a cache with the largest capacity and the highest latency, used to store data that is not accessed so frequently. By analyzing task access patterns, the allocation strategy of each level of cache can be dynamically adjusted to further optimize cache utilization efficiency.

3. The energy efficiency control method for NPU computing unit sleep triggering as described in claim 2, characterized in that: The steps for designing the intelligent prefetching mechanism include: Collect current task access patterns and historical data, and analyze the locality and frequency of data access; Based on the analysis results, predict the data blocks that may need to be accessed in the future and mark these data blocks as prefetch targets; In a multi-level caching structure, the prefetch target data is loaded into the L2 cache first. If the L2 cache space is insufficient, the data is loaded into the L1 or L3 cache according to the access priority. Monitor data access in real time and dynamically adjust the prefetching strategy to adapt to changes in task access patterns.

4. The energy efficiency control method for NPU computing unit sleep triggering as described in claim 3, characterized in that: The steps of optimizing the task scheduling algorithm include: Based on task priority and computational complexity, tasks are divided into high-priority tasks and low-priority tasks. For high-priority tasks, computing resources are allocated first, and the data required by the task is ensured to be loaded into the cache based on the output of the prefetch mechanism. For low-priority tasks, their execution order is dynamically adjusted based on the current load of the computing unit in order to make full use of computing resources; Regularly assess the execution status of tasks and the load on computing units, and dynamically adjust task scheduling strategies to achieve load balancing.

5. The energy efficiency control method for NPU computing unit sleep triggering as described in claim 4, characterized in that: The steps of the improved load balancing algorithm include: Real-time monitoring of the load of each NPU computing unit, including the execution status of the current task and the usage of the cache; Based on the load, the task allocation is dynamically adjusted, and some tasks on the computing units with high load are migrated to the computing units with low load. During task migration, data locality and the output of the prefetching mechanism are considered to ensure that the required data can be quickly obtained after the task migration. Regularly evaluate the effectiveness of load balancing and further optimize the load balancing algorithm based on the evaluation results.

6. The energy efficiency control method for NPU computing unit sleep triggering as described in claim 5, characterized in that: The steps for designing the intelligent sleep triggering mechanism include: Set a preset load threshold; when the load on the computing unit falls below this threshold, trigger a sleep mechanism. Before triggering hibernation, check the execution status of the current task and the validity of the prefetched data to ensure that the task will not be interrupted due to hibernation; Once the hibernation conditions are met, the computing unit is put into hibernation mode and the state information before hibernation is recorded for quick wake-up. During hibernation, the system status is continuously monitored, and the computing unit is immediately woken up once a new task arrives or the load increases.

7. The energy efficiency control method for NPU computing unit sleep triggering as described in claim 6, characterized in that: The steps of the integrated and optimized multi-level cache structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm, and sleep triggering mechanism include: The multi-level caching structure, intelligent prefetching mechanism, task scheduling algorithm, load balancing algorithm and sleep triggering mechanism are integrated into a unified energy efficiency control framework; Within the framework, the parameters and strategies of each mechanism are dynamically adjusted according to different application scenarios and task requirements; Implement a central controller to coordinate the operation of various mechanisms and ensure they work together to achieve optimal energy efficiency control. Regularly assess the energy efficiency of the entire system and further optimize the integration strategy of each mechanism based on the assessment results.

Images