NPU instruction level context switching method and device

Abstract

The invention provides an NPU instruction level context switching method and device, and relates to the technical field of data processing.The method comprises the steps that when a context switching instruction is received, a non-refresh instruction is forbidden, it is waited for completion of an instruction being executed, and context data is stored in a main storage module; after completion, recovering the context switching related instruction of the NPU module, and executing the target task; and after the target task is executed, recovering the context data to the NPU module, and recovering the execution of the non-refresh instruction. According to the method, optimization is carried out aiming at the problem that an existing NPU architecture generally can only support limited preemption at an algorithm boundary or a specific context switching point and even cannot preemption, so that a high-priority task must wait for a low-priority task to be completed, and remarkable priority inversion and / or key inversion are / is caused. A full-stack solution from a system on chip to an instruction set architecture and then to an operating system kernel is provided, and preemption of an NPU instruction level is achieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an NPU instruction-level context switching method and device. Background Art

[0002] The NPU context switching problem has attracted widespread attention in the research community. Related research is mainly divided into two categories: software methods and hardware methods. However, due to the limitations of its structural characteristics and circuit principles, the existing technology inevitably has some problems and defects. First, the existing NPU architecture usually does not support preemption or only supports limited preemption, and can only preempt at the algorithm boundary or certain specific context switching points. This means that when scheduling tasks, if a low-priority / low-criticality task arrives first, the high-priority / high-criticality task needs to wait for the low-priority / low-criticality task to complete, resulting in significant priority and / or criticality inversion problems. Priority inversion refers to a system with tasks of different priorities in which a low-priority task holds a resource that a high-priority task needs, so the high-priority task is forced to wait for the low-priority task to complete. In this case, the execution of the high-priority task is hindered during the execution of the low-priority task, resulting in system performance and real-time performance being affected. Criticality inversion is a special case of priority inversion in a mixed-criticality system, which occurs when the execution of a high-criticality task is blocked by a low-criticality task in the system degradation mode, and this blocking will cause the system's high-criticality task to be unable to complete on time. Criticality inversion not only affects the execution of tasks sorted according to their criticality, but may also have a serious impact on the security and reliability of the entire system. Second, because the NPU needs to process a large number of parallel computing tasks, its context switching usually involves the preservation and recovery of a large amount of data. This process is not only time-consuming, but also leads to a significant decrease in system performance. Third, when performing context switching, the consistency of the NPU's computing data and configuration data is difficult to guarantee, which may lead to inconsistent data when the task is restored, thus affecting the stability and correctness of the system. Fourth, due to the excessive overhead of traditional context switching, a set of efficient acceleration strategies is crucial. It can effectively reduce the overhead of context switching, optimize resource utilization, and significantly improve the overall performance and real-time performance of the system while ensuring data consistency. This strategy should be designed with comprehensive consideration of the parallel computing characteristics of the NPU and the requirements of task scheduling to ensure that efficient and stable system operation can be maintained in scenarios where tasks are frequently switched. Summary of the invention

[0003] The present application provides an NPU instruction-level context switching method and device, aiming to solve the limitation problem of the existing NPU architecture in preemption capability. The traditional NPU architecture usually does not support preemption or only supports limited preemption at the algorithm boundary or specific context switching point, resulting in high-priority tasks needing to wait for low-priority tasks to complete, thereby causing significant priority inversion and / or criticality inversion problems.

[0004] In a first aspect disclosed in the present application, a method for NPU instruction-level context switching is provided, which is applied to an NPU instruction-level context switching device, wherein the device comprises a software module and a hardware module, wherein the hardware module comprises an NPU module and a main memory module, and wherein the software module executes the steps comprising: when a context switching instruction is received, obtaining a first type of queued instruction sequence, a first execution instruction set and context data of the NPU module, wherein the first type of queued instruction is a non-refresh instruction, and the first execution instruction set is an instruction that is still being executed when the context switching instruction is received; prohibiting execution of the first type of queued instruction sequence, and waiting for the first execution instruction set to be executed to be completed, and storing the context data to the main memory module; when the first execution instruction set is executed to be completed, obtaining a second type of instruction sequence of the NPU module, wherein the second type of instruction is a context switching related instruction; restoring the second type of instruction sequence in the NPU module to execute a target task; when the target task is executed to be completed, restoring the context data to the NPU module through the main memory module, and simultaneously resuming execution of the first type of queued instruction sequence.

[0005] The second aspect disclosed in the present application provides an NPU instruction-level context switching device, which includes: a software module, which is used to execute context switching process control; a hardware module, including: an NPU module, which is used to implement context switching; a main memory module, which is used to store context data; and a CPU module, which is used to assist the software module in executing context switching process control.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] In the first aspect, the present application provides an NPU instruction-level context switching method, which, after receiving a context switching instruction, obtains a first type of queued instruction sequence of the NPU module, a first execution instruction set that is still being executed, and related context data, thereby ensuring that the system can capture the precise state at the switching moment, and laying a foundation for the subsequent switching process; after obtaining the first type of queued instruction sequence, immediately prohibits its execution, ensuring that the NPU module will not continue to execute instructions that may affect the switching process, and at the same time, waits for the completion of the first execution instruction set, so that all related context data can be accurately stored in the main memory module later; when the first execution instruction set is executed, the second type of queued instruction sequence of the NPU module is obtained. The acquisition of these instruction sequences enables the system to accurately control the state and behavior of the NPU module during the switching process, ensuring the accuracy and efficiency of the switching process; the NPU module restores the second type of instruction sequence to execute the target task, ensuring that the system can quickly respond to the execution requirements of the new task after the switch, improving the overall response speed and efficiency of the system; when the target task is completed, the previously stored context data is restored to the NPU module through the main memory module, and the first type of queued instruction sequence is allowed to continue to execute, restoring the system configuration, ensuring that the system can be restored to the previous low-priority task execution state after completing the high-priority task, maintaining the overall stability and continuity of the system. The NPU instruction-level context switching method effectively manages and controls the execution state and data flow of the NPU module in a multi-tasking environment through precise software module execution steps, thereby improving the system's response speed, the accuracy of task switching, and the effective utilization of system resources.

[0008] On the second aspect, the present application further enhances the efficiency and resource utilization of NPU instruction-level context switching by introducing an address remapper. The address remapper realizes efficient hardware resource allocation and management of tasks in the NPU by dynamically adjusting the memory address of the DMA stream. During the context switching process, the address remapper can effectively manage and control the flow of data in the scratchpad, without the need to save and restore a large amount of workspace data during switching, thereby significantly reducing the context switching overhead and speeding up task switching. In this way, the system can divide and manage hardware resources more flexibly, ensuring that in a multi-tasking environment, the NPU can respond quickly and execute new tasks while maintaining data consistency and system stability. This optimization measure improves the performance of the system and provides a solid guarantee for the execution of complex computing tasks.

[0009] Thirdly, the NPU instruction-level context switching device optimizes the synergy between software and hardware, thereby realizing the efficient ability to process complex computing tasks in the system and the ability to flexibly respond to multi-tasking environments. It not only improves the system's performance and response speed, but also optimizes the utilization efficiency of system resources.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a NPU instruction-level context switching method flow provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of priority and / or criticality inversion blocking time suffered by NPU tasks at different preemption levels in an NPU instruction-level context switching method provided in an embodiment of the present application;

[0013] Figure 3 A schematic diagram of the structure of an NPU instruction-level context switching device provided in an embodiment of the present application;

[0014] Figure 4 An exemplary structural diagram of an NPU instruction-level context switching device architecture provided in an embodiment of the present application;

[0015] Figure 5 A schematic diagram of a default configuration path micro-architecture and matrix transmission process of a load-class instruction of an NPU instruction-level context switching device provided in an embodiment of the present application;

[0016] Figure 6 A schematic diagram of the micro-architecture of an address remapper of an NPU instruction-level context switching device provided in an embodiment of the present application.

[0017] Explanation of the accompanying drawings: software module 10, hardware module 20, task monitor 11, interrupt handler 12, task scheduler 13, memory management 14, task control module 15, NPU module 21, main memory module 22, CPU module 23, Gemmini-RT architecture 211, configuration copy buffer 212, data default configuration channel 213, address remapper 24. DETAILED DESCRIPTION

[0018] The embodiments of the present application provide an NPU instruction-level context switching method and device to solve the limitation problem of the existing NPU architecture in preemption capability, as well as the significant priority inversion and / or criticality inversion problem caused by the limitation.

[0019] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically introduced below in conjunction with the accompanying drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] Embodiment 1, as Figure 1 As shown, an embodiment of the present application provides an NPU instruction-level context switching method, which is applied to an NPU instruction-level context switching device, wherein the device includes a software module and a hardware module, wherein the hardware module includes an NPU module and a main memory module, and wherein the software module executes the following steps:

[0021] When a context switch instruction is received, a first type of queued instruction sequence, a first execution instruction set and context data of the NPU module are obtained, wherein the first type of queued instruction is a non-refresh instruction, and the first execution instruction set is the instruction still being executed when the context switch instruction is received.

[0022] An NPU instruction-level context switching method provided in an embodiment of the present application is applied to an NPU instruction-level context switching device, wherein the device includes a software module and a hardware module, the hardware module includes an NPU module and a main memory module, wherein the NPU module is responsible for graphics processing and context switching, and the main memory module is used to store context data.

[0023] like Figure 2 As shown in the figure, a schematic diagram of the priority and / or criticality inversion blocking time suffered by NPU tasks under different preemption levels is provided through a heterogeneous MCS example, where τ x (P x ,L x ) represents the task τ x The priority is P x , the criticality is L x , P x The smaller the value, the higher the priority. When the NPU is non-preemptive, tasks experience significant priority and / or criticality inversions because high priority / critical tasks must wait until the accelerator completes all ongoing computations before they can continue execution. When the NPU is designed to allow limited preemption, the context switching capability becomes coarse-grained. However, since preemption can only occur at the boundaries of the execution algorithm, the system still experiences considerable priority and / or criticality inversions. When the NPU is equipped with instruction-level context switching, the system's real-time responsiveness and task scheduling capabilities are greatly enhanced.

[0024] In certain situations, such as interrupt signals, task scheduling, etc., context switching is performed, and a context switching instruction is generated and sent. When a context switching instruction is received, the first type of queued instruction sequence, the first execution instruction set, and context data of the NPU module are obtained.

[0025] Among them, the first type of queued instructions are instructions currently waiting to be executed in the instruction queue of the NPU in sequence. They are non-refresh instructions, that is, they do not include instructions that need to clear or reset the NPU state, including load, store, preload, calculate and other instructions; the first execution instruction set is the instruction set that the NPU is executing at the moment the context switch instruction is received. These instructions being executed are recorded to ensure that they can continue to be executed after the context switch is completed; the context data includes calculation data, address data, configuration data, etc.

[0026] When receiving a context switch instruction, the current NPU instruction and context data are accurately obtained and saved so that important information is not lost when switching tasks, and the data can be restored when needed to ensure the stable operation of the system.

[0027] The execution of the first type queued instruction sequence is prohibited, and the execution of the first execution instruction set is waited for to be completed, and the context data is stored in the main memory module.

[0028] Suspend the continued queuing and execution of non-refresh instructions in the NPU queue. This can be achieved by setting a flag or control signal to make the NPU ignore the execution request of these instructions. Continuously monitor the execution status of the NPU to ensure that the instruction set being executed, that is, the first execution instruction set, can be fully executed. The execution status can be obtained through hardware registers or status monitors. When all instructions are executed, it can be confirmed that the NPU is in an idle state and context switching can be performed. After confirming that all execution instructions are completed, the collected context data is stored in a predetermined location in the main memory module, which ensures the integrity of the context data and its availability when restored.

[0029] When the first execution instruction set is executed, a second type of instruction sequence of the NPU module is obtained, wherein the second type of instructions are context switching related instructions.

[0030] After the first execution instruction set is completed, the second type of instruction sequence of the NPU module is obtained. These instructions are specific instructions related to context switching, including instructions such as saving the current context, loading a new context, and initializing a new task. The context switching related instructions are extracted through the task scheduler. These instructions are pre-defined and stored in the instruction buffer to ensure the correct execution of the context switching process.

[0031] The second type of instruction sequence is restored in the NPU module to execute the target task.

[0032] Load the previously acquired second type instruction sequence into the NPU module, and load these instructions one by one into the instruction queue of the NPU module through the instruction buffer or task scheduler, and ensure that they are executed in sequence. Load and execute the initial instructions of the target task to ensure that the new task can be started correctly, including initializing the task context, setting initial parameters, etc. Monitor the execution status of the target task in real time through the task monitor or task scheduler to ensure that the task is carried out as expected.

[0033] When the target task is executed, the context data is restored to the NPU module through the main memory module, and the execution of the first type queued instruction sequence is restored.

[0034] When the target task is completed, the previously stored context data is read from the main memory module, the context data is transferred back to the corresponding location in the NPU module, and the data is restored to the corresponding registers and memory locations one by one to ensure the complete recovery of the context data. At the same time, the previously paused first-type queued instruction sequence is reloaded into the instruction queue of the NPU module, the previously set prohibition execution flag is cleared, and the execution of these instructions is resumed.

[0035] In addition, the NPU also includes an address remapper module to accelerate context switching.

[0036] Given that not every task needs to use all NPU resources, in order to achieve more efficient resource utilization, the present invention introduces an address remapper. This enables effective hardware partitioning when local memory is sufficient, eliminating the need to save and restore work area data during context switching, thereby accelerating the context switching process. The address remapper relocates the DMA data streams entering the work area to the appropriate storage area by intercepting them and adding dynamic offsets. This ensures that from the outside, the data still looks like it is directed to the initial location in the work area. The present invention establishes a remapping block of configurable size in the work area to store the mapping from the original address to the actual address to facilitate address translation and data management. Specifically, the present invention assigns a semaphore to each storage area, called a "storage area lock", which is activated when valid data is loaded in the storage area. During the context switching process when a task is abnormal or a task is completed, the operating system identifies the storage area involved by querying the address mapping in the remapping block. It then deactivates the corresponding semaphore and clears the data in the storage area. When a DMA stream initiates a write request to the work area, the system intercepts the stream and applies a dynamic offset to the work area address bits before entering the write queue. This offset locates the data to a memory area that is partially filled and locked by the task, or to a memory area that is currently unlocked. The system then locks the memory area and records the start and end positions of the address in the reimage block. Similarly, when a DMA stream requests to read data from the work area, the system first queries the reimage block to find the corresponding mapping and then adjusts the work area address bits in the DMA stream. When the task is preempted and subsequently resumed, the reimage block adjusts the target address based on the newly allocated memory area entered by the task.

[0037] Furthermore, the hardware module also includes a CPU module, and the software module includes a real-time operating system with a task monitor, a task scheduler, and a fine-tuned interrupt handler. The task monitor monitors in real time whether the current task is completed and includes a timer to periodically check whether a new task has arrived, including:

[0038] When the system call signal passes through the task monitor, the CPU module responds to the interrupt and determines whether to generate the context switch instruction through the interrupt handler; when the current interrupt is triggered by the task monitor detecting the completion of the current task, the scheduler searches for the next task to be executed and determines whether to generate the context switch instruction based on whether it has been executed before; when the current interrupt is triggered by the periodic check of the timer in the task monitor, the scheduler determines whether the priority of the newly arrived task is higher than the current task, and determines whether to generate the context switch instruction based on this.

[0039] Specifically, when the current interrupt is triggered by the task monitor detecting the completion of the current task, the scheduler will first check the status of the current task and confirm that it has been fully executed. Subsequently, the scheduler starts looking for the next task to be executed. If the scheduler finds that the task has been partially executed before, it will determine whether a context switch instruction needs to be generated based on the task's status information and context data. If the context data of the task already exists and the current system resources allow the task to continue directly, only partial context switch instructions are generated, that is, the data in the NPU and accumulator are reconfigured without processing the calculation data in the scratch board, thereby reducing the switching overhead and speeding up the execution process of the task. If the task is a new task, the task will be initialized and executed directly without generating a context switch instruction. In other cases, that is, the task context data needs to be reloaded, the scheduler will generate a context switch instruction to ensure that the system can correctly switch to the execution environment of the new task.

[0040] When the current interrupt is triggered by a periodic check of the timer in the task monitor, the scheduler first evaluates the priority of the newly arrived task. The scheduler compares the priority of the new task with the priority of the currently executing task. If the priority of the newly arrived task is higher than the current task, the scheduler generates a context switch instruction to immediately terminate the execution of the current low-priority task and schedule system resources to the new task with a higher priority to ensure timely response of the high-priority task. If the priority of the new task is lower than or equal to the current task, the scheduler may choose to continue executing the current task without generating a context switch instruction.

[0041] The CPU module receives the interrupt signal from the task monitor and calls the interrupt handler, which goes to the task scheduler. The scheduler evaluates whether a context switch is needed based on the execution status, priority, and system resource usage of the current task. The criteria of the mixed criticality system theoretical model are adopted here. Specifically, mixed criticality systems (MCS) are safety-critical systems in which functions of different criticality levels are integrated on a shared platform. These systems are very common in practical applications. For example, in the automotive industry, advanced driver assistance systems (ADAS) may involve multiple functions of different criticality levels, where collision avoidance is of high criticality and route planning may be of low criticality. In order to meet the computing needs of diverse functions in MCSs, such as workloads running deep neural networks (DNNs), hardware vendors have developed system-on-chips with different architectures. These system-on-chips are highly heterogeneous, that is, the system-on-chip combines general-purpose CPUs with heterogeneous coprocessors to accelerate algorithm execution.

[0042] Traditional theoretical models of mixed-criticality systems usually assume that the worst-case execution time (WCET) of tasks is estimated with different degrees of confidence. The high-criticality WCET (HI-WCET) of a task is associated with high confidence but is often too pessimistic, while the low-criticality WCET (LO-WCET) is less pessimistic but has lower confidence. The correctness criterion states that if all tasks complete execution within their LO-WCET, then they will all complete before their deadline. However, if the execution time of any high-criticality task (HI-task) exceeds its LO-WCET, the high-criticality task must complete execution before its deadline.

[0043] To satisfy the above criteria, the mode switching strategy is a straightforward one: the system initially runs in low-criticality mode (LO-mode), assuming that the execution time of all tasks will not exceed their LO-WCET. If this assumption is violated, i.e., if any task fails to complete before its LO-WCET, the system switches to high-criticality mode (HI-mode). In HI-mode, the scheduling policy may allow high-criticality tasks (HI-tasks) to execute longer than their LO-WCET, but ensure that their execution time does not exceed their HI-WCET. Therefore, to ensure that high-criticality tasks are completed on time, low-criticality tasks (LO-tasks) may need to be terminated or executed within a very small time budget.

[0044] Furthermore, the software module further includes a task control module, and the NPU module further includes a data default configuration channel, which stores the context data in a main memory module, including:

[0045] The context data includes calculation data, address data and configuration data; the calculation data is transmitted to the main memory module through the data default configuration channel; the configuration data is transmitted to the main memory module; and the address data is stored in the task control module.

[0046] The task control module is part of the software module and is responsible for managing task interruption, recovery, and execution control. It can work with the task monitor and task scheduler to ensure that tasks are scheduled and managed according to priority and execution requirements. The data default configuration channel is part of the NPU module and is used to transfer configuration information and context data between the NPU and the main memory module. It can efficiently handle data transmission, reduce transmission delays, and increase data transmission speed.

[0047] Computational data refers to the input data in the initial state of the task and the intermediate results and calculation information generated during the task execution, including the calculation results of a certain operator, etc. Address data refers to the memory address information that needs to be accessed when the task is executed, including instruction addresses, data addresses, etc. Configuration data refers to the data information used to configure the working status and parameters of the NPU, including register configuration, cache configuration, etc.

[0048] The computing data to be transmitted is packaged into data packets to ensure data integrity. The packaged computing data is transmitted from the NPU module to the main memory module through the data default configuration channel to ensure that data is not lost when the task is switched.

[0049] The configuration data needs to be saved during task execution so that it can be reloaded when the task is resumed. During context switching, the NPU module collects all configuration data of the current task, packages the collected configuration data into a standardized data packet, starts the transmission of the configuration data, and sends the packaged configuration data to the main memory module through the data default configuration channel.

[0050] The address data needs to be saved when the task is switched so that it can continue to execute when the task is restored. When the context is switched, the NPU module collects all the address data of the current task, packages the collected address data into a standardized data packet, starts the storage process of the address data, and stores the packaged address data in the task control module.

[0051] By storing context data, the system can efficiently manage and save key data during task context switching, thereby improving system performance and reliability.

[0052] Furthermore, the NPU module includes configuring a copy buffer, including:

[0053] The configuration copy buffer stores different categories of previously executed configuration instructions of the NPU module.

[0054] The configuration copy buffer is a dedicated buffer in the NPU module, which is used to store different categories of configuration instructions previously executed by the NPU module. The configuration copy buffer can temporarily store configuration information to ensure that these instructions can be quickly accessed and loaded during context switching and task recovery.

[0055] Configuration instructions are used to set and adjust the working status of the NPU, including register configuration, cache configuration, computing unit configuration, etc. Configuration instructions need to be saved and restored during the context switch process to ensure that the task can continue to execute correctly after the switch.

[0056] When a task is executed on the NPU, all configuration instructions of the current task are collected and stored in the configuration copy buffer for quick access during context switching. When the task context is switched, the configuration instructions in the configuration copy buffer are transferred to the main memory module to ensure that these instructions can be reloaded when the task is restored. When the task is restored from the context switch, the configuration instructions are obtained from the main memory module and reloaded into the configuration copy buffer. In this way, the NPU module can continue to execute the task according to the previous configuration state, ensuring data consistency and ensuring that the task can be restored seamlessly.

[0057] By configuring the copy buffer, configuration instructions can be efficiently managed and stored to ensure that these instructions can be quickly accessed and loaded during task context switching and recovery. This method optimizes the context switching process and improves system performance and reliability.

[0058] Furthermore, storing the context data in a main memory module includes:

[0059] The context data is stored in the main memory module through pipeline technology.

[0060] Pipeline technology is a method to improve data processing efficiency. It breaks down complex data processing processes into multiple stages. Each stage performs a specific task, and each stage can work in parallel with other stages to improve overall processing speed and efficiency. It is commonly used in processor design to increase instruction execution speed.

[0061] The context data is stored through pipeline technology. Specifically, the context data storage process is decomposed into multiple stages, such as data collection, data packaging, data transmission and data storage. Each stage is executed independently and can work in parallel with other stages.

[0062] During task execution, the NPU module collects all context data of the current task and classifies the collected context data by type. The classified context data is packaged into standardized data packets for transmission and storage. The data transmission process is started through the data default configuration channel, the packaged context data is transferred to the main memory module, and the data packets transferred to the main memory module are written to the specified storage location to ensure that the data can be quickly accessed when the task is resumed.

[0063] By breaking down data collection, packaging, transmission, and storage into multiple independent stages, pipeline technology can improve the efficiency and speed of data storage and ensure the integrity and rapid storage of data during context switching. This approach optimizes the management of context data and improves system performance and reliability.

[0064] Further, when the first execution instruction set is executed, a second type of instruction sequence of the NPU module is obtained, the software module includes a task scheduler, and the NPU module includes a configuration copy buffer, which includes:

[0065] Refresh the NPU instruction queue through the task scheduler to determine whether the non-configuration copy buffer has instructions; if not, obtain the second type of instruction sequence of the NPU module; if yes, repeatedly execute the prohibition instruction and wait instruction process.

[0066] The task scheduler is responsible for managing the instruction queue of the NPU. During the context switch process, the task scheduler refreshes the current instruction queue to ensure that the latest instruction status is correctly recorded. After refreshing the instruction queue, it checks whether there are instructions waiting to be executed in the non-configured copy buffer, that is, the ordinary instruction queue.

[0067] If there are no queued instructions in the non-configuration copy buffer, the task scheduler obtains a second type of instruction sequence from the NPU module, which instructions are related to context switching, such as instructions for saving and restoring context data.

[0068] If there are still queued instructions in the non-configured copy buffer, continue to prohibit the execution of these instructions to ensure that the context data of the current task will not be affected by the new instructions. The task scheduler repeatedly checks and waits for the execution process of these instructions until all queued instructions are executed.

[0069] Through these steps, the context switching process can be efficiently managed and executed to ensure the integrity of context data and the correct execution of tasks. This method improves the performance and reliability of the system and optimizes the context switching process.

[0070] In summary, the NPU instruction-level context switching method provided in the embodiment of the present application has the following technical effects:

[0071] After receiving the context switch instruction, the first type of queued instruction sequence of the NPU module, the first execution instruction set that is still being executed, and the related context data are obtained, ensuring that the system can capture the precise state at the switching moment, laying the foundation for the subsequent switching process; after obtaining the first type of queued instruction sequence, its execution is immediately prohibited to ensure that the NPU module will not continue to execute instructions that may affect the switching process, and at the same time, wait for the completion of the first execution instruction set, so that all related context data can be accurately stored in the main memory module later; when the first execution instruction set is executed, the second type of instruction sequence of the NPU module is obtained, and the acquisition of these instruction sequences enables the system to accurately control the state and behavior of the NPU module during the switching process, ensuring the accuracy and efficiency of the switching process; the second type of instruction sequence is restored in the NPU module to execute the target task, ensuring that the system can quickly respond to the execution requirements of the new task after the switch, and improving the overall response speed and efficiency of the system; when the target task is executed, the previously stored context data is restored to the NPU module through the main memory module, and the first type of queued instruction sequence is allowed to continue to execute, ensuring that the system can restore to the previous low-priority task execution state after completing the high-priority task, maintaining the overall stability and continuity of the system. The NPU instruction-level context switching method effectively manages and controls the execution state and data flow of the NPU module in a multi-tasking environment through precise software module execution steps, thereby improving the system response speed, task switching accuracy and effective utilization of system resources.

[0072] Embodiment 2, as Figure 3 As shown, an embodiment of the present application provides an NPU instruction-level context switching device, the device comprising:

[0073] A software module 10, the software module 10 is used to execute context switching process control; a hardware module 20, including: an NPU module 21, the NPU module is used to implement context switching; a main memory module 22, the main memory module is used to store context data; a CPU module 23, the CPU module is used to assist the software module 10 in executing context switching process control.

[0074] An NPU instruction-level context switching device provided in an embodiment of the present application mainly includes a software module 10 and a hardware module 20, wherein the software module 10 includes a task monitor, a task scheduler and an interrupt handler, which is used to execute context switching process control, and is responsible for managing and coordinating the software logic in the entire context switching process, including issuing context switching instructions, monitoring instruction execution status, and coordinating the operation of the hardware module when necessary. The specific operation steps refer to the corresponding part of the first embodiment.

[0075] The hardware module 20 includes an NPU module 21, a main memory module 22, and a CPU module 23. The NPU module 21 is used to implement specific context switching operations. After receiving the context switching instruction issued by the software module, it is responsible for saving the context data of the current task and restoring the context data of the new task when needed. The NPU module performs these operations through a hardware instruction set to ensure efficient and fast context switching.

[0076] The main memory module 22 is used to store context data to ensure the security and reliability of data during the task switching process; the CPU module 23 is used to assist the software module in executing context switching process control. The CPU module 23 supports the processing power and data transmission required by the software module when executing context switching through the processor's computing power and system bus interface. The CPU module 23 is responsible for interrupt processing, system call response, and coordinated communication with the NPU module during the context switching process.

[0077] like Figure 4 As shown, an exemplary structure of an NPU instruction-level context switching device architecture is provided, wherein the modules such as the scratch plate not described in this embodiment are existing modules and are omitted for the sake of brevity of the specification.

[0078] like Figure 5 As shown, a default configuration path microarchitecture of a load class instruction in an NPU instruction-level context switch device architecture is provided (as shown in the black part on the left), and a process of transferring matrices to or from a temporary storage area and an accumulator using step-in and step-out instructions (on the right). In Gemmini-RT, matrix processing units (such as transposers, shifters) exist as functional modules that terminate after completing their instructions and do not store data. Similarly, the systolic array can also be regarded as a dedicated computing unit. It performs operations according to instructions and stores intermediate results in the accumulator after completing a single calculation. Subsequent instructions retrieve values ​​from the accumulator instead of relying on the registers of the pipeline array. Therefore, all calculation data is stored only in temporary storage and accumulators. Data and memory management between Gemmini-RT and the main CPU is explicit, which means that data must be explicitly moved between the main address space of the processor and the private address space of the NPU through a series of move instructions. However, configuration instructions may affect these move instructions, that is, the data saved and restored under certain configurations may be incomplete. Therefore, to ensure the consistency of the configuration and to fully retrieve the computed data safely, we introduce new move instructions such as Figure 5 These new instructions take advantage of the data default configuration channels we developed in the NPU control unit (such as Figure 5As shown on the left, it is possible to completely save and restore the compute data without changing the current accelerator configuration environment. Specifically, during the context save process, by using these move instructions and recording the target main memory address before data transfer, we can ensure the complete preservation of the compute data in the main memory and manage it through the scheduling of the operating system. Similarly, during the context restore process, by retrieving the main memory address index of the compute data, we can completely load it back to the NPU. Through these processes, we can completely save the compute data and restore it at any time as needed.

[0079] This NPU instruction-level context switching device can optimize system resource utilization and task execution efficiency in modern systems and improve system reliability.

[0080] Furthermore, the NPU module 21 includes:

[0081] Gemmini-RT architecture 211; configuration copy buffer 212, the configuration copy buffer 212 is used to store NPU module configuration instructions; data default configuration channel 213, the data default configuration channel 213 is used to transmit context data.

[0082] The Gemmini-RT architecture 211 is an NPU microarchitecture whose key architecture is based on a systolic array of multiple units, each of which consists of a configurable number of processing units, and a set of local memories, including scratch pads and accumulators. Gemmini-RT is designed to work with CPU cores by executing customized, well-defined instructions. It is commonly used in systems and small computing devices. It optimizes the processing speed and energy efficiency of matrix operations and is particularly suitable for deep learning inference and other applications that require large-scale matrix calculations. In Gemmini-RT, configuration instructions are unique compared to other types of instructions. First, configuration instructions are divided into four types: load, store, execute, and norm, where subsequent configuration instructions of the same category can overwrite the previous one. Second, configuration instructions do not need to be issued, but take effect immediately when they are received at the reservation station, directly affecting the corresponding registers to achieve the configuration purpose and complete execution within two cycles.

[0083] The configuration copy buffer 212 is used to store configuration instructions of the NPU module. During the context switching process, these configuration instructions involve the state switching, function setting or other parameter adjustment of the NPU. By configuring the copy buffer, these configurations can be quickly loaded and applied to support the execution of new tasks after the context switching.

[0084] The data default configuration channel 213 is responsible for transmitting context data between the NPU module and the main memory module. These data include the running status of the current task, register contents or other information related to task execution. The data default configuration channel provides efficient data transmission capabilities to ensure fast transmission and correct loading of data during context switching.

[0085] Furthermore, the software module 10 includes:

[0086] A task monitor 11, the task monitor 11 is used to monitor interrupt signals through a timer; an interrupt handler 12, the interrupt handler 12 is used to process non-timer interrupt signals; a task scheduler 13, the task scheduler 13 is used to perform context switching process control; a memory manager 14, the memory manager 14 is used to manage the storage and recovery of context data; a task control module 15, the task control module 15 is used to control the interruption and execution of tasks, and store NPU instruction address data.

[0087] The task monitor 11 is used to monitor interrupt signals through a timer. Specifically, the task monitor 11 periodically checks the timer in the system to detect whether there are signals or events that require context switching, such as task timeout, resource competition or other conditions that require system response.

[0088] The interrupt handler 12 is used to process non-timer interrupt signals. Specifically, the interrupt handler 12 is responsible for responding to non-timer interrupt signals from outside or inside the system, such as hardware errors, user input events, etc. In the context switching device, the interrupt handler 12 needs to ensure that it can correctly handle the interrupt when it occurs and decide whether to trigger the context switch.

[0089] The task scheduler 13 is used to perform context switching process control. Specifically, the task scheduler 13 decides when to perform context switching based on the priority, status and availability of system resources of the task. It is responsible for scheduling and managing the execution order of tasks to maximize the efficiency and responsiveness of the system.

[0090] Memory management 14 is used to manage the storage and recovery of context data. Specifically, memory management 14 is responsible for allocating, releasing and managing the space in the main memory module used to store context data. It needs to ensure that the task status data can be efficiently saved and loaded during context switching, and clean and organize the data when necessary.

[0091] The task control module 15 is used to control the interruption and execution of tasks and store NPU instruction address data. Specifically, the task control module 15 ensures that the task can run normally and respond to the system scheduling during the execution process by managing the execution status of the task, interrupt response and storage of NPU instruction address data. It involves interaction with hardware modules, such as transmitting the address data of NPU instructions to support the operation and control of the NPU module.

[0092] Furthermore, the hardware module 20 includes:

[0093] The address remapper 24 is used to dynamically adjust the memory address of the DMA stream to achieve efficient hardware resource allocation and management of tasks in the NPU.

[0094] Given that not every task needs to use all NPU resources, in order to achieve more efficient resource utilization, this application introduces an address remapper. This allows effective hardware partitioning when local memory is sufficient, eliminating the need to save and restore workspace data during context switching, thereby speeding up the context switching process. Figure 6 As shown, a microarchitecture schematic diagram of the address remapper is provided, blue line: path of the read signal; gray line: path of the write signal; black line: general path. The address remapper relocates the DMA data streams entering the work area by intercepting them and adding dynamic offsets. This ensures that from the outside, the data still looks like it is directed to the initial position in the work area. The present invention establishes a remapping block of configurable size in the work area to store the mapping from the original address to the actual address to facilitate address translation and data management. Specifically, the present invention allocates a semaphore to each storage area, called a "storage area lock", which is activated when valid data is loaded in the storage area. During the context switch process when the task is abnormal or the task is completed, the operating system identifies the storage area involved by querying the address mapping in the remapping block. Then, it deactivates the corresponding semaphore and clears the data in the storage area. When the DMA stream initiates a write request to the work area, the system intercepts the stream and applies a dynamic offset to the work area address bit before entering the write queue. This offset locates the data to a memory area that was partially filled and locked by the task, or to a memory area that is currently unlocked. The system then locks the memory area and records the start and end locations of the address in the reimage block. Similarly, when a DMA stream requests to read data from the working area, the system first queries the reimage block to find the corresponding mapping and then adjusts the working area address bits in the DMA stream. When the task is preempted and subsequently resumed, the reimage block adjusts the target address based on the newly allocated memory area that the task entered.

[0095] These software module components work together to implement complex task management and scheduling functions in the NPU instruction-level context switching device, thereby improving the performance, stability and reliability of the system.

[0096] In summary, the NPU instruction-level context switching device achieves the efficient ability to process complex computing tasks in the system and the ability to flexibly cope with multi-tasking environments by optimizing the synergy between software and hardware. It not only improves the system's performance and response speed, but also optimizes the utilization efficiency of system resources.

[0097] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A NPU instruction-level context switching method, characterized in that: Applied to an NPU instruction-level context switching device, the device includes a software module and a hardware module, the hardware module includes an NPU module and a main memory module, and the software module executes the steps including: When a context switch instruction is received, a first type of queued instruction sequence, a first execution instruction set and context data of the NPU module are obtained, wherein the first type of queued instruction is a non-refresh instruction, and the first execution instruction set is an instruction that is still being executed when the context switch instruction is received; prohibiting execution of the first type of queued instruction sequence, waiting for the first execution instruction set to be executed, and storing the context data in a main memory module; When the first execution instruction set is executed, a second type of instruction sequence of the NPU module is obtained, wherein the second type of instruction is context switch related instruction; Recover the second type of instruction sequence in the NPU module to execute the target task; When the target task is executed, the context data is restored to the NPU module through the main memory module, and the execution of the first type queued instruction sequence is restored.

2. A NPU instruction level context switching method as claimed in claim 1, characterized in that: The hardware module also includes a CPU module, and the software module includes a real-time operating system with a task monitor, a task scheduler, and a fine-tuned interrupt handler. The task monitor monitors in real time whether the current task is completed and includes a timer to periodically check whether a new task has arrived, including: When the system call signal passes through the task monitor, the CPU module responds to the interrupt and determines whether to generate the context switch instruction through the interrupt processing program; When the current interrupt is triggered by the task monitor detecting the completion of the current task, the scheduler searches for the next task to be executed and determines whether to generate the context switch instruction based on whether the task has been executed before; When the current interrupt is triggered by a periodic check of a timer in the task monitor, the scheduler determines whether the priority of the newly arrived task is higher than that of the current task, and determines whether to generate the context switch instruction based on this.

3. The NPU instruction level context switching method as claimed in claim 1, characterized in that: The software module further includes a task control module, and the NPU module further includes a data default configuration channel, which stores the context data in the main memory module, including: The context data includes calculation data, address data and configuration data; Transmitting the calculation data to the main memory module through a data default configuration channel; transferring the configuration data to the main memory module; The address data is stored in the task control module.

4. A NPU instruction level context switching method as claimed in claim 3, characterized in that: The NPU module includes configuring a copy buffer, including: The configuration copy buffer stores different categories of previously executed configuration instructions of the NPU module.

5. The NPU instruction level context switching method as claimed in claim 1, characterized in that: Storing the context data in a main memory module includes: The context data is stored in the main memory module through pipeline technology.

6. The NPU instruction level context switching method according to claim 1, characterized in that: When the first execution instruction set is executed, a second type instruction sequence of the NPU module is obtained, the software module includes a task scheduler, and the NPU module includes a configuration copy buffer, which includes: Refresh the NPU instruction queue through the task scheduler to determine whether the copy buffer has instructions; If not, obtaining a second type of instruction sequence of the NPU module; If yes, repeat the prohibition instruction and the waiting instruction process.

7. A NPU instruction level context switching device, characterized in that: include: A software module, the software module being used to perform context switching process control; Hardware modules, including: An NPU module, wherein the NPU module is used to implement context switching; A main memory module, the main memory module is used to store context data; A CPU module is used to execute a context switch control flow of a software module.

8. The NPU instruction level context switching device as claimed in claim 7, characterized in that: The NPU module includes: Gemmini-RT architecture; A configuration copy buffer, wherein the configuration copy buffer is used to store NPU module configuration instructions; A data default configuration channel is used to transmit context data.

9. The NPU instruction level context switching device as claimed in claim 7, characterized in that: The software modules include: A task monitor, wherein the task monitor is used to monitor an interrupt signal through a timer; An interrupt processing program, wherein the interrupt processing program is used to process non-timer interrupt signals; A task scheduler, the task scheduler is used to perform context switching process control; Memory management, which is used to manage the storage and retrieval of context data; The task control module is used to control the interruption and execution of tasks and store NPU instruction address data.

10. The NPU instruction level context switching device according to claim 7, characterized in that: The hardware module includes: The address remapper is used to dynamically adjust the memory address of the DMA stream to achieve efficient hardware resource allocation and management of tasks in the NPU.

Images