Server, task execution method and device, and nonvolatile readable storage medium

Abstract

The present application relates to the technical field of computers, and discloses a server, a task execution method and device, and a nonvolatile readable storage medium. The server comprises a processor, a processor memory and an accelerator connected to the processor, the accelerator comprising an accelerator memory, and the processor memory and the accelerator memory being uniformly addressed as a unified memory by means of a compute express link protocol; the processor is configured to acquire raw data of a target task, store the raw data into the unified memory, write instruction data into a register of the accelerator on the basis of the target task, and acquire a task execution result of the target task from the unified memory; and the accelerator is configured to generate an instruction on the basis of the instruction data written in the register, acquire the raw data from the unified memory by means of the compute express link protocol, execute the instruction on the basis of the raw data to complete the target task, and write the task execution result into the unified memory by means of the compute express link protocol. The present application reduces the delay and energy consumption of task execution.

Description

A server, a task execution method, a device and a non-volatile readable storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on February 27, 2024, with application number 202410214089.5, entitled “A Server and a Task Execution Method, Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular, to a server and a task execution method, device, and non-volatile readable storage medium. Background Art

[0004] Currently, mainstream computing approaches can be categorized as centralized data centers and distributed edge computing. Because edge computing applications are initiated at the edge, they deliver faster service responses, meeting consumers' fundamental needs for real-time services, application intelligence, security, and privacy. As edge computing tasks become increasingly complex and diverse, local accelerators are being deployed in edge computing servers to accelerate task execution. During task execution, data is copied multiple times between processor memory and accelerator memory, resulting in higher latency and power consumption.

[0005] Therefore, there are technical problems in the related art such as delay in task execution and high power consumption. Summary of the Invention

[0006] The purpose of this application is to provide a server and a task execution method, device and non-volatile readable storage medium, which reduce the delay and power consumption of task execution.

[0007] To achieve the above objectives, the present application provides a server comprising a processor, a processor memory, and an accelerator connected to the processor, wherein the accelerator includes an accelerator memory, and the processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol;

[0008] The processor is configured to obtain raw data of a target task, store the raw data in a unified memory, write instruction data to a register of the accelerator based on the target task, and obtain a task execution result of the target task from the unified memory;

[0009] The accelerator is configured to generate instructions based on the instruction data written in the register, obtain the original data from the unified memory through the computing high-speed interconnection protocol, execute instructions based on the original data to complete the target task, and write the task execution results into the unified memory through the computing high-speed interconnection protocol.

[0010] Among them, the accelerator includes a computing high-speed interconnect core, an instruction generation module, and an accelerator core;

[0011] A computing high-speed interconnect core configured to communicate with the processor;

[0012] an instruction generation module, configured to generate instructions according to instruction data written in the register and send the instructions to the accelerator core;

[0013] The accelerator core is configured to obtain raw data from the unified memory through a computing high-speed interconnection protocol, execute instructions based on the raw data to complete the target task, and write the task execution results into the unified memory through a computing high-speed interconnection protocol.

[0014] The computing high-speed interconnect core includes a first interface and a second interface. The processor writes data to the register of the accelerator through the first interface, and the accelerator accesses the unified memory through the second interface.

[0015] Among them, the instruction generation module includes a register stack submodule, a state machine corresponding to instructions of different instruction types, and an instruction processing submodule;

[0016] The register file submodule includes a plurality of instruction data registers corresponding to different state machines, and is configured to store instruction data corresponding to different instruction types;

[0017] The state machine is configured to generate an instruction of a corresponding instruction type according to instruction data in a corresponding instruction data register, and send the instruction to an instruction processing submodule;

[0018] The instruction processing submodule is configured to process the instruction and then send it to the accelerator core.

[0019] The instruction processing submodule is configured to sort the instructions generated by different state machines according to their startup timestamps, and send the instructions generated by different state machines to the accelerator core in sequence according to the sorting results.

[0020] The instruction processing submodule is further configured to reset the target state machine if the difference between the current timestamp and the startup timestamp of the target state machine is greater than a threshold.

[0021] Among them, the register stack submodule also includes a start signal register and a stop signal register corresponding to different state machines. The start signal register is set to describe whether the state machine is started, and the stop signal register is set to describe whether the instruction data is written.

[0022] When the state machine is in an idle state, the corresponding start signal register is a first preset value. When it is detected that the processor sets the start signal register to a second preset value and the timeout reset signal is the first preset value, the state machine jumps to a waiting information filling state and updates the start timestamp.

[0023] When the state machine is in the waiting information filling state, it waits for the processor to fill the instruction data into the corresponding instruction data register. When it is detected that the processor sets the corresponding stop signal register to the second preset value and the timeout reset signal is not the first preset value, the state machine jumps to the generating command state;

[0024] When the state machine is in the command generation state, it generates an instruction based on the instruction data in the corresponding instruction data register and sends it to the instruction processing sub-module, and resets the corresponding instruction data register, start signal register, and stop signal register to the first preset value, and the state machine jumps to the idle state.

[0025] Among them, instructions of different instruction types include load instructions, store instructions and execute instructions. Load instructions are used to load data from the unified memory to the cache of the accelerator core, store instructions are used to store data from the cache of the accelerator core to the unified memory, and execute instructions are used to execute operations.

[0026] The instruction data includes a physical address of the original data, and the accelerator is configured to obtain the original data from the unified memory based on the physical address.

[0027] The accelerator is an accelerator implemented based on a field programmable gate array.

[0028] The processor is connected to the accelerator via a high-speed serial computer expansion bus standard protocol interface.

[0029] To achieve the above objectives, the present application provides a task execution method, which is applied to a processor in a server, wherein the server includes a processor, a processor memory, and an accelerator connected to the processor, wherein the accelerator includes an accelerator memory, and the processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol; the method includes:

[0030] Obtain the original data of the target task and store the original data in the unified memory;

[0031] Writing instruction data into the register of the accelerator based on the target task so that the accelerator generates instructions based on the instruction data written in the register, obtains original data from the unified memory through the computing high-speed interconnect protocol, executes the instructions based on the original data to complete the target task, and writes the task execution results into the unified memory through the computing high-speed interconnect protocol;

[0032] Get the task execution results from the unified memory.

[0033] The instruction data includes a physical address of the original data, and the accelerator is configured to obtain the original data from the unified memory based on the physical address.

[0034] Among them, the target tasks include neural network reasoning tasks.

[0035] To achieve the above objectives, the present application provides a task execution method, which is applied to an accelerator in a server. The server includes a processor, a processor memory, and an accelerator connected to the processor. The accelerator includes an accelerator memory. The processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol. The method includes:

[0036] Generate instructions based on instruction data written by the processor into the register;

[0037] Obtain raw data from the unified memory through computing high-speed interconnection protocols, and execute instructions based on the raw data to complete the target task;

[0038] The task execution results are written into the unified memory through the computing high-speed interconnection protocol.

[0039] The step of generating an instruction according to instruction data written by the processor into the register includes:

[0040] Utilizing state machines corresponding to instructions of different instruction types to generate instructions of corresponding instruction types according to instruction data in corresponding instruction data registers;

[0041] Sorting the instructions generated by different state machines according to the startup timestamps of different state machines, and sending the instructions generated by different state machines to the accelerator core for execution in sequence according to the sorting results;

[0042] If the difference between the current timestamp and the start timestamp of the target state machine is greater than a threshold, the target state machine is reset.

[0043] To achieve the above objectives, the present application provides a task execution device, which is applied to a processor in a server. The server includes a processor, a processor memory, and an accelerator connected to the processor. The accelerator includes an accelerator memory. The processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol. The device includes:

[0044] The storage unit is configured to obtain the raw data of the target task and store the raw data into the unified memory;

[0045] a first writing unit configured to write instruction data into a register of the accelerator based on a target task, so that the accelerator generates instructions according to the instruction data written in the register, obtains original data from the unified memory through a computing high-speed interconnect protocol, executes the instructions based on the original data to complete the target task, and writes the task execution result into the unified memory through the computing high-speed interconnect protocol;

[0046] The acquisition unit is configured to obtain the task execution results from the unified memory.

[0047] To achieve the above objectives, the present application provides a task execution device for use with an accelerator in a server. The server includes a processor, processor memory, and an accelerator connected to the processor. The accelerator includes an accelerator memory. The processor memory and the accelerator memory are uniformly addressed as a unified memory via a computing high-speed interconnection protocol. The device includes:

[0048] a generation unit configured to generate instructions based on instruction data written into the register by the processor;

[0049] The execution unit is configured to obtain raw data from the unified memory through a computing high-speed interconnection protocol and execute instructions based on the raw data to complete the target task;

[0050] The second writing unit is configured to write the task execution result into the unified memory through a computing high-speed interconnection protocol.

[0051] To achieve the above objectives, the present application provides a computer non-volatile readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the task execution method as described above are implemented.

[0052] From the above scheme, it can be seen that a server provided by this application includes a processor, a processor memory, and an accelerator connected to the processor, the accelerator includes an accelerator memory, and the processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol; the processor is configured to obtain the original data of the target task, store the original data in the unified memory, write instruction data to the register of the accelerator based on the target task, and obtain the task execution result of the target task from the unified memory; the accelerator is configured to generate instructions according to the instruction data written in the register, obtain the original data from the unified memory through a computing high-speed interconnection protocol, execute the instructions based on the original data to complete the target task, and write the task execution result into the unified memory through a computing high-speed interconnection protocol.

[0053] In this application, the processor memory and accelerator memory are uniformly addressed based on the CXL (Compute Express Link) protocol, enabling direct access by the processor and accelerator to the uniformly addressed memory space, reducing the number of times data is copied between the processor memory and the accelerator memory, and reducing the delay and energy consumption of task execution. In addition, the processor writes instruction data to the accelerator's register, and the accelerator generates instructions based on the instruction data written in the register, thereby realizing the conversion between the processor configuration register information and the accelerator command, providing the processor with effective control logic for the accelerator, and realizing independent deployment of the accelerator for a compatible CXL server host. This application also discloses a task execution device, an electronic device, and a computer non-volatile readable storage medium, which can also achieve the above-mentioned technical effects.

[0054] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings are used to provide a further understanding of the present disclosure and constitute part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure, but do not constitute a limitation of the present disclosure. In the drawings:

[0056] Figure 1 is a structural diagram of a server based on GPU acceleration;

[0057] FIG2 is a structural diagram of a server according to an exemplary embodiment;

[0058] FIG3 is a structural diagram of an accelerator according to an exemplary embodiment;

[0059] FIG4 is a structural diagram of an instruction generation module according to an exemplary embodiment;

[0060] FIG5 is a structural diagram of an accelerator core according to an exemplary embodiment;

[0061] FIG6 is a flowchart showing a task execution method according to an exemplary embodiment;

[0062] FIG7 is a flowchart showing another task execution method according to an exemplary embodiment;

[0063] FIG8 is a structural diagram of a task execution device according to an exemplary embodiment;

[0064] Fig. 9 is a structural diagram of a task execution device according to an exemplary embodiment. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0066] In related technologies, the tasks performed by the edge computing server are accelerated by the GPU (Graphics Processing Unit), and the CPU (Central Processing Unit) is connected to the high-speed RDMA (Remote Direct Memory Access) network card, processor memory, and GPU through the PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) bus.

[0067] Figure 1 shows a server based on GPU acceleration. The CPU receives the raw data required to execute tasks through a high-speed RDMA network card and stores it in the processor memory. The GPU reads the raw data from the processor memory through H2D (Host-to-Device) and stores it in the GPU memory. The GPU acceleration core reads the raw data from the GPU memory for calculation and writes the result data back to the GPU memory. The CPU reads the result data from the GPU memory through D2H (Device-to-Host) and writes it back to the processor memory. It also reads the result data from the processor memory through the high-speed RDMA network card and sends it to the consumer.

[0068] Both the processor and GPU memory are off-chip DRAM (Dynamic Random Access Memory). In this solution, data is copied multiple times between the processor and GPU memory, and thus multiple times between off-chip DRAM. This results in high task execution latency and power consumption.

[0069] Therefore, in this application, processor memory and accelerator memory are uniformly addressed based on the CXL protocol, enabling direct access by the processor and accelerator to this uniformly addressed memory space. This reduces the number of data copies between the processor and accelerator memory, thereby reducing task execution latency and energy consumption. Furthermore, the processor writes instruction data to the accelerator's registers, and the accelerator generates instructions based on the instruction data written in the registers. This enables the conversion between processor configuration register information and accelerator commands, provides effective control logic for the processor over the accelerator, and enables independent deployment of the accelerator on CXL-compatible server hosts.

[0070] The embodiment of the present application discloses a server, as shown in FIG2 , including a processor, a processor memory, and an accelerator connected to the processor, the accelerator including the accelerator memory, and the processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol;

[0071] The processor is configured to obtain raw data of a target task, store the raw data in a unified memory, write instruction data to a register of the accelerator based on the target task, and obtain a task execution result of the target task from the unified memory;

[0072] The accelerator is configured to generate instructions based on the instruction data written in the register, obtain the original data from the unified memory through the computing high-speed interconnection protocol, execute instructions based on the original data to complete the target task, and write the task execution results into the unified memory through the computing high-speed interconnection protocol.

[0073] The server in this embodiment can be an edge computing server, including a processor and an accelerator, and the processor can be connected to the accelerator through a high-speed serial computer expansion bus standard protocol interface. The accelerator can be an accelerator based on a field programmable gate array (Field-Programmable Gate Array). The server includes a processor memory, and the accelerator includes an accelerator memory. The processor memory and the accelerator memory are uniformly addressed as a unified memory through the CXL protocol. The CXL protocol is a new type of interconnection protocol for CPUs and various types of dedicated accelerators to achieve efficient and stable memory access between the host and the device, and support cache consistency.

[0074] The CXL standard defines three sub-protocols: CXL.io, CXL.cache, and CXL.mem. Different devices support different sub-protocols. For example, Type 1 devices support CXL.io and CXL.cache, Type 2 devices support all three sub-protocols, and Type 3 devices support CXL.io and CXL.mem. In Type 2 devices, the two memories are uniformly addressed, forming a unified memory. The accelerator in this embodiment is equivalent to a Type 2 device.

[0075] It can be seen that since the server in this embodiment implements unified addressing and data consistency, compared with Figure 1, the data from the network is stored in the unified memory and can be used by the processor and accelerator at the same time, reducing the data movement between the processor memory and the accelerator memory.

[0076] As a feasible implementation method, the accelerator includes a computing high-speed interconnect core, an instruction generation module, and an accelerator core; the computing high-speed interconnect core is configured to communicate with the processor; the instruction generation module is configured to generate instructions based on the instruction data written in the register and send the instructions to the accelerator core; the accelerator core is configured to obtain original data from the unified memory through the computing high-speed interconnect protocol, execute instructions based on the original data to complete the target task, and write the task execution results into the unified memory through the computing high-speed interconnect protocol.

[0077] In an optional implementation, the accelerator's architecture, shown in Figure 3, includes a high-speed computational interconnect core, an instruction generation module, and an accelerator core. The high-speed computational interconnect core is a hardware implementation of the CXL protocol. Working with a series of host-side software, it implements CXL's cache coherence and unified memory addressing features. This unifies the addressing of processor and accelerator memory at the system level. The CXL protocol and certain CPU-side software are responsible for ensuring data (cache) coherence between the physically separate but logically unified memory of the processor and accelerator compute engines.

[0078] As a feasible implementation, the computing high-speed interconnect core includes a first interface and a second interface. The processor writes data to the register of the accelerator through the first interface, and the accelerator accesses the unified memory through the second interface.

[0079] In an optional implementation, the high-speed computing interconnect core handles communication with the processor and provides a first interface for the instruction generation module, configured to read and write registers in the instruction generation module. Additionally, the high-speed computing interconnect core provides a second interface for the accelerator core, configured to access unified memory.

[0080] The accelerator core in this embodiment can be optimized and modified based on the open source project Gemmini. The original application architecture includes a RISC-V (Reduced Instruction Set Computer-V) processor system and a Gemmini accelerator. The two are located in the same chip because they are called a SoC (System on Chip). The Gemmini accelerator is physically completely attached to the RISC-V processor, and the command configured to control the Gemmini accelerator is RoCC CMD (Rocket Chip Coprocessor Command), which is also the instruction format of RISC-V. However, in this embodiment, the processor and the accelerator are located on different chips, and the processor and the accelerator are connected via a physical bus such as a PCIe bus to achieve communication between the two chips. The physical bus is not compatible with the RoCC CMD command, resulting in the processor being unable to control the Gemmini accelerator.

[0081] Therefore, in this embodiment, an instruction generation module is implemented in the accelerator. The processor can access the registers in the instruction generation module through the first interface provided by the computing high-speed interconnect core. The instruction generation module will generate corresponding different instructions according to the values ​​of the registers under the control of different internal state machines and send them to the accelerator core.

[0082] As a feasible implementation method, the instruction generation module includes a register stack sub-module, a state machine corresponding to instructions of different instruction types, and an instruction processing sub-module; the register stack sub-module includes multiple instruction data registers corresponding to different state machines, which are configured to store instruction data corresponding to different instruction types; the state machine is configured to generate instructions of corresponding instruction types based on the instruction data in the corresponding instruction data register, and send the instructions to the instruction processing sub-module; the instruction processing sub-module is configured to process the instructions and then send them to the accelerator core.

[0083] In an optional implementation, the instruction generation module is shown in Figure 4. The processor configures the register stack submodule through CXL. The registers in the register stack submodule store information used to generate RoCC CMD instructions or control various state machines. Each register has an independent address, and the processor can access different registers by combining different addresses through CXL.

[0084] The accelerator core implemented based on Gemmini is shown in Figure 5. The instructions generated by the instruction generation module are stored in the original instruction cache queue. The convolution command expansion module is set to expand the convolution command, and the matrix multiplication command expansion module is set to expand the matrix multiplication command. The expanded commands are stored in the expanded command cache queue. Different types of instructions are temporarily stored in the reservation station. The reservation station is a cache that is set to temporarily store input data because it may be necessary to wait for other data to be ready before performing certain operations. Instructions of different types include load instructions, store instructions, and execute instructions. That is, the accelerator core includes a load operation controller, a store operation controller, and an execute operation controller. Load instructions are used to load data from the unified memory to the cache of the accelerator core, store instructions are used to store data from the cache of the accelerator core to the unified memory, and execute instructions are used to execute operations. The accelerator core includes a large cache queue, which includes a DMA read-write controller, an SRAM (Static Random-Access Memory) cache read-write controller, and a cache. The load operation controller loads data from the unified memory to the cache of the accelerator core through the DMA read-write controller. The storage operation controller stores data from the cache of the accelerator core to the unified memory through the DMA read-write controller. After the execution operation controller executes the operation, it stores the execution result in the cache of the accelerator core through the SRAM cache read-write controller.

[0085] Different state machines are configured to generate different types of instructions, which speeds up instruction generation efficiency, avoids mutual interference between different types of instructions, and improves the accuracy of instruction generation. Furthermore, due to the differences in the number of registers required to be configured between different types of instructions, or between instructions of the same type but with different operations, it is possible that the processor starts configuring registers to generate a certain instruction (assuming it is instruction A) first, and then starts configuring registers to generate another instruction (assuming it is instruction B). However, since instruction A requires more registers to be configured than instruction B, the registers for instruction B are configured first, and instruction B is generated first, followed by instruction A. This is inconsistent with the order of operations initiated by the processor.

[0086] In order to solve the problem that the order in which the instructions are generated by the instruction generation module is inconsistent with the order in which the processor initiates operations, the instruction processing submodule in the instruction generation module sorts the instructions generated by different state machines according to the startup timestamps of different state machines, and sends the instructions generated by different state machines to the accelerator core in sequence according to the sorting results.

[0087] In an optional implementation, the instruction processing submodule tracks the timestamps of the state machine startup moments of different instructions, that is, the timestamps of the processor initiating the operation, and sends corresponding instructions to the accelerator core according to the timestamp sequence when different state machines are started, so that the order in which the accelerator core receives instructions is consistent with the order in which the processor initiates the operation, thereby avoiding errors in the order in which the accelerator core executes instructions.

[0088] Furthermore, the instruction processing submodule is configured to reset the target state machine if the difference between the current timestamp and the target state machine's startup timestamp exceeds a threshold. In an optional implementation, the current timestamp of each state machine is updated in real time. By tracking the difference between the startup timestamp and the current timestamp of state machines for different instructions, if the difference exceeds a threshold, the state machine that has not completed its work for a long time is forcibly reset to correct the error and avoid prolonged state machine unavailability caused by instruction generation failure.

[0089] Different state machines correspond to multiple instruction data registers, which are configured to store the instruction data required to generate instructions. For example, each state machine corresponds to five instruction data registers, each of which is 32 bits, so each state machine can store up to 160 bits of instruction data. Each state machine can also have corresponding start signal registers and stop signal registers. The start signal register is configured to describe whether the state machine is started, and the stop signal register is configured to describe whether the instruction data has been written.

[0090] As a feasible implementation method, when the state machine is in the idle state, the corresponding start signal register is the first preset value. When it is detected that the processor sets the start signal register to the second preset value and the timeout reset signal is the first preset value, the state machine jumps to the waiting information filling state and updates the start timestamp; when the state machine is in the waiting information filling state, it waits for the processor to fill the instruction data into the corresponding instruction data register. When it is detected that the processor sets the corresponding stop signal register to the second preset value and the timeout reset signal is not the first preset value, the state machine jumps to the generate command state; when the state machine is in the generate command state, it generates an instruction according to the instruction data in the corresponding instruction data register and sends it to the instruction processing sub-module, and resets the corresponding instruction data register, start signal register, and stop signal register to the first preset value, and the state machine jumps to the idle state.

[0091] In an optional implementation, the workflow of the state machine for each state machine to generate instructions is shown in FIG6 . The initial state of the state machine after power-on is the idle state. When the start signal register corresponding to the state machine is at a first preset value (e.g., 0), the state machine will remain in the idle state. When the state machine detects that the processor has written the corresponding start signal register to a second preset value (e.g., 1) and the timeout reset signal is 0, it jumps to the waiting information filling state and simultaneously updates the timestamp register of the state machine to indicate the start time of the state machine's operation.

[0092] After the processor writes the start signal register corresponding to the state machine to the second preset value, it will continue to operate the instruction data register corresponding to the state machine. During this period, the state machine maintains the waiting information filling state; when the waiting time is too long, it will be reset to the idle state by the reset signal from the instruction processing sub-module.

[0093] After the processor completes the operation on the instruction data register corresponding to the state machine, it will continue to operate the stop signal state machine corresponding to the state machine and write it as the second preset value. When the state machine detects that its stop signal register is the second preset value and the timeout reset signal is not the first preset value, it will jump to the generate command state.

[0094] In the command generation state, the instruction generation module will use the instruction data in the instruction data register corresponding to the state machine to generate the corresponding instruction and send it to the instruction processing module. At the same time, it will reset all the registers corresponding to the state machine in the register stack sub-module and the timestamp register inside the state machine, and then unconditionally jump to the idle state.

[0095] In addition, since the RISC-V processor system and the Gemmini accelerator are located in the same chip in the original application architecture, when Gemmini accesses the L2 cache (secondary cache) or DRAM, it uses the virtual address given by the RoCC CMD command. When accessing the L2 cache or DRAM through the DMA read-write controller, it is necessary to first map the virtual address to a physical address through the TLB (Translation Lookaside Buffer) processing module, and then use the physical address to access. The TLB processing module needs to update the correspondence between the physical address and the virtual address with the CPU through the RoCC PTW (Post-Training Weighting Interface) interface, which takes a long time, and the implementation of the TLB processing module itself also requires hardware resources. However, in this embodiment, the processor and the accelerator are located on different chips, and the address mapping operation of the TLB processing module cannot be performed, and the data access requests received by the computing high-speed interconnect core all require the use of physical addresses.

[0096] Therefore, in this embodiment, the processor directly uses physical addresses when configuring registers. That is, the instruction data includes the physical address of the original data, and the accelerator is configured to retrieve the original data from the unified memory based on the physical address. In an alternative implementation, the processor stores the physical address of the data in a register for use in generating instructions. The DMA read / write controller in the accelerator core provides the physical address via a second interface, directly connecting to the computing high-speed interconnect core to access the unified memory. This avoids the conversion between virtual and physical addresses and reduces the latency of the accelerator accessing the unified memory.

[0097] The server task execution process provided in this embodiment is as follows: the processor receives the original task data via the network and stores it in the processor memory of the unified memory. The processor controls the instruction generation module via the CXL interface to generate instructions, operating the accelerator to directly perform data access operations between the processor memory and the accelerator memory in the unified memory, avoiding H2H data copy operations between the processor memory and the accelerator memory. The accelerator performs the task calculation and stores the task execution results in the accelerator memory of the unified memory. The processor directly sends the task execution results in the unified memory to the client via the network, avoiding D2H data copy operations between the processor memory and the accelerator memory.

[0098] In the embodiments of this application, processor memory and accelerator memory are uniformly addressed based on the CXL protocol, enabling direct access by the processor and accelerator to the uniformly addressed memory space. This reduces the number of data copies between the processor and accelerator memory, thereby reducing task execution latency and energy consumption. Furthermore, the processor writes instruction data to the accelerator's registers, and the accelerator generates instructions based on the instruction data written to the registers. This enables the conversion between processor configuration register information and accelerator commands, provides effective processor control logic for the accelerator, and enables independent deployment of the accelerator for CXL-compatible server hosts.

[0099] The embodiment of the present application discloses a task execution method, which reduces the delay and power consumption of task execution.

[0100] 6 , which is a flowchart of a task execution method according to an exemplary embodiment, includes:

[0101] S101: Obtain the original data of the target task and store the original data in the unified memory;

[0102] The execution subject of this embodiment is the processor in the server. The server includes a processor, processor memory, and an accelerator connected to the processor. The accelerator includes an accelerator memory. The processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol. The server can be an edge computing server. The purpose of this embodiment is to achieve accelerated execution of the target task through the accelerator. The target task can be a neural network reasoning task. Due to the limitations of the computing power, network, memory and other resources of the edge computing server, in order to achieve better task execution effects, usually only neural network reasoning tasks are run in the edge computing server, rather than neural network training tasks.

[0103] In this step, the processor receives the original data of the target task through the network and stores it in the processor memory of the unified memory.

[0104] S102: Writing instruction data into a register of the accelerator based on the target task, so that the accelerator generates instructions based on the instruction data written in the register, obtains original data from the unified memory through the computing high-speed interconnect protocol, executes the instructions based on the original data to complete the target task, and writes the task execution result into the unified memory through the computing high-speed interconnect protocol;

[0105] In this step, the processor controls the instruction generation module through the CXL interface to generate instructions, allowing the accelerator to directly access data between the processor memory and the accelerator memory in the unified memory, avoiding H2D data copy operations between the processor memory and the accelerator memory. The accelerator performs the task computation and stores the task results in the accelerator memory within the unified memory.

[0106] As a feasible implementation, the instruction data includes a physical address of the original data, and the accelerator is configured to obtain the original data from the unified memory based on the physical address.

[0107] In an optional implementation, the processor directly uses the physical address when configuring the register, that is, the processor stores the physical address of the data in the register for generating instructions. The DMA read and write controller in the accelerator core provides the physical address through the second interface and is directly connected to the computing high-speed interconnect core to access the unified memory, avoiding the conversion between the virtual address and the physical address, and reducing the latency of the accelerator accessing the unified memory.

[0108] S103: Obtain the task execution result from the unified memory.

[0109] In this step, the processor directly sends the task execution results in the unified memory to the client through the network, avoiding the D2H data copy operation between the processor memory and the accelerator memory.

[0110] In the embodiments of this application, processor memory and accelerator memory are uniformly addressed based on the CXL protocol, enabling direct access by the processor and accelerator to the uniformly addressed memory space. This reduces the number of data copies between the processor and accelerator memory, thereby reducing task execution latency and energy consumption. Furthermore, the processor writes instruction data to the accelerator's registers, and the accelerator generates instructions based on the instruction data written to the registers. This enables the conversion between processor configuration register information and accelerator commands, provides effective processor control logic for the accelerator, and enables independent deployment of the accelerator for CXL-compatible server hosts.

[0111] The present application discloses a method for executing a task. In some embodiments:

[0112] Referring to FIG7 , a flowchart of another task execution method according to an exemplary embodiment is shown. As shown in FIG7 , the method includes:

[0113] S201: Generate an instruction according to the instruction data written by the processor into the register;

[0114] The execution subject of this embodiment is an accelerator in a server. The server includes a processor, processor memory, and an accelerator connected to the processor. The accelerator includes an accelerator memory. The processor memory and accelerator memory are uniformly addressed as a unified memory via a high-speed computing interconnect protocol. The purpose of this embodiment is to achieve accelerated execution of target tasks through the accelerator.

[0115] In an optional implementation, the processor receives raw data of the target task via the network and stores it in the processor memory of the unified memory. The processor writes instruction data to the register of the accelerator via the CXL interface, and the instruction generation module of the accelerator generates instructions based on the instruction data in the register.

[0116] As a feasible implementation method, instructions are generated based on the instruction data written by the processor into the register, including: using the state machine corresponding to the instructions of different instruction types to generate instructions of the corresponding instruction type according to the instruction data in the corresponding instruction data register; sorting the instructions generated by different state machines according to the startup timestamps of different state machines, and sending the instructions generated by different state machines to the accelerator core for execution in sequence according to the sorting results; if the difference between the current timestamp and the startup timestamp of the target state machine is greater than a threshold, resetting the target state machine.

[0117] Different state machines are set to generate different types of instructions. Since the number of registers required to be configured varies between different types of instructions and between instructions of the same type but with different operations, it is possible that the processor starts configuring registers to generate a certain instruction (assuming it is instruction A) and then starts configuring registers to generate another instruction (assuming it is instruction B). However, since instruction A requires more registers to be configured than instruction B, the registers of instruction B are configured first, and instruction B is generated first, followed by instruction A. This is inconsistent with the order of operations initiated by the processor.

[0118] In order to solve the problem that the order in which the instructions are generated by the instruction generation module is inconsistent with the order in which the processor initiates operations, the instruction processing submodule in the instruction generation module sorts the instructions generated by different state machines according to the startup timestamps of different state machines, and sends the instructions generated by different state machines to the accelerator core in sequence according to the sorting results.

[0119] In an optional implementation, the instruction processing submodule tracks the timestamps of the state machine startup times of different instructions, that is, the timestamps of the processor initiating the operation, and sends the corresponding instructions to the accelerator core according to the timestamp sequence when the different state machines are started, so that the order in which the accelerator core receives the instructions is consistent with the order in which the processor initiates the operation.

[0120] Furthermore, the current timestamp of each state machine is updated in real time. By tracking the difference between the timestamp of the startup moment of the state machine of different instructions and the current timestamp, when the difference is greater than the threshold, the state machine that has not completed work for a long time is forcibly reset to correct the error, thereby avoiding the state machine being unavailable for a long time due to instruction generation failure.

[0121] The workflow of each state machine generating an instruction includes the following: the initial state of the state machine after power-on is an idle state. When the start signal register corresponding to the state machine is at a first preset value (e.g., 0), the state machine will remain in the idle state. When the state machine detects that the processor has written the corresponding start signal register to a second preset value (e.g., 1) and the timeout reset signal is 0, it jumps to the waiting information filling state and simultaneously updates the timestamp register of the state machine to indicate the start time of the state machine's operation.

[0122] After the processor writes the start signal register corresponding to the state machine to the second preset value, it will continue to operate the instruction data register corresponding to the state machine. During this period, the state machine maintains the waiting information filling state; when the waiting time is too long, it will be reset to the idle state by the reset signal from the instruction processing sub-module.

[0123] After the processor completes the operation on the instruction data register corresponding to the state machine, it will continue to operate the stop signal state machine corresponding to the state machine and write it as the second preset value. When the state machine detects that its stop signal register is the second preset value and the timeout reset signal is not the first preset value, it will jump to the generate command state.

[0124] In the command generation state, the instruction generation module will use the instruction data in the instruction data register corresponding to the state machine to generate the corresponding instruction and send it to the instruction processing module. At the same time, it will reset all the registers corresponding to the state machine in the register stack sub-module and the timestamp register inside the state machine, and then unconditionally jump to the idle state.

[0125] S202: Obtaining original data from the unified memory by computing a high-speed interconnection protocol, and executing instructions based on the original data to complete the target task;

[0126] In this step, the accelerator directly accesses data between the processor memory and the accelerator memory in the unified memory using the CXL protocol, avoiding the H2D data copy operation between the processor memory and the accelerator memory. Furthermore, the accelerator performs the task calculation based on the acquired raw data.

[0127] S203: Writing the task execution result into the unified memory via the computing high-speed interconnection protocol.

[0128] In this step, the accelerator saves the task execution result in the accelerator memory of the unified memory, and the processor directly sends the task execution result in the unified memory to the client through the network, avoiding the D2H data copy operation between the processor memory and the accelerator memory.

[0129] In the embodiments of this application, processor memory and accelerator memory are uniformly addressed based on the CXL protocol, enabling direct access by the processor and accelerator to the uniformly addressed memory space. This reduces the number of data copies between the processor and accelerator memory, thereby reducing task execution latency and energy consumption. Furthermore, the processor writes instruction data to the accelerator's registers, and the accelerator generates instructions based on the instruction data written to the registers. This enables the conversion between processor configuration register information and accelerator commands, provides effective processor control logic for the accelerator, and enables independent deployment of the accelerator for CXL-compatible server hosts.

[0130] A task execution device provided in an embodiment of the present application is introduced below. The task execution device described below and the task execution method described above can be referenced to each other.

[0131] Referring to FIG8 , a structural diagram of a task execution device according to an exemplary embodiment is shown. As shown in FIG8 , the device includes:

[0132] The storage unit 101 is configured to obtain the original data of the target task and store the original data in a unified memory;

[0133] The first writing unit 102 is configured to write instruction data into a register of the accelerator based on a target task, so that the accelerator generates instructions based on the instruction data written in the register, obtains raw data from the unified memory via the computing high-speed interconnect protocol, executes the instructions based on the raw data to complete the target task, and writes the task execution result into the unified memory via the computing high-speed interconnect protocol;

[0134] The acquiring unit 103 is configured to acquire the task execution result from the unified memory.

[0135] In the embodiments of this application, processor memory and accelerator memory are uniformly addressed based on the CXL protocol, enabling direct access by the processor and accelerator to the uniformly addressed memory space. This reduces the number of data copies between the processor and accelerator memory, thereby reducing task execution latency and energy consumption. Furthermore, the processor writes instruction data to the accelerator's registers, and the accelerator generates instructions based on the instruction data written to the registers. This enables the conversion between processor configuration register information and accelerator commands, provides effective processor control logic for the accelerator, and enables independent deployment of the accelerator for CXL-compatible server hosts.

[0136] Based on the above embodiment, as an optional implementation manner, the instruction data includes a physical address of the original data, and the accelerator is configured to obtain the original data from the unified memory based on the physical address.

[0137] Based on the above embodiment, as an optional implementation manner, the target task includes a neural network reasoning task.

[0138] Another task execution device provided in an embodiment of the present application is introduced below. The task execution device described below and the other task execution method described above can be referenced to each other.

[0139] Referring to FIG9 , a structural diagram of another task execution device according to an exemplary embodiment is shown. As shown in FIG9 , the device includes:

[0140] A generating unit 201 is configured to generate an instruction according to instruction data written into a register by a processor;

[0141] The execution unit 202 is configured to obtain raw data from the unified memory through a computing high-speed interconnect protocol and execute instructions based on the raw data to complete the target task;

[0142] The second writing unit 203 is configured to write the task execution result into the unified memory through the computing high-speed interconnection protocol.

[0143] In the embodiments of this application, processor memory and accelerator memory are uniformly addressed based on the CXL protocol, enabling direct access by the processor and accelerator to the uniformly addressed memory space. This reduces the number of data copies between the processor and accelerator memory, thereby reducing task execution latency and energy consumption. Furthermore, the processor writes instruction data to the accelerator's registers, and the accelerator generates instructions based on the instruction data written to the registers. This enables the conversion between processor configuration register information and accelerator commands, provides effective processor control logic for the accelerator, and enables independent deployment of the accelerator for CXL-compatible server hosts.

[0144] Based on the above embodiment, as an optional implementation, the generation unit 201 is configured to: use the state machine corresponding to the instructions of different instruction types to generate instructions of corresponding instruction types according to the instruction data in the corresponding instruction data register; sort the instructions generated by different state machines according to the startup timestamps of different state machines, and send the instructions generated by different state machines to the accelerator core for execution in sequence according to the sorting results; if the difference between the current timestamp and the startup timestamp of the target state machine is greater than a threshold, reset the target state machine.

[0145] Regarding the apparatus in the above embodiment, the manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0146] The embodiment of the present application also provides a non-volatile readable storage medium, which can be a computer non-volatile readable storage medium, for example, including a memory 3 storing a computer program, and the computer program can be executed by the processor 2 to complete the above method steps. The computer non-volatile readable storage medium can be FRAM (ferromagnetic random access memory), ROM (non-volatile memory can be read-only memory, Read Only Memory), PROM (programmable read-only memory, Programmable Read-Only Memory), EPROM (erasable programmable read-only memory, Erasable Programmable Read-Only Memory), EEPROM (electrically erasable programmable read-only memory, Electrically Erasable Programmable Read-Only Memory), Flash Memory (flash memory), magnetic surface memory, optical disk, CD-ROM (compact disc read-only memory), random access memory (RAM, Random Access Memory), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct rambus random access memory (DRRAM), and the like.

[0147] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer non-volatile readable storage medium, which, when executed, executes the steps including the above method embodiment.

[0148] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer non-volatile readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a non-volatile readable storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, network device, etc.) to execute all or part of the methods of each embodiment of the present application.

[0149] The above methods are only optional implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the protection scope of the present application.

Claims

1. A server, characterized in that: The system comprises a processor, a processor memory, and an accelerator connected to the processor, wherein the accelerator includes an accelerator memory, and the processor memory and the accelerator memory are uniformly addressed as a unified memory through a computing high-speed interconnection protocol; The processor is configured to obtain raw data of a target task, store the raw data in the unified memory, write instruction data into a register of the accelerator based on the target task, and obtain a task execution result of the target task from the unified memory; The accelerator is configured to generate instructions based on the instruction data written in the register, obtain the original data from the unified memory through a computing high-speed interconnection protocol, execute the instructions based on the original data to complete the target task, and write the task execution results into the unified memory through a computing high-speed interconnection protocol.

2. The server according to claim 1, characterized in that The accelerator includes a computing high-speed interconnect core, an instruction generation module, and an accelerator core; The computing high-speed interconnect core is configured to communicate with the processor; The instruction generation module is configured to generate instructions according to the instruction data written in the register and send the instructions to the accelerator core; The accelerator core is configured to obtain the original data from the unified memory through a computing high-speed interconnection protocol, execute the instruction based on the original data to complete the target task, and write the task execution result into the unified memory through a computing high-speed interconnection protocol.

3. The server according to claim 2, wherein: The computing high-speed interconnect core includes a first interface and a second interface. The processor writes data to the register of the accelerator through the first interface, and the accelerator accesses the unified memory through the second interface.

4. The server according to claim 2, characterized in that The instruction generation module includes a register stack submodule, a state machine corresponding to instructions of different instruction types, and an instruction processing submodule; The register stack submodule includes a plurality of instruction data registers corresponding to different state machines, for storing instruction data corresponding to different instruction types; The state machine is configured to generate an instruction of a corresponding instruction type according to the instruction data in the corresponding instruction data register, and send the instruction to the instruction processing submodule; The instruction processing submodule is used to process the instruction and then send it to the accelerator core.

5. The server according to claim 4, characterized in that: The instruction processing submodule is configured to sort the instructions generated by different state machines according to the startup timestamps of the different state machines, and send the instructions generated by the different state machines to the accelerator core in sequence according to the sorting results.

6. The server according to claim 4, characterized in that: The instruction processing submodule is further configured to reset the target state machine if the difference between the current timestamp and the startup timestamp of the target state machine is greater than a threshold.

7. The server according to claim 4, characterized in that: The register stack submodule further includes a start signal register and a stop signal register corresponding to different state machines. The start signal register is configured to describe whether the state machine is started, and the stop signal register is configured to describe whether the instruction data is written successfully.

8. The server according to claim 7, characterized in that: When the state machine is in an idle state, the corresponding start signal register is a first preset value. When it is detected that the processor sets the start signal register to a second preset value and the timeout reset signal is the first preset value, the state machine jumps to a waiting information filling state and updates the start timestamp. When the state machine is in the waiting information filling state, it waits for the processor to fill the corresponding instruction data register with instruction data, and when it is detected that the processor sets the corresponding stop signal register to the second preset value and the timeout reset signal is not the first preset value, the state machine jumps to the generating command state; When the state machine is in the command generation state, an instruction is generated according to the instruction data in the corresponding instruction data register and sent to the instruction processing sub-module, and the corresponding instruction data register, start signal register, and stop signal register are reset to the first preset value, and the state machine jumps to the idle state.

9. The server according to claim 4, characterized in that: Instructions of different instruction types include load instructions, store instructions, and execute instructions. The load instructions are used to load data from the unified memory to the cache of the accelerator core, the store instructions are used to store data from the cache of the accelerator core to the unified memory, and the execute instructions are used to execute operations.

10. The server according to claim 1, wherein: The instruction data includes a physical address of the original data, and the accelerator is configured to obtain the original data from the unified memory based on the physical address.

11. The server according to claim 1, wherein: The accelerator is an accelerator implemented based on a field programmable logic gate array.

12. The server according to claim 1, wherein: The processor is connected to the accelerator via a high-speed serial computer expansion bus standard protocol interface.

13. A task execution method, characterized in that: A processor applied to a server, the server comprising the processor, processor memory, and an accelerator connected to the processor, the accelerator comprising the accelerator memory, the processor memory and the accelerator memory being uniformly addressed as a unified memory via a computing high-speed interconnection protocol; the method comprising: Acquire the original data of the target task, and store the original data in the unified memory; Writing instruction data into a register of the accelerator based on the target task, so that the accelerator generates instructions according to the instruction data written in the register, obtains the original data from the unified memory through a computing high-speed interconnect protocol, executes the instructions based on the original data to complete the target task, and writes the task execution result to the unified memory through the computing high-speed interconnect protocol; Obtaining task execution results from the unified memory.

14. The task execution method according to claim 13, characterized in that: The instruction data includes a physical address of the original data, and the accelerator is configured to obtain the original data from the unified memory based on the physical address.

15. The task execution method according to claim 13, characterized in that: The target task includes a neural network reasoning task.

16. A task execution method, characterized in that: An accelerator applied to a server, the server comprising a processor, a processor memory, and the accelerator connected to the processor, the accelerator comprising the accelerator memory, the processor memory and the accelerator memory being uniformly addressed as a unified memory via a computing high-speed interconnection protocol; the method comprising: Generate instructions based on instruction data written by the processor into the register; Obtaining original data from the unified memory by computing a high-speed interconnection protocol, and executing the instruction based on the original data to complete the target task; The task execution result is written into the unified memory through the computing high-speed interconnection protocol.

17. The task execution method according to claim 16, characterized in that: Generating an instruction according to instruction data written by the processor into the register includes: Utilizing state machines corresponding to instructions of different instruction types to generate instructions of corresponding instruction types according to instruction data in corresponding instruction data registers; Sorting the instructions generated by different state machines according to the startup timestamps of different state machines, and sending the instructions generated by different state machines to the accelerator core for execution in sequence according to the sorting results; If the difference between the current timestamp and the startup timestamp of the target state machine is greater than a threshold, the target state machine is reset.

18. A task execution device, characterized in that: A processor used in a server, the server comprising the processor, processor memory, and an accelerator connected to the processor, the accelerator comprising the accelerator memory, the processor memory and the accelerator memory being uniformly addressed as a unified memory via a computing high-speed interconnection protocol; the device comprising: A storage unit is configured to obtain raw data of a target task and store the raw data in the unified memory; a first writing unit configured to write instruction data into a register of the accelerator based on the target task, so that the accelerator generates instructions according to the instruction data written in the register, obtains the original data from the unified memory through a computing high-speed interconnect protocol, executes the instructions based on the original data to complete the target task, and writes the task execution result into the unified memory through the computing high-speed interconnect protocol; The acquisition unit is configured to acquire the task execution result from the unified memory.

19. A task execution device, characterized in that: An accelerator applied to a server, the server comprising a processor, a processor memory, and the accelerator connected to the processor, the accelerator comprising the accelerator memory, the processor memory and the accelerator memory being uniformly addressed as a unified memory via a computing high-speed interconnection protocol; the device comprising: a generation unit configured to generate instructions based on instruction data written into the register by the processor; an execution unit, configured to obtain raw data from the unified memory through a computing high-speed interconnect protocol, and execute the instruction based on the raw data to complete a target task; The second writing unit is configured to write the task execution result into the unified memory through a computing high-speed interconnection protocol.

20. A computer-readable non-volatile storage medium, characterized in that: The computer non-volatile readable storage medium stores a computer program, and when the computer program is executed, the steps of the task execution method according to any one of claims 13 to 17 are implemented.

Images