Operation task execution method and device, computer equipment, readable storage medium and program product
By determining the idle state of the neural network computing kernel and allocating data processing units before the kernel performs computational tasks, the problem of low computational efficiency of computing kernels in multi-core architectures is solved, and more efficient execution of computational tasks is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-23
- Publication Date
- 2026-04-07
AI Technical Summary
In existing technologies, when multi-core architectures process neural network computations, the computational cores need to distinguish and sort the data processing units for different computational tasks, resulting in low computational efficiency.
Before executing the computation task, the idle state of the neural network computing kernel is determined, and the data processing units are allocated to the idle kernels. The data processing units are sorted and cached based on the dependencies between them to ensure that the dependent data processing units are launched to the target computing kernel in order for computation.
It improves the computational efficiency of the computing core, saves the computing power of the computing core, avoids the computing core from processing different computational tasks at the same time, and ensures that the computational process conforms to the algorithm logic.
Smart Images
Figure CN121807484A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, readable storage medium, and program product for performing computational tasks. Background Technology
[0002] With the continuous development of artificial intelligence technology, neural networks have achieved groundbreaking success in many application fields. However, as the scale of neural network models continues to expand and the computational complexity increases dramatically, it poses significant challenges to the underlying computing hardware, especially the processing power and energy efficiency of neural network accelerators.
[0003] To address these challenges, existing technologies employ a multi-core architecture, integrating multiple neural network computing cores to collaboratively complete computational tasks. However, this approach may result in a single computing core processing multiple tasks simultaneously. This necessitates that the computing core distinguish data processing units (tokens) for different tasks and sort the different data processing units for the same task before executing the task, leading to lower computational efficiency. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, apparatus, computer device, readable storage medium, and program product that can improve computational efficiency in order to address the above-mentioned technical problems.
[0005] Firstly, this application provides a method for executing a computational task, the method comprising:
[0006] Receive configuration information for computing tasks from the host;
[0007] The configuration information is parsed to obtain the number of data processing units required to execute the computation task, the storage address of any data processing unit in the memory, and the dependency relationship between different data processing units. Based on the number and the storage address, the data processing unit is retrieved from the memory.
[0008] Obtain the current running state of any neural network computing kernel, and identify the neural network computing kernel that is currently in an idle state as an idle computing kernel;
[0009] At least one idle computing core is identified as the target computing core for executing the computational task, and the correspondence between the data processing unit and the target computing core is determined; wherein, different data processing units with dependencies correspond to the same target computing core;
[0010] The data processing unit is cached in the launch queue of the corresponding target computing core, and based on the dependency relationship between different data processing units, the data processing unit is launched to the corresponding target computing core in sequence to trigger the target computing core to perform operation.
[0011] In one embodiment, parsing the configuration information to obtain the storage address of any data processing unit in the memory includes:
[0012] Based on the parsing results of the configuration information, obtain the base address of all data processing units for the computing task, and the address offset of any data processing unit;
[0013] Based on the base address and the address offset, obtain the storage address of any data processing unit in the memory.
[0014] In one embodiment, the step of sequentially sending the data processing units to their corresponding target computing cores based on the dependencies between different data processing units to trigger the target computing cores to perform operations includes:
[0015] For different data processing units that have dependencies, determine the order of operations between the different data processing units;
[0016] Based on the aforementioned operation order, the microinstruction code of the data processing unit is sequentially retrieved from the memory, and the data processing unit is sequentially sent to the target computing core, so that the target computing core executes the microinstruction code based on the data processing unit.
[0017] In one embodiment, the step of sequentially retrieving the microinstruction code of the data processing unit from the memory based on the operation order, and sequentially sending the data processing unit to the target computing core, includes:
[0018] For any current data processing unit among the different data processing units, before the target computing core executes the current microinstruction code of the current data processing unit, the execution status of the microinstruction codes of all data processing units located before the current data processing unit in the operation order is obtained;
[0019] When the execution state indicates that the microinstruction code of all data processing units has been executed, the current microinstruction code of the current data processing unit is retrieved from the memory, and the current data processing unit is launched to the target computing core.
[0020] In one embodiment, the method further includes:
[0021] When there is no dependency between the data processing unit and any other data processing unit, at the start of the operation of the target computing core, the microinstruction code of the data processing unit is retrieved from the memory, and the data processing unit is sent to the target computing core so that the target computing core executes the microinstruction code based on the data processing unit.
[0022] In one embodiment, the method further includes:
[0023] At the start of operation of any target computing core, the status bits of the target computing core are adjusted in the status register to change the running state of the target computing core from idle state to busy state.
[0024] Secondly, this application also provides a computing task execution device, the device comprising:
[0025] The receiving module is used to receive configuration information for computing tasks sent by the host.
[0026] The parsing module is used to parse the configuration information to obtain the number of data processing units required to execute the computing task, the storage address of any data processing unit in the memory, and the dependency relationship between different data processing units, and to obtain the data processing unit from the memory based on the number and the storage address.
[0027] The acquisition module is used to acquire the current running state of any neural network computing kernel and identify the neural network computing kernel that is currently in an idle state as an idle computing kernel;
[0028] A determination module is used to determine at least one idle computing core as the target computing core for executing the computing task, and to determine the correspondence between the data processing unit and the target computing core; wherein different data processing units with dependencies correspond to the same target computing core;
[0029] The caching module is used to cache the data processing unit in the launch queue of the corresponding target computing core, and launch the data processing unit to the corresponding target computing core in sequence based on the dependency relationship between different data processing units, so as to trigger the target computing core to perform operation.
[0030] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the methods in any of the above embodiments.
[0031] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.
[0032] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the methods in any of the above embodiments.
[0033] The aforementioned computation task execution method, apparatus, computer equipment, readable storage medium, and program product receive configuration information for the computation task from the host; parse the configuration information to obtain the number of data processing units required to execute the computation task, the storage address of any data processing unit in the memory, and the dependencies between different data processing units; and retrieve data processing units from the memory based on the number and storage address; obtain the current running state of any neural network computing core and determine the neural network computing core currently in an idle state as an idle computing core; determine at least one idle computing core as the target computing core for executing the computation task, and determine the correspondence between data processing units and target computing cores; wherein different data processing units with dependencies correspond to the same target computing core; cache the data processing units in the corresponding target computing core's launch queue, and based on the dependencies between different data processing units, sequentially launch the data processing units to the corresponding target computing cores to trigger the target computing cores to perform computation. The method provided in this application determines whether a computing core is in an idle state before it executes a computational task, and then assigns the computational task to an idle computing core. In this way, the same computing core will not execute different computational tasks at the same time, and the computing core does not need to distinguish the data processing units of different computational tasks. This can effectively improve the computational efficiency of the computing core and save its computing power. Furthermore, the dependencies between different data processing units are directly obtained from memory, so the computing core does not need to sort different data processing units of the same computational task, which can further improve the computational efficiency of the computing core and save its computing power. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0035] Figure 1 This is an application environment diagram of a computation task execution method in one embodiment;
[0036] Figure 2 This is a flowchart illustrating a computation task execution method in one embodiment;
[0037] Figure 3 This is a flowchart illustrating a configuration information parsing method in one embodiment;
[0038] Figure 4 This is an architecture diagram of the computation task execution method in another embodiment;
[0039] Figure 5 This is a schematic diagram of a multi-network model parallel acceleration scenario in another embodiment;
[0040] Figure 6 This is a schematic diagram of a multi-core acceleration scenario for a single model in another embodiment;
[0041] Figure 7 This is a schematic diagram of a multi-core acceleration scenario for a single network layer in another embodiment;
[0042] Figure 8 This is a structural block diagram of a computing task execution device in one embodiment;
[0043] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0045] The computational task execution method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, HOST 101 communicates with NPU (Neural Processing Unit) 102 via a hardware interface. HOST 101 issues computational task execution instructions to NPU 102. After NPU 102 triggers the neural network computing core to execute the computational task, it feeds back the execution result to HOST 101. HOST 101 is the host machine where the CPU (Central Processing Unit) is located.
[0046] In one exemplary embodiment, such as Figure 2 As shown, a method for executing computational tasks is provided, which can be applied to... Figure 1 Taking the NPU as an example, the explanation includes steps 202 to 210. Wherein:
[0047] S202, Receive configuration information for the computing task sent by the host.
[0048] In this context, a computational task is a sub-computational unit after the neural network model is decomposed. When the division granularity is based on operators, one computational task corresponds to at least one operator. Here, an operator refers to the basic, indivisible computational unit in the neural network model, which is the smallest functional module that constitutes the neural network model.
[0049] Optionally, the HOST sends the configuration information of the computing task to the NPU through apb2reg (APB bus to register interface module) with the help of the NPU's internal configuration registers; among them, apb2reg is the communication relay hub between the HOST and the internal modules of the NPU, and its core function is to enable the HOST to configure and read the status of the NPU's internal registers through the APB (Advanced Peripheral Bus) bus.
[0050] S204. Parse the configuration information to obtain the number of data processing units required to execute the computation task, the storage address of any data processing unit in the memory, and the dependencies between different data processing units, and obtain the data processing units from the memory based on the number and storage address.
[0051] Among them, the data processing unit (token) is the smallest carrier that carries the complete data and control information required for a single operator operation.
[0052] Optionally, the memory may be, but is not limited to, DDR (Double Data Rate SDRAM).
[0053] Optionally, if there is a dependency between two data processing units, the token executed later must wait for the token executed earlier to complete the calculation and output the result before it can start its own calculation process. There is a strict sequential constraint between the two in terms of execution time, and they cannot be executed in parallel.
[0054] Alternatively, DMA (Direct Memory Access) technology can be used to retrieve data processing units from memory based on the number of data processing units and their storage addresses in memory, and then temporarily store the retrieved data processing units in a FIFO (First-In First-Out) buffer.
[0055] S206. Obtain the current running state of any neural network computing kernel, and determine the neural network computing kernel that is currently in an idle state as an idle computing kernel.
[0056] The neural network computing core is the core hardware unit located inside the NPU and serves as the carrier for the NPU to perform neural network operations.
[0057] Optionally, if the neural network computing core is currently in an idle state, it means that the neural network computing core is not currently performing other computing tasks and can be used as a candidate computing core for performing the computing tasks in the embodiments of this application.
[0058] S208. At least one idle computing core is identified as the target computing core for performing the computation task, and the correspondence between the data processing unit and the target computing core is determined; wherein, different data processing units with dependencies correspond to the same target computing core.
[0059] Optionally, the corresponding computing core type can be determined based on the operator type of the operator corresponding to the data processing unit, and the idle computing core of the type can be determined as the target computing core corresponding to the data processing unit. Alternatively, data processing units located in the same continuous area in the memory can be allocated to the same target computing core. This application embodiment does not specifically limit this.
[0060] S210. Cache the data processing unit in the launch queue of the corresponding target computing core, and launch the data processing unit to the corresponding target computing core in sequence based on the dependency relationship between different data processing units, so as to trigger the target computing core to perform operation.
[0061] The launch queue is a dedicated buffer queue for each target computing core, used to temporarily store data processing units to be executed.
[0062] Optionally, if the data processing unit of the computation task includes token1, token2, token3, and token4, where token1 corresponds to the target computing core N1, token2 and token3 correspond to the target computing core N2, and token4 corresponds to the target computing core N3, and there is a dependency relationship between token2 and token3, with token3 depending on token2; then token1 is cached in the launch queue corresponding to N1, token2 and token3 are cached in the launch queue corresponding to N2, and token4 is cached in the launch queue corresponding to N3; during the computation of N2 based on token2 and token3, token2 is first launched into N2, and after the computation of N2 based on token2 is completed, token3 is launched into N2, so that N2 can perform computation based on token3.
[0063] Optionally, data processing units can be cached in batches to the launch queue of the target computing core. Each batch includes at least one data processing unit, and different data processing units with dependencies are located in the same batch. After the target computing core finishes its operation based on the data processing units of the current batch, it is determined whether the data processing units of the current batch are the last batch of data processing units in the operation task that have not yet been operated. If not, the data processing units of the next batch are cached in the launch queue of the corresponding target computing core so that the target computing core can perform operation based on the data processing units of the next batch.
[0064] In the above-described method for executing computational tasks, the following steps are taken: receiving configuration information for the computational task from the host; parsing the configuration information to obtain the number of data processing units required to execute the computational task, the storage address of any data processing unit in memory, and the dependencies between different data processing units; retrieving data processing units from memory based on the number and storage address; obtaining the current running state of any neural network computing core and identifying the currently idle neural network computing core as an idle computing core; identifying at least one idle computing core as the target computing core for executing the computational task and determining the correspondence between data processing units and target computing cores; wherein different data processing units with dependencies correspond to the same target computing core; caching the data processing units in the corresponding target computing core's launch queue; and, based on the dependencies between different data processing units, sequentially launching the data processing units to the corresponding target computing cores to trigger the target computing cores to perform computations. The method provided in this application determines whether a computing core is in an idle state before it executes a computational task, and then assigns the computational task to an idle computing core. In this way, the same computing core will not execute different computational tasks at the same time, and the computing core does not need to distinguish the data processing units of different computational tasks. This can effectively improve the computational efficiency of the computing core and save its computing power. Furthermore, the dependencies between different data processing units are directly obtained from memory, so the computing core does not need to sort different data processing units of the same computational task, which can further improve the computational efficiency of the computing core and save its computing power.
[0065] In some embodiments, such as Figure 3 As shown, by parsing the configuration information, the storage address of any data processing unit in memory is obtained, including:
[0066] S302. Based on the parsing results of the configuration information, obtain the base address of all data processing units for the computing task, and the address offset of any data processing unit.
[0067] S304. Based on the base address and address offset, obtain the storage address of any data processing unit in memory.
[0068] The base address refers to the starting physical or virtual address of a contiguous storage area, which is the reference point for locating data in memory.
[0069] In this embodiment, by using a base address combined with an offset address, the storage address of each token in the memory can be uniquely determined, avoiding data reading misalignment and ensuring that the computing core can accurately obtain the structured data and control information required for computation.
[0070] In some embodiments, based on the dependencies between different data processing units, data processing units are sequentially launched to corresponding target computing cores to trigger the target computing cores to perform operations, including: determining the operation order between different data processing units that have dependencies; based on the operation order, sequentially retrieving the microinstruction code of the data processing units from the memory, and sequentially launching the data processing units to the target computing cores so that the target computing cores execute the microinstruction code based on the data processing units.
[0071] Among them, the micro-instruction code (ucode) is the smallest instruction unit executed by the hardware, that is, the micro-instruction of the target computing core. The ucode is the low-level instruction when the target computing core performs token data operations. It is the operation code directly executed by the hardware and corresponds to the basic function of the computing core.
[0072] In this embodiment, the order of operations is determined by following the dependency relationships between tokens, avoiding data errors caused by out-of-order execution and ensuring that the operation flow of the neural network operator chain conforms to the algorithm logic.
[0073] In some embodiments, based on the order of operations, the microinstruction codes of data processing units are sequentially retrieved from memory, and the data processing units are sequentially launched to the target computing core. This includes: for any current data processing unit among different data processing units, before the target computing core executes the current microinstruction code of the current data processing unit, obtaining the execution status of the microinstruction codes of all data processing units preceding the current data processing unit in the order of operations; when the execution status indicates that the microinstruction codes of all data processing units have been executed, retrieving the current microinstruction code of the current data processing unit from memory, and launching the current data processing unit to the target computing core.
[0074] Optionally, if the execution status indicates that the microinstruction code of the data processing unit has not been executed, it is necessary to wait for the unexecuted microinstruction code to be executed before retrieving the current microinstruction code of the current data processing unit from memory.
[0075] In this embodiment, by verifying the execution status of the preceding token micro-instruction, the tokens with dependencies are forced to be executed in order, avoiding data input errors caused by out-of-order emission, and ensuring that the operation flow of the neural network operator chain conforms to the algorithm design logic.
[0076] In some embodiments, the method further includes: when there is no dependency between the data processing unit and any other data processing unit, at the start of the operation of the target computing core, retrieving the microinstruction code of the data processing unit from the memory and sending the data processing unit to the target computing core, so that the target computing core executes the microinstruction code based on the data processing unit.
[0077] Optionally, if there is no dependency between the data processing unit and any other data processing unit, it means that the operation process of the data processing unit is not affected by other data processing units, and the data processing unit can be directly operated.
[0078] In this embodiment, the dependency-free token does not need to wait for the execution of the preceding task and can be launched and executed directly when the computing core is idle, which fully releases the parallel computing power of the NPU multi-core and shortens the overall task execution time.
[0079] In some embodiments, the method further includes: at the start of operation of any target computing core, adjusting the status bit of the target computing core in the status register to change the running state of the target computing core from an idle state to a busy state.
[0080] The status register is a hardware register inside the NPU, specifically used to record the current working state of all neural network computing cores in real time. It is the information entry point for the Host or scheduling module to obtain the core status. The status bit is used to characterize the running state of the target computing core.
[0081] In this embodiment, the "idle or busy" status of the computing core is marked in real time by the bits of the status register. The host can quickly identify available computing resources, prevent repeated issuance of tokens to busy cores, and avoid computing task conflicts and hardware resource waste.
[0082] In one exemplary embodiment, another method for executing computational tasks is provided, the architecture of which is shown in the figure below. Figure 4As shown in the diagram, based on the architecture above, the NPU's operation flow changes to Task => Token => Ucode (i.e., the configured Task request instructs TD_sync (the scheduler) to fetch a batch of tokens, which are then distributed to each NN core (neural network computation core) via dispatch (the distribution module). The tokens then instruct the NN (Neural Network) to fetch ucode and complete the computation task). Simultaneously, to effectively reduce the hardware implementation complexity and timing pressure on the allocation circuitry, and to avoid excessive multiplexers and judgment logic, the allocation of cores for different tasks must be considered mutually exclusive. If not mutually exclusive, tokens shared by different tasks on NN cores would require a large amount of additional logic for judgment and sorting, which is detrimental to the flexibility of hardware design. Each task request will move a batch of tokens from the corresponding location in DDR, and then the relevant modules in TD_sync will distribute the tokens to the corresponding cores for computation. A Task can be understood as a task request initiated by the HOST to the NPU. Under the current design, the number of tasks that can be parallelized is related to the available hardware resources.
[0083] in accordance with Figure 4 The architecture diagram shown illustrates that the execution method for this computational task includes the following:
[0084] (1) The HOST external host will write the task information of the current operation and the base address of DDR through the configuration register; this process is executed by ahb2reg.
[0085] (2) The configured task information will be parsed. The task in the current preliminary design will contain the offset address and number of tokens stored in DDR for this batch, and will also contain the data correlation information of all tokens in this batch; this process is executed by task_ctrl (task control module).
[0086] (3) Based on the base address and num information obtained from parsing the task, the token in DDR is moved to the token FIFO via DMA; this process is executed by DMA.
[0087] (4) The dispatch module will distribute the token command according to the token info issued in the corresponding task control. It mainly sends the token to the corresponding launch queue and decides when to launch based on the core allocation and the correlation information between tokens specified in the task; this process is executed by the dispatch module and the issue (launch module).
[0088] (5) With the issuance of the token, each NN core will be automatically triggered to start calculation. At the same time, the working core will rewrite the corresponding bit in the NN-status register to record whether each NN is in an idle state, which is used by the HOST to perform register lookup. This process is executed by the NN.
[0089] (6) The NN core completes the calculation of the current token, raises the end signal, and sends it back to the dispatch module. If it is not the last token of the task, it returns to step (4) and repeats this process; otherwise, it executes step (7).
[0090] (7) After the NN completes the calculation of all tokens in this task, the dispatch module will raise the end signal and send it back to the corresponding task_ctrl. Then the task_ctrl will raise the corresponding bit in the end register and trigger the HOST interrupt.
[0091] Based on this computational task execution method, scenarios with multi-core parallelism can be explained:
[0092] (1) Parallel acceleration of multiple network models
[0093] In this scenario, the toolchain will compile the network using a compiler, and the software can then use task requests to assign the network model to the corresponding neural network core for execution. For example... Figure 5 As shown, task0 is configured to perform operations on NN1 using model-1, while task1 is configured to perform operations on NN2 using model-2.
[0094] (2) Multi-core acceleration of a single model and multi-core acceleration of a single network layer
[0095] like Figure 6 and Figure 7 As shown, Figure 6 This is a schematic diagram illustrating a multi-core acceleration scenario for a single model. Figure 7 This is a schematic diagram of a multi-core acceleration scenario for a single network layer. These two scenarios are actually the same from the perspective of the hardware because the hardware decomposes the network from the perspective of operators. Therefore, whether it is the decomposition of a certain layer of the network or the parallel processing of branches in the network, the hardware actually operates as two operators.
[0096] Furthermore, the two scenarios mentioned above can also be combined if sufficient hardware resources are available.
[0097] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0098] Based on the same inventive concept, this application also provides a computational task execution device for implementing the computational task execution method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the computational task execution device provided below can be found in the limitations of the computational task execution method above, and will not be repeated here.
[0099] In one exemplary embodiment, such as Figure 8 As shown, a computing task execution device 800 is provided, including: a receiving module 801, a parsing module 802, an acquisition module 803, a determining module 804, and a caching module 805, wherein:
[0100] The receiving module 801 is used to receive the configuration information of the computing task sent by the host.
[0101] The parsing module 802 is used to parse the configuration information to obtain the number of data processing units required to execute the computation task, the storage address of any data processing unit in the memory, and the dependency relationship between different data processing units, and to obtain the data processing unit from the memory based on the number and the storage address.
[0102] The acquisition module 803 is used to acquire the current running state of any neural network computing kernel and identify the neural network computing kernel that is currently in an idle state as an idle computing kernel.
[0103] The determination module 804 is used to determine at least one idle computing core as the target computing core for executing the computing task, and to determine the correspondence between the data processing unit and the target computing core; wherein different data processing units with dependencies correspond to the same target computing core.
[0104] The caching module 805 is used to cache the data processing unit in the launch queue of the corresponding target computing core, and launch the data processing unit to the corresponding target computing core in sequence based on the dependency relationship between different data processing units, so as to trigger the target computing core to perform operation.
[0105] In some embodiments, the parsing module 802 is further configured to obtain, based on the parsing result of the configuration information, the base address of all data processing units for the computing task and the address offset of any data processing unit; and to obtain, based on the base address and the address offset, the storage address of any data processing unit in the memory.
[0106] In some embodiments, the cache module 805 is further configured to determine the operation order between different data processing units that have dependencies; based on the operation order, sequentially retrieve the microinstruction code of the data processing unit from the memory, and sequentially send the data processing unit to the target computing core, so that the target computing core executes the microinstruction code based on the data processing unit.
[0107] In some embodiments, the cache module 805 is further configured to, for any one of the different data processing units, before the target computing core executes the current microinstruction code of the current data processing unit, obtain the execution status of the microinstruction codes of all data processing units preceding the current data processing unit in the operation order; when the execution status indicates that the microinstruction codes of all data processing units have been executed, retrieve the current microinstruction code of the current data processing unit from the memory, and send the current data processing unit to the target computing core.
[0108] In some embodiments, the cache module 805 is further configured to, when there is no dependency between the data processing unit and any other data processing unit, at the start time of the operation of the target computing core, retrieve the microinstruction code of the data processing unit from the memory and send the data processing unit to the target computing core so that the target computing core executes the microinstruction code based on the data processing unit.
[0109] In some embodiments, the computation task execution device 800 is specifically used to adjust the status bit of the target computing core in the status register at the start time of computation of any target computing core, so as to change the running state of the target computing core from an idle state to a busy state.
[0110] Each module in the aforementioned computing task execution device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0111] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores task data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a computational task execution method.
[0112] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0113] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.
[0114] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0115] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0116] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0117] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0119] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for executing a computational task, characterized in that, The method includes: Receive configuration information for computing tasks from the host; The configuration information is parsed to obtain the number of data processing units required to execute the computation task, the storage address of any data processing unit in the memory, and the dependency relationship between different data processing units. Based on the number and the storage address, the data processing unit is retrieved from the memory. Obtain the current running state of any neural network computing kernel, and identify the neural network computing kernel that is currently in an idle state as an idle computing kernel; At least one idle computing core is identified as the target computing core for executing the computational task, and the correspondence between the data processing unit and the target computing core is determined; wherein, different data processing units with dependencies correspond to the same target computing core; The data processing unit is cached in the launch queue of the corresponding target computing core, and based on the dependency relationship between different data processing units, the data processing unit is launched to the corresponding target computing core in sequence to trigger the target computing core to perform operation.
2. The method according to claim 1, characterized in that, The step of parsing the configuration information to obtain the storage address of any data processing unit in the memory includes: Based on the parsing results of the configuration information, obtain the base address of all data processing units for the computing task, and the address offset of any data processing unit; Based on the base address and the address offset, obtain the storage address of any data processing unit in the memory.
3. The method according to claim 1, characterized in that, The method of sequentially sending the data processing units to their corresponding target computing cores based on the dependencies between different data processing units to trigger the target computing cores to perform operations includes: For different data processing units that have dependencies, determine the order of operations between the different data processing units; Based on the aforementioned operation order, the microinstruction code of the data processing unit is sequentially retrieved from the memory, and the data processing unit is sequentially sent to the target computing core, so that the target computing core executes the microinstruction code based on the data processing unit.
4. The method according to claim 3, characterized in that, The step of sequentially retrieving the microinstruction code of the data processing unit from the memory based on the operation order, and sequentially sending the data processing unit to the target computing core, includes: For any current data processing unit among the different data processing units, before the target computing core executes the current microinstruction code of the current data processing unit, the execution status of the microinstruction codes of all data processing units located before the current data processing unit in the operation order is obtained; When the execution state indicates that the microinstruction code of all data processing units has been executed, the current microinstruction code of the current data processing unit is retrieved from the memory, and the current data processing unit is launched to the target computing core.
5. The method according to claim 3, characterized in that, The method further includes: When there is no dependency between the data processing unit and any other data processing unit, at the start of the operation of the target computing core, the microinstruction code of the data processing unit is retrieved from the memory, and the data processing unit is sent to the target computing core so that the target computing core executes the microinstruction code based on the data processing unit.
6. The method according to claim 1, characterized in that, The method further includes: At the start of operation of any target computing core, the status bits of the target computing core are adjusted in the status register to change the running state of the target computing core from idle state to busy state.
7. A computational task execution device, characterized in that, The device includes: The receiving module is used to receive configuration information for computing tasks sent by the host. The parsing module is used to parse the configuration information to obtain the number of data processing units required to execute the computing task, the storage address of any data processing unit in the memory, and the dependency relationship between different data processing units, and to obtain the data processing unit from the memory based on the number and the storage address. The acquisition module is used to acquire the current running state of any neural network computing kernel and identify the neural network computing kernel that is currently in an idle state as an idle computing kernel; A determination module is used to determine at least one idle computing core as the target computing core for executing the computing task, and to determine the correspondence between the data processing unit and the target computing core; wherein different data processing units with dependencies correspond to the same target computing core; The caching module is used to cache the data processing unit in the launch queue of the corresponding target computing core, and launch the data processing unit to the corresponding target computing core in sequence based on the dependency relationship between different data processing units, so as to trigger the target computing core to perform operation.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.