Artificial intelligence computing pipeline instruction execution method and apparatus, and computer device

Abstract

This disclosure provides an AI computing pipeline instruction execution method, apparatus, and computer device, relating to the fields of computer and artificial intelligence technologies. The solution involves: parsing the AI ​​computing instruction stream generated by the compiler and identifying the first, second, and third instructions; generating a pipeline task identifier and associating it with instructions in the same group in response to the second instruction by incrementing the global pipeline index; configuring dependency information for the second instruction; and controlling each hardware unit to execute instructions and updating the count and triggering corresponding execution based on the identifier and dependency information. This achieves asynchronous and ordered execution of memory access, computation, and write-back, reducing pipeline bubbling, lowering hardware logic complexity, adapting to multiple instruction streams, and improving the utilization rate and execution performance of AI computing resources.

Description

Technical Field

[0001] This disclosure relates to the fields of computer and artificial intelligence (AI) technology, particularly to AI chip architecture design, pipeline scheduling control, and especially to an artificial intelligence computing pipeline instruction execution method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] In related technologies, AI computing typically includes three stages: data memory access, core computation, and result write-back. It often adopts a synchronous pipeline architecture for instruction scheduling and achieves task alignment through a global synchronization mechanism. Summary of the Invention

[0003] This disclosure provides a data fitting method, apparatus, computer device, computer-readable storage medium, and computer program product for implementation of application-specific integrated circuits.

[0004] According to one aspect of this disclosure, an artificial intelligence (AI) computing pipeline instruction execution method is provided. The method includes: parsing an AI computing instruction stream generated by a compiler to output instruction recognition results, including: parsing the encoding information of each instruction in the AI ​​computing instruction stream to extract instruction opcodes; identifying a first instruction, a second instruction, and a third instruction based on the instruction opcodes; generating an instruction sequence with pipeline task identifiers in response to a second instruction in the instruction recognition results, including: incrementing a global pipeline index and using the global pipeline index as a pipeline task identifier for the corresponding second instruction; associating the same group of first, second, and third instructions with the pipeline task identifiers; configuring dependency information for each second instruction based on the instruction sequence with pipeline task identifiers, including: obtaining the dependency information for the second instruction. The process involves: setting corresponding prerequisite operation requirements; generating dependency information based on these requirements, whereby the dependency information indicates the prerequisite first instruction completion conditions that must be met before the second instruction can be executed; binding the dependency information to the second instruction; and controlling the first, second, and third hardware execution units to execute corresponding instructions based on the pipeline task identifier and the dependency information. This includes: controlling the first hardware execution unit to execute the first instruction and updating the first instruction completion count corresponding to the pipeline task identifier associated with the first instruction; responding to the first instruction completion count reaching the prerequisite first instruction completion condition indicated by the dependency information of the second instruction corresponding to the pipeline task identifier, controlling the second hardware execution unit to execute the second instruction; and responding to the completion of the second instruction execution, controlling the third hardware execution unit to execute the third instruction.

[0005] In some embodiments, the first instruction is a memory access instruction, the second instruction is a computation instruction, and the third instruction is a write-back instruction; the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit are hardware execution units that operate independently and asynchronously.

[0006] In some embodiments, the global pipeline index is a continuously increasing integer. A new second instruction triggers the global pipeline index to increment. The first instruction in the same group as the second instruction uses the incremented global pipeline index as the pipeline task identifier, and the third instruction in the same group as the second instruction uses the pipeline task identifier.

[0007] In some embodiments, the dependency information is mask information bound to the second instruction, which indicates the number or type of first instructions to be completed before the second instruction is executed.

[0008] In some embodiments, controlling the first hardware execution unit to execute the first instruction includes: checking the occupancy status of the target storage resource; and controlling the first hardware execution unit to execute the first instruction in response to the target storage resource being in an idle state.

[0009] In some embodiments, at least two buffers are used to manage memory access-related storage resources, and the target storage resource is any one of the memory access-related storage resources; when data in one of the at least two buffers is read by the second hardware execution unit, the other buffer in the at least two buffers is used to preload the data corresponding to the next instruction sequence with pipeline task identifier.

[0010] According to another aspect of this disclosure, an artificial intelligence (AI) computing pipeline instruction execution apparatus is provided. The apparatus includes: an instruction parsing unit for parsing an AI computing instruction stream generated by a compiler to output instruction recognition results; the instruction parsing unit includes: a decoding subunit for parsing the encoding information of each instruction in the AI ​​computing instruction stream to extract instruction opcodes; an instruction recognition subunit for recognizing a first instruction, a second instruction, and a third instruction based on the instruction opcodes; a task identifier generation unit for generating an instruction sequence with pipeline task identifiers in response to the second instruction in the instruction recognition results; the task identifier generation unit includes: an index updating subunit for incrementing a global pipeline index and using the global pipeline index as the pipeline task identifier for the corresponding second instruction; an instruction association subunit for associating the same group of first, second, and third instructions with the pipeline task identifiers; and a dependency configuration unit for configuring dependency information for each second instruction based on the instruction sequence with pipeline task identifiers; the dependency configuration unit includes: The system comprises: a requirement acquisition subunit for acquiring the prerequisite operation requirements corresponding to the second instruction; a dependency generation subunit for generating dependency information based on the prerequisite operation requirements, the dependency information indicating the prerequisite first instruction completion conditions that must be met before the second instruction is executed; an information binding subunit for binding the dependency information to the second instruction; and an execution control unit for controlling the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit to execute corresponding instructions based on the pipeline task identifier and dependency information. The execution control unit includes: a first control subunit for controlling the first hardware execution unit to execute the first instruction and updating the first instruction completion count corresponding to the pipeline task identifier associated with the first instruction; a second control subunit for controlling the second hardware execution unit to execute the second instruction in response to the first instruction completion count reaching the prerequisite first instruction completion conditions indicated by the dependency information of the second instruction corresponding to the pipeline task identifier; and a third control subunit for controlling the third hardware execution unit to execute the third instruction in response to the completion of the second instruction.

[0011] According to another aspect of this disclosure, a computer device is provided, comprising: a processor; and a memory having a computer program stored thereon, wherein when executed by the processor, the computer program causes the processor to perform the methods provided above in this disclosure.

[0012] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0013] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0014] According to one or more embodiments of this disclosure, by parsing the AI ​​computing instruction stream, generating pipeline task identifiers with the second instruction as the core, configuring dependency information, and controlling the execution of each hardware unit based on the identifier and dependency information, asynchronous and ordered execution of memory access, computing, and write-back is achieved, reducing pipeline bubbling, reducing hardware logic complexity, flexibly adapting to multiple types of instruction streams, and improving the resource utilization and overall execution performance of AI computing.

[0015] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0016] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0017] Figure 1 This is a flowchart illustrating the AI ​​computing pipeline instruction execution method in some embodiments of this disclosure.

[0018] Figure 2 This is a schematic diagram of the hardware architecture for AI computing pipeline instruction execution in some embodiments of this disclosure.

[0019] Figure 3 This is a schematic block diagram of an AI computing pipeline instruction execution apparatus according to some embodiments of the present disclosure.

[0020] Figure 4 An example computer device is shown in which any of the embodiments described herein may be implemented. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0023] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0024] As mentioned earlier, in related technologies, AI computing typically includes three stages: data access to memory, core computing, and result writing back. It often adopts a synchronous pipeline architecture for instruction scheduling and achieves task alignment through a global synchronization mechanism.

[0025] However, in practical applications, the execution time of each stage of AI computation varies significantly. Memory access operations fluctuate greatly due to bandwidth and latency, and computation operation duration changes with the matrix dimension. Traditional fixed-step synchronous pipelines are prone to generating a large number of cavitation bubbles, resulting in low hardware resource utilization. At the same time, the instruction flow structure of modern AI operators is diverse and not all of them are standard memory access-computation-write-back combinations. Related technologies rely on complex logic when processing non-fixed-step instruction flows, which can easily lead to data conflicts or out-of-order errors.

[0026] Furthermore, traditional global synchronization mechanisms incur excessive hardware logic area overhead when handling a large number of heterogeneous tasks, making it difficult to balance flexibility and high performance. When adapting to different types of AI computing tasks, it is often necessary to redesign or modify the hardware circuitry, increasing development costs and time.

[0027] To address this, this disclosure proposes an AI computing pipeline instruction execution method and apparatus. The method and apparatus parse the AI ​​computing instruction stream generated by the compiler to identify different types of instructions; allocate pipeline task identifiers only based on the second instruction, allowing the same identifier to be shared by the same group of instructions; configure dependency information indicating pre-completion conditions for each second instruction; and control the execution timing of each independent hardware execution unit based on the task identifier and dependency information. This scheme effectively reduces pipeline cavitation, lowers hardware logic complexity, and flexibly adapts to various instruction stream combinations, thereby improving the overall performance of AI computing.

[0028] Exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart illustrating the AI ​​computational pipeline instruction execution method 100 in some embodiments of this disclosure. For example... Figure 1As shown, the AI ​​computing pipeline instruction execution method 100 includes steps S110 to S140.

[0030] In step S110, the AI ​​computation instruction stream generated by the compiler is parsed, and the instruction recognition result is output, including: Sub-step S111: Parse the encoding information of each instruction in the AI ​​calculation instruction stream and extract the instruction opcode; Sub-step S112 identifies the first instruction, the second instruction, and the third instruction based on the instruction opcode.

[0031] In step S110, the instruction parsing process is the starting point of the entire pipeline execution. In some implementations, the AI ​​computation instruction stream is pre-compiled by a compiler and sequentially input into the hardware parsing module for processing according to a preset order. It is understood that the instruction encoding information contains all the functional definitions and operation parameters of the instruction, where the opcode is an identifier field used to distinguish different instruction types. By extracting and parsing the opcode, the functional category of each instruction in the instruction stream can be quickly and accurately identified, providing a foundation for subsequent task partitioning and execution scheduling.

[0032] In step S120, in response to the second instruction in the instruction recognition result, an instruction sequence with a pipeline task identifier is generated, including: Sub-step S121: Increment the global pipeline index and use the global pipeline index as the pipeline task identifier for the corresponding second instruction; Sub-step S122 associates the same group of first, second and third instructions with pipeline task identifiers.

[0033] In step S120, the second instruction is used as the trigger point for partitioning independent computational tasks. In some implementations, the global pipeline index is a global state variable maintained internally in the hardware, used to generate unique task identifiers. It's easy to understand that by associating the same set of instructions with the same pipeline task identifier, instructions from different stages of the same computational task, originally scattered throughout the instruction stream, can be logically integrated into an inseparable whole. This task partitioning method, centered on the second instruction, ensures that pipeline scheduling always revolves around the core computational operations.

[0034] In step S130, based on the instruction sequence with pipeline task identifiers, dependency information is configured for each second instruction, including: Sub-step S131: Obtain the prerequisite operation requirements corresponding to the second instruction; Sub-step S132: Generate dependency information based on the requirements of the preceding operation. The dependency information indicates the conditions for the completion of the preceding first instruction that must be met before the second instruction is executed. Sub-step S133: Bind the dependency information to the second instruction.

[0035] In step S130, dependency information is used to define the execution order relationship between instructions. In some implementations, the pre-operation requirements are directly related to the specific computational logic and data requirements of the second instruction. It is understood that binding dependency information to the second instruction allows the dependency relationship to be passed and processed along with the instruction in the pipeline, eliminating the need to maintain a separate global dependency table. This design makes the definition of dependencies more flexible and adaptable to the differentiated needs of different computational tasks.

[0036] In step S140, based on the pipeline task identifier and dependency information, the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit are controlled to execute corresponding instructions, including: Sub-step S141: Control the first hardware execution unit to execute the first instruction and update the first instruction completion count corresponding to the pipeline task identifier associated with the first instruction; Sub-step S142: In response to the first instruction completion count reaching the dependency information of the second instruction corresponding to the pipeline task identifier, the second hardware execution unit is controlled to execute the second instruction. Sub-step S143, in response to the completion of the second instruction, controls the third hardware execution unit to execute the third instruction.

[0037] In step S140, each hardware execution unit independently executes instructions of the corresponding type according to a unified scheduling logic. In some implementations, the first instruction completion count is a status variable corresponding to each pipeline task identifier, used to record in real time the number of first instructions completed under that task. As an example and not a limitation, when the first instruction completion count meets the conditions indicated by the dependency information, it indicates that all the preparatory data work required for the execution of the second instruction has been completed, and the execution of the second instruction can be safely started at this time.

[0038] The AI ​​computing pipeline instruction execution method provided in the above embodiments of this disclosure, by allocating pipeline task identifiers with a second instruction as the core and using computing tasks as the basic unit of pipeline scheduling, enables asynchronous parallel execution of operations at different execution stages, effectively masking the waiting delays caused by differences in execution time at each stage; by dynamically defining the execution sequence relationship between instructions through dependency information, it eliminates the need for complex global instruction reordering logic, significantly reducing the complexity of hardware implementation; by associating instructions in the same task group through pipeline task identifiers, it can flexibly adapt to instruction streams with different combinations, meeting the needs of various AI computing tasks without changing the core scheduling logic.

[0039] In some embodiments, the first instruction is a memory access instruction, the second instruction is a computation instruction, and the third instruction is a write-back instruction; the first, second, and third hardware execution units are independent and asynchronously operating hardware execution units. As an example, and not a limitation, the memory access instruction loads the data required for computation from external memory into the internal storage area, the computation instruction performs the core matrix operation, and the write-back instruction stores the computation result back to external memory. Each of the independent and asynchronously operating hardware execution units has its own independent control logic and data path, enabling them to complete tasks according to their own execution rhythm without needing to execute synchronously in a fixed sequence. This decouples the operations of different functions, allowing operations at each stage to proceed in parallel, effectively improving the utilization of hardware resources.

[0040] In some embodiments, the global pipeline index is a continuously incrementing integer. A new second instruction triggers the global pipeline index to increment. The first instruction in the same group as the second instruction uses the incremented global pipeline index as the pipeline task identifier, and the third instruction in the same group uses the pipeline task identifier. As an example, and not a limitation, the global pipeline index starts from an initial value, and its value increases by 1 each time a new second instruction is identified, generating a unique pipeline task identifier. All instructions within the same group are marked with the same pipeline task identifier, ensuring that instructions from different task groups are not confused. This incremental task identifier generation method is logically simple and intuitive, has extremely low hardware implementation costs, and can quickly establish a mapping relationship between instructions and task groups.

[0041] In some embodiments, the dependency information is mask information bound to the second instruction, indicating the number or type of first instructions to be completed before the second instruction is executed. The mask information and the corresponding second instruction are transmitted and processed together in the pipeline, and the hardware can dynamically determine the executable timing of the second instruction by parsing the mask information. Different second instructions can be configured with different mask information to adapt to their respective preceding data requirements. This allows the synchronization logic to be transferred from hardware circuitry to the software configuration level, eliminating the need for hard-coded fixed synchronization rules and significantly improving the flexibility of the scheduling mechanism.

[0042] In some embodiments, controlling the first hardware execution unit to execute the first instruction includes checking the occupancy status of the target storage resource, and controlling the first hardware execution unit to execute the first instruction in response to the target storage resource being in an idle state. Before initiating the data loading operation, the system first queries the usage status of the target storage resource to confirm whether the resource has been released by a previous task. Only when the target storage resource is available will new data be allowed to be written to that resource. It can be understood that by checking the occupancy status of the storage resource in advance, it is possible to prevent new data from overwriting old data that has not yet been used, completely eliminate read-after-write data conflicts, and ensure the consistency and correctness of data during the computation process.

[0043] In some embodiments, at least two buffers are used to manage memory access-related storage resources, with the target storage resource being any one of the buffers in the memory access-related storage resources. When data in one of the at least two buffers is read by a second hardware execution unit, the other buffer is used to preload data corresponding to the next instruction sequence with a pipeline task identifier. Multiple buffers can alternately perform data loading and data reading operations; while data in one buffer is being used for computation, another buffer can preload data required by the next task. This allows data loading operations and computation operations to overlap in time, effectively masking the inherent latency of memory access operations and improving the overall pipeline throughput.

[0044] Figure 2 This is a schematic diagram of a hardware architecture 200 for AI computation pipeline instruction execution in some embodiments of this disclosure. For example... Figure 2 As shown, the hardware architecture 200 may include an instruction stream input 201, a decoder 210, an index allocation and mask configuration module 220, a memory access module 230, a calculation module 240, a write-back module 250, a ping-pong buffer 260, and a memory 270.

[0045] In the example, instruction stream input 201 is used to input a continuous sequence of AI computation instructions to decoder 210. In some implementations, instruction stream input 201 is pre-generated by a compiler and sequentially input to decoder 210 for processing according to a preset execution order.

[0046] In the example, decoder 210 is communicatively connected to instruction stream input 201 to receive and parse each instruction in instruction stream input 201. In some embodiments, decoder 210 decodes the input instruction bit by bit, extracts the opcode and operand information of the instruction, distinguishes different types of instructions, and sends the decoded instruction information to index allocation and mask configuration module 220.

[0047] In the example, the index allocation and mask configuration module 220 is communicatively connected to the decoder 210 and is used to allocate pipeline task identifiers to computation instructions and configure corresponding dependency information for each computation instruction. In some implementations, the index allocation and mask configuration module 220 increments the global pipeline index and generates a new pipeline task identifier only when a computation instruction is detected. Memory access instructions and write-back instructions in the same group as the computation instruction all use this pipeline task identifier. The index allocation and mask configuration module 220 packages the index, instruction, and dependency information and sends them to the memory access module 230, the computation module 240, and the write-back module 250, respectively.

[0048] In the example, the memory access module 230 is communicatively connected to the index allocation and mask configuration module 220, memory 270, and ping-pong buffer 260, and is used to execute external data loading operations corresponding to memory access instructions. In some embodiments, the memory access module 230 receives the index and memory access instruction sent by the index allocation and mask configuration module 220, reads the data required for calculation from memory 270, and writes the read data into the ping-pong buffer 260. After completing data loading, the memory access module 230 sends a completion signal containing the corresponding index to the calculation module 240.

[0049] In the example, the calculation module 240 is communicatively connected to the index allocation and mask configuration module 220, the memory access module 230, the ping-pong buffer 260, and the write-back module 250, and is used to execute the core matrix operation corresponding to the calculation instruction. In some embodiments, the calculation module 240 receives the index, calculation instruction, and dependency information sent by the index allocation and mask configuration module 220. When the index corresponding to the received memory access completion signal is consistent with the index of the current calculation instruction, and the memory access completion count meets the requirements of the dependency information, it reads data from the ping-pong buffer 260 for calculation. After the calculation is completed, the calculation module 240 writes the result to the ping-pong buffer 260 and sends a completion signal containing the corresponding index to the write-back module 250.

[0050] In the example, the write-back module 250 is communicatively connected to the index allocation and mask configuration module 220, the calculation module 240, the ping-pong buffer 260, and the memory 270, and is used to execute the result storage operation corresponding to the write-back instruction. In some embodiments, the write-back module 250 receives the index and write-back instruction sent by the index allocation and mask configuration module 220. When it receives the completion signal of the corresponding index sent by the calculation module 240, it reads the calculation result from the ping-pong buffer 260 and writes the result back to the memory 270.

[0051] In the example, the ping-pong buffer 260 is communicatively connected to the memory access module 230, the calculation module 240, and the write-back module 250, and is used to cache the input data loaded by the memory access module 230 and the output data generated by the calculation module 240. In some implementations, the ping-pong buffer 260 uses at least two independent buffers for data management, and the system maintains the occupancy status of each buffer to ensure that the data is not overwritten by new data before it is fully used.

[0052] In the example, memory 270 is communicatively connected to memory access module 230 and write-back module 250, and is used to store the raw data required for AI calculations and the result data after the calculations are completed. It can be understood that memory 270 is an external storage device, and memory access module 230 and write-back module 250 interact with memory 270 to perform external read and write operations on the data.

[0053] exist Figure 2 In the AI ​​computing pipeline instruction execution hardware architecture 200 shown, the modules work collaboratively to achieve asynchronous, ordered execution of AI computing instructions in a pipelined manner. Hardware architecture 200 identifies different types of instructions through instruction parsing, allocates a global pipeline index as a task identifier based on the computing instruction, and configures dependency information for each computing instruction. The memory access module, computing module, and write-back module independently and asynchronously execute corresponding operations according to the scheduling of task identifiers and dependency information, ultimately achieving efficient parallel processing of memory access, computing, and write-back stages. While ensuring data consistency and execution order, it effectively masks memory access latency and improves overall computing performance.

[0054] It should be noted that, Figure 2 The hardware architecture 200 and its module division shown are merely illustrative, and this disclosure is not limited to the specific structure shown in the figure. Without departing from the concept of this disclosure, those skilled in the art can merge, split, add, delete, or adjust the functions of the modules. For example, the functions of the index allocation and mask configuration module can be integrated into the decoder, or the specific implementation of the memory access module, calculation module, and write-back module can be adjusted. All such changes fall within the protection scope of this disclosure.

[0055] Figure 3 This is a schematic block diagram of an AI computing pipeline instruction execution apparatus 300 according to some embodiments of this disclosure. For example... Figure 3 As shown, the AI ​​computing pipeline instruction execution device 300 includes: Instruction parsing unit 310 is used to parse the AI ​​computation instruction stream generated by the compiler and output the instruction recognition result. Instruction parsing unit 310 includes: Decoding subunit 311 is used to parse the encoding information of each instruction in the AI ​​calculation instruction stream and extract the instruction opcode; The instruction recognition subunit 312 is used to identify the first instruction, the second instruction, and the third instruction based on the instruction opcode; The task identifier generation unit 320 is used to generate an instruction sequence with pipeline task identifiers in response to the second instruction in the instruction recognition result. The task identifier generation unit 320 includes: The index update subunit 321 is used to increment the global pipeline index and use the global pipeline index as the pipeline task identifier for the corresponding second instruction. The instruction association subunit 322 is used to associate the same group of first instructions, second instructions and third instructions with pipeline task identifiers; Dependency configuration unit 330 is used to configure dependency information for each second instruction based on the instruction sequence with pipeline task identifier. Dependency configuration unit 330 includes: The requirement acquisition subunit 331 is used to acquire the prerequisite operation requirements corresponding to the second instruction. Dependency generation subunit 332 is used to generate dependency information according to the requirements of the preceding operation. The dependency information indicates the conditions for the completion of the preceding first instruction that must be met before the second instruction is executed. Information binding subunit 333 is used to bind dependency information to the second instruction; Execution control unit 340 is used to control the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit to execute corresponding instructions based on pipeline task identifiers and dependency information. Execution control unit 340 includes: The first control subunit 341 is used to control the first hardware execution unit to execute the first instruction and update the first instruction completion count corresponding to the pipeline task identifier associated with the first instruction; The second control subunit 342 is used to control the second hardware execution unit to execute the second instruction in response to the first instruction completion condition indicated by the dependency information of the second instruction corresponding to the pipeline task identifier when the first instruction completion count reaches the second instruction completion condition. The third control subunit 343 is used to control the third hardware execution unit to execute the third instruction in response to the completion of the second instruction.

[0056] The AI ​​computing pipeline instruction execution device provided in the above embodiments of this disclosure, by allocating pipeline task identifiers with a second instruction as the core, and using computing tasks as the basic unit of pipeline scheduling, enables asynchronous parallel execution of operations at different execution stages, effectively masking the waiting delays caused by differences in execution time at each stage; by dynamically defining the execution sequence relationship between instructions through dependency information, it eliminates the need for complex global instruction reordering logic, significantly reducing the complexity of hardware implementation; by associating instructions in the same task group through pipeline task identifiers, it can flexibly adapt to instruction streams with different combinations, meeting the needs of various AI computing tasks without changing the core scheduling logic.

[0057] In some embodiments, the first instruction is a memory access instruction, the second instruction is a computation instruction, and the third instruction is a write-back instruction; the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit are independent and asynchronous hardware execution units. This decouples the operations of different functions, allowing operations at each stage to proceed in parallel, effectively improving the utilization of hardware resources.

[0058] In some embodiments, the global pipeline index is a continuously incrementing integer. A new second instruction triggers the global pipeline index to increment. The first instruction in the same group as the second instruction uses the incremented global pipeline index as the pipeline task identifier, and the third instruction in the same group as the second instruction uses the pipeline task identifier. This incremental task identifier generation method is logically simple and intuitive, has extremely low hardware implementation cost, and can quickly establish a mapping relationship between instructions and task groups.

[0059] In some embodiments, the dependency information is mask information bound to the second instruction, which indicates the number or type of first instructions to be completed before the second instruction is executed. This mask-based dependency configuration method can transfer synchronization logic from hardware circuits to the software configuration level, eliminating the need for hard-coded fixed synchronization rules and significantly improving the flexibility of the scheduling mechanism.

[0060] In some embodiments, controlling the first hardware execution unit to execute the first instruction includes checking the occupancy status of the target storage resource, and in response to the target storage resource being in an idle state, controlling the first hardware execution unit to execute the first instruction. By checking the storage resource occupancy status in advance, it is possible to prevent new data from overwriting old data that has not yet been used, completely eliminate read-after-write data conflicts, and ensure the consistency and correctness of data during the computation process.

[0061] In some embodiments, at least two buffers are used to manage memory access-related storage resources, with the target storage resource being any one of the buffers in the memory access-related storage resources. When data in one of the at least two buffers is read by a second hardware execution unit, the other buffer is used to preload data corresponding to the next instruction sequence with a pipeline task identifier. This multi-buffer management mechanism enables data loading operations and computation operations to overlap in time, effectively masking the inherent latency of memory access operations and improving the overall pipeline throughput.

[0062] The operations of the aforementioned instruction parsing unit 310, task identifier generation unit 320, dependency configuration unit 330, execution control unit 340, and their respective subunits can be combined. Figure 1 The steps S110, S120, S130, and S140, and their respective sub-steps, are similar, so the details of each aspect will not be repeated here.

[0063] According to one aspect of this disclosure, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0064] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0065] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0066] Figure 4 An example computer device 400 is shown in which any of the embodiments described herein may be implemented. Computer device 400 may be used to implement one or more components of the systems and methods described above. Computer device 400 may include a bus 402 or other communication mechanism for communicating information, and one or more processors 404 coupled to the bus 402 for processing information. Processor 404 may be, for example, one or more general-purpose microprocessors.

[0067] Computer device 400 may also include main memory 406, such as random access memory (RAM), cache, and / or other dynamic storage devices, coupled to bus 402, for storing information and instructions to be executed by processor 404. Main memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 404. Such instructions, when stored in a storage medium accessible to processor 404, can make computer device 400 a special-purpose machine customized to perform the operations specified in the instructions. Main memory 406 may include non-volatile media and / or volatile media. Non-volatile media may include, for example, optical discs or magnetic disks. Volatile media may include dynamic memory. Common media formats may include, for example, floppy disks, collapsible disks, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage media, CD-ROMs (read-only optical disc drives), any other optical data storage media, any physical media with a perforated arrangement, RAM (random access memory), DRAM (dynamic random access memory), PROM (programmable read-only memory) and EPROM (erasable programmable read-only memory), FLASH-EPROM (fast erase programmable read-only memory), NVRAM (non-volatile random access memory), any other memory chips or tape cartridges, or network versions of the above.

[0068] Computer device 400 may implement the techniques described herein using custom hardwired logic, one or more ASICs (Application-Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays), firmware, and / or program logic, which, when combined with computer device 400, enable computer device 400 to become a special-purpose machine or to be programmed therein. According to one embodiment, the techniques herein are executed by computer device 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Such instructions may be read into main memory 406 from another storage medium, such as storage device 408. Executing the sequence of instructions contained in main memory 406 causes processor 404 to perform the processing steps described herein. For example, the processes / methods disclosed herein may be implemented by computer program instructions stored in main memory 406. When these instructions are executed by processor 404, they may perform the steps shown in the corresponding figures and as described above. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0069] Computer device 400 also includes a network interface 410 coupled to bus 402. Network interface 410 can provide bidirectional data communication coupled to one or more network links connected to one or more networks. As another example, network interface 410 can be a local area network (LAN) card to provide data communication connectivity with a compatible LAN (or a WAN component communicating with a WAN (wide area network)). Wireless links can also be implemented.

[0070] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or processor-implemented engine may be distributed across many geographic locations.

[0071] Each process, method, and algorithm described in the preceding sections can be embodied in a code module executed by one or more computer systems or computer processors including computer hardware, and can be fully or partially automated by them. These processes and algorithms can be implemented, in part or in whole, in a specific application circuit.

[0072] When the functions disclosed herein are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, can be embodied in the form of a software product. This software product can be stored in a storage medium and includes instructions to cause a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods described in the embodiments of this application. The storage medium may include a flash drive, a portable hard drive, ROM, RAM, a magnetic disk, an optical disk, another medium suitable for storing program code, or any combination thereof.

[0073] The embodiments disclosed herein can be implemented via a cloud platform, server, or group of servers that interact with a client. The client can be a terminal device or a client registered by a user on the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can install platform applications.

[0074] The various features and processes described above can be used independently or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in a non-specifically disclosed order, or multiple blocks or states may be combined in a single block or state. Exemplary blocks or states may be executed serially, in parallel, or otherwise. Blocks or states may be added to or removed from the disclosed exemplary embodiments. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the disclosed exemplary embodiments.

[0075] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. An algorithm may consist of program code or instructions stored in memory (such as the non-transitory computer-readable storage medium described above). Such an algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not be explicitly programmed into the computer to perform the function, but may learn from training data to obtain a predictive model for performing that function.

[0076] The various operations of the exemplary methods described herein can be performed at least in part by one or more processors, which are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute the engine of a processor implementation whose operation is to perform one or more of the operations or functions described herein.

[0077] Similarly, the methods described herein can be implemented at least partially by a processor, where a specific processor or one or more processors are examples of hardware. For example, at least some operations of the methods can be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors can also run in a “cloud computing” environment or as “Software as a Service” (SaaS) to support the execution of the relevant operations. For example, at least some operations can be performed by a group of computers (as an example of a machine including processors), which can be accessed via a network (e.g., the Internet) and through one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0078] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or processor-implemented engine may be distributed across many geographic locations.

[0079] In this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are described and illustrated as independent operations, one or more individual operations may be performed concurrently, and these operations are not required to be performed in the order shown. Structures and functionalities presented as independent components in the example configuration may be implemented as combined structures or components. Similarly, structures and functionalities presented as individual components may be implemented as independent components. These and other variations, modifications, additions, and improvements are all within the scope of this document.

[0080] As used herein, “or” is inclusive rather than exclusive unless explicitly stated or indicated by context. Furthermore, “and” is both common and individual unless explicitly stated or indicated by context. Moreover, multiple instances may be provided for the resources, operations, or structures described herein as a single example. Furthermore, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are illustrated within the context of a particular illustrative configuration. The allocation of other functionalities is conceivable and may fall within the scope of various embodiments of this disclosure. Generally, structures and functionalities presented as independent resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functionalities presented as individual resources may be implemented as independent resources. These and other variations, modifications, additions, and improvements are all within the scope of embodiments of this disclosure. Therefore, this specification and accompanying drawings should be viewed in an illustrative rather than restrictive sense.

[0081] The terms “comprising” or “including” are used to indicate the presence of a subsequently stated feature, but do not preclude the addition of other features. Conditional language, in particular, such as “may,” “can,” or “may,” unless specifically stated or otherwise understood in the context of use, is generally intended to express that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Therefore, such conditional language generally does not imply that a feature, element, and / or step is necessary in any way for one or more embodiments, or that one or more embodiments must include logic that, with or without user input or prompting, determines whether such features, elements, and / or steps are included in any particular embodiment, or whether they are to be performed in any particular embodiment.

Claims

1. A method for executing instructions in an artificial intelligence (AI) computing pipeline, characterized in that, The method includes: Parse the AI ​​computation instruction stream generated by the compiler to output instruction recognition results, including: The encoding information of each instruction in the AI ​​computing instruction stream is parsed to extract the instruction opcode; Based on the instruction opcode, identify the first instruction, the second instruction, and the third instruction; In response to the second instruction in the instruction recognition result, a sequence of instructions with pipeline task identifiers is generated, including: Increment the global pipeline index and use the global pipeline index as the pipeline task identifier corresponding to the second instruction; Associate the same set of first, second, and third instructions with the pipeline task identifier; Based on the instruction sequence with pipeline task identifier, dependency information is configured for each second instruction, including: Obtain the prerequisite operation requirements corresponding to the second instruction; Dependency information is generated based on the prerequisite operation requirements, and the dependency information indicates the prerequisite first instruction completion conditions that must be met before the second instruction is executed. Bind the dependency information to the second instruction; Based on the pipeline task identifier and the dependency information, the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit are controlled to execute corresponding instructions, including: Control the first hardware execution unit to execute the first instruction, and update the first instruction completion count corresponding to the pipeline task identifier associated with the first instruction; In response to the completion condition of the first instruction indicated by the dependency information of the second instruction corresponding to the pipeline task identifier when the first instruction completion count reaches the first instruction completion count, the second hardware execution unit is controlled to execute the second instruction. In response to the completion of the second instruction, the third hardware execution unit is controlled to execute the third instruction.

2. The method according to claim 1, characterized in that, The first instruction is a memory access instruction, the second instruction is a computation instruction, and the third instruction is a write-back instruction; the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit are hardware execution units that operate independently and asynchronously.

3. The method according to claim 1, characterized in that, The global pipeline index is a continuously increasing integer. A new second instruction triggers the global pipeline index to increment. The first instruction in the same group as the second instruction uses the incremented global pipeline index as the pipeline task identifier. The third instruction in the same group as the second instruction uses the pipeline task identifier.

4. The method according to claim 1, characterized in that, The dependency information is mask information bound to the second instruction, and the mask information indicates the number or type of first instructions to be completed before the second instruction is executed.

5. The method according to claim 1, characterized in that, Controlling the first hardware execution unit to execute the first instruction includes: Check the occupancy status of the target storage resources; In response to the target storage resource being idle, the first hardware execution unit is controlled to execute the first instruction.

6. The method according to claim 5, characterized in that, At least two buffers are used to manage memory access-related storage resources, and the target storage resource is any one of the memory access-related storage resources; when data in one of the at least two buffers is read by the second hardware execution unit, the other buffer in the at least two buffers is used to preload the data corresponding to the next instruction sequence with pipeline task identifier.

7. An artificial intelligence (AI) computing pipeline instruction execution device, characterized in that, The device includes: An instruction parsing unit is used to parse the AI ​​computation instruction stream generated by the compiler to output instruction recognition results. The instruction parsing unit includes: The decoding subunit is used to parse the encoded information of each instruction in the AI ​​computing instruction stream in order to extract the instruction opcode; The instruction recognition subunit is used to identify the first instruction, the second instruction, and the third instruction based on the instruction opcode; A task identifier generation unit, configured to generate an instruction sequence with pipeline task identifiers in response to a second instruction in the instruction recognition result, wherein the task identifier generation unit includes: The index update subunit is used to increment the global pipeline index and use the global pipeline index as the pipeline task identifier for the corresponding second instruction. The instruction association subunit is used to associate the same group of first, second, and third instructions with pipeline task identifiers. A dependency configuration unit, used to configure dependency information for each second instruction based on the instruction sequence with pipeline task identifiers, the dependency configuration unit comprising: The requirement acquisition subunit is used to acquire the prerequisite operation requirements corresponding to the second instruction. A dependency generation subunit is used to generate dependency information based on the requirements of the preceding operation, wherein the dependency information indicates the conditions for the completion of the preceding first instruction that must be met before the execution of the second instruction. The information binding subunit is used to bind dependency information to the second instruction; The execution control unit is configured to control the first hardware execution unit, the second hardware execution unit, and the third hardware execution unit to execute corresponding instructions based on pipeline task identifiers and dependency information. The execution control unit includes: The first control subunit is used to control the first hardware execution unit to execute the first instruction and update the first instruction completion count corresponding to the pipeline task identifier associated with the first instruction; The second control subunit is used to control the second hardware execution unit to execute the second instruction in response to the first instruction completion condition indicated by the dependency information of the second instruction corresponding to the pipeline task identifier when the first instruction completion count reaches the second instruction completion condition. The third control subunit is used to control the third hardware execution unit to execute the third instruction in response to the completion of the second instruction.

8. A computer device, characterized in that, The computer device includes: processor; A memory having a computer program stored thereon, wherein, when executed by the processor, the computer program causes the processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-6.

Images