Instruction scheduling method and apparatus, electronic device, and storage medium
By encapsulating instructions into a unified structure in asynchronous mode and utilizing a circular scheduling queue and parameter object cyclic reuse mechanism, the performance overhead of instruction scheduling in high-load scenarios is solved, improving the throughput efficiency and execution continuity of the computing system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU TSINGMICRO INTELLIGENT TECH CO LTD
- Filing Date
- 2026-05-19
- Publication Date
- 2026-06-16
AI Technical Summary
In high-performance computing systems, existing instruction scheduling processes suffer from performance overhead due to frequent dynamic memory allocation and reclamation under high load scenarios, increased context switching due to lock contention, and the impact of single instruction wake-up mechanisms on execution continuity, resulting in high system throughput and latency.
In asynchronous mode, the instructions to be processed in the processor are encapsulated into a first instruction structure according to a preset instruction structure. The first instruction structure is carried by an idle parameter object and written into a circular scheduling queue. When the conditions are met, the consumer thread extracts and executes the instructions in batches and marks the executed parameter object as idle, thereby realizing the circular reuse of the parameter object.
It reduces the performance overhead of instruction scheduling, improves system throughput and the collaborative efficiency between the processor and computing accelerator, reduces the resource consumption of frequent thread wake-ups, and ensures the stability of memory management.
Smart Images
Figure CN122220076A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to an instruction scheduling method and apparatus, electronic device and storage medium. Background Technology
[0002] In collaborative computing architectures involving a Central Processing Unit (CPU) and computing accelerators (such as Neural-network Processing Units (NPUs), Graphics Processing Units (GPUs), and Digital Signal Processors (DSPs), the processor typically submits computational task instructions to the accelerator through the runtime system. With the increasing prevalence of high-performance computing and heterogeneous computing architectures, the demand for high-frequency task submission and concurrent scheduling is constantly increasing, placing higher requirements on throughput efficiency and low latency in the system.
[0003] The instruction scheduling process in related technologies typically includes dynamically allocating instruction parameter structures, adding instructions to the scheduling queue, performing thread synchronization based on locks or semaphores, having consumer threads retrieve and execute instructions one by one, and implementing central processing unit waiting through additional mechanisms.
[0004] However, the instruction scheduling process in related technologies suffers from the following problems under high-load scenarios: frequent dynamic memory allocation and reclamation incur significant performance overhead; lock contention leads to increased context switching; the single instruction wake-up mechanism causes frequent scheduling of consumer threads, affecting execution continuity; and synchronization control logic increases system complexity and maintenance costs. Therefore, improving the efficiency of instruction scheduling and reducing the performance overhead during the instruction scheduling process are problems that need to be solved. Summary of the Invention
[0005] This application provides an instruction scheduling method, apparatus, electronic device, and storage medium. Its main objective is to address how to improve the efficiency of instruction scheduling and reduce the performance overhead during the instruction scheduling process.
[0006] According to a first aspect of this application, an instruction scheduling method is provided, comprising:
[0007] The instruction to be processed in the processor is encapsulated into a first instruction structure according to a preset instruction structure. The first instruction structure includes at least instruction parameters for performing the operation corresponding to the instruction to be processed. The scheduling mode in response to the pending instruction is asynchronous. It obtains an idle parameter object from a set of preset parameter objects, carries the first instruction structure through the idle parameter object, and writes the first instruction structure into the circular scheduling queue. In response to the number of first instruction structures in the circular scheduling queue meeting the preset instruction processing conditions, a consumer thread is activated. The consumer thread is used to extract at least one first instruction structure from the circular scheduling queue and execute the operation corresponding to each of the at least one first instruction structure. The parameter object corresponding to the first instruction structure after execution will be re-identified as an idle parameter object.
[0008] According to a second aspect of this application, an instruction scheduling apparatus is provided, comprising: The encapsulation module is used to encapsulate the instruction to be processed in the processor into a first instruction structure according to a preset instruction structure. The first instruction structure includes at least instruction parameters for performing the operation corresponding to the instruction to be processed. The acquisition module is used to respond to the asynchronous mode of the scheduling mode of the instruction to be processed, to acquire an idle parameter object from a set of preset parameter objects, to carry the first instruction structure through the idle parameter object, and to write the first instruction structure into the circular scheduling queue. The activation module is used to activate the consumer thread in response to the number of first instruction structures in the circular scheduling queue meeting the preset instruction processing conditions. The consumer thread is used to extract at least one first instruction structure from the circular scheduling queue and execute the operation corresponding to each of the at least one first instruction structure. The reset module is used to re-identify the parameter object corresponding to the first instruction structure after execution as an idle parameter object.
[0009] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, such that the at least one processor is able to perform the method described in the first aspect above.
[0010] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method of the first aspect described above.
[0011] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method as described in the first aspect above.
[0012] The instruction scheduling method, apparatus, electronic device, and storage medium provided in this application, by obtaining idle parameter objects from a preset set of parameter objects in asynchronous mode to carry the encapsulated instruction structure and writing the instruction structure into a circular scheduling queue, can perform cyclic scheduling of instructions based on the cyclic reuse of parameter objects, avoiding the overhead of high-frequency dynamic memory allocation. When the number of instructions in the circular scheduling queue meets a preset condition, a consumer thread is activated to batch extract and execute instructions, reducing the resource consumption of frequent thread wake-ups. After the instruction is executed, the corresponding parameter object is re-marked as idle, ensuring the stability of memory management. Therefore, this application can reduce the performance loss during instruction scheduling and improve system throughput efficiency.
[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0014] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein: Figure 1 A flowchart illustrating an instruction scheduling method provided in an embodiment of this application; Figure 2 This application provides an overall block diagram of an asynchronous instruction execution system. Figure 3 This is a schematic diagram of a program synchronization mechanism provided in an embodiment of this application; Figure 4 A block diagram of a lock-free object pool structure based on CAS provided in an embodiment of this application; Figure 5 This is a schematic diagram of a batch ring instruction queue structure provided in an embodiment of this application; Figure 6 This is a schematic diagram of a production thread instruction submission process provided in an embodiment of this application; Figure 7 This is a schematic diagram of a batch execution process for a consumer thread provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an instruction scheduling device provided in an embodiment of this application; Figure 9 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0015] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0016] This application provides an instruction scheduling method applicable to computing system environments requiring high-frequency task submission, high-concurrency scheduling, and high-throughput execution capabilities, such as heterogeneous computing systems containing a main processor and computing accelerators, including but not limited to: CPU and NPU collaborative computing systems, CPU and GPU collaborative computing systems, CPU and DSP collaborative computing systems, FPGA acceleration systems, and ASIC (Application-Specific Integrated Circuit) acceleration platforms. High-performance computing and data-intensive systems, including but not limited to: scientific computing platforms, parallel computing frameworks, big data processing systems, graph computing systems, and streaming computing systems. Other high-concurrency task execution systems, including but not limited to: task execution systems involving high-frequency task submission, multi-threaded concurrent execution, task queue scheduling, and synchronization control mechanisms.
[0017] The instruction scheduling method of this application can run in the runtime system or driver layer to improve the system throughput and response efficiency in high-frequency instruction submission scenarios.
[0018] The instruction scheduling method, apparatus, electronic device, and storage medium of embodiments of this application are described below with reference to the accompanying drawings.
[0019] Figure 1 This is a flowchart illustrating an instruction scheduling method provided in an embodiment of this application.
[0020] like Figure 1 As shown, the method includes the following steps: Step 101: The instruction to be processed in the processor is encapsulated into a first instruction structure according to a preset instruction structure. The first instruction structure includes at least instruction parameters for performing the operation corresponding to the instruction to be processed.
[0021] In the embodiments of this application, the preset instruction structure is a predefined unified data encapsulation format that is compatible with various types of instructions. That is, the instruction structure supports the extension of different types of instructions, including but not limited to calculation instructions, data handling instructions, and synchronization control instructions. Through the preset instruction structure, various instructions follow a unified processing logic during scheduling, which can reduce the branch processing overhead caused by differences in instruction types. In other words, all types of instructions to be processed are encapsulated into a unified Command structure. The preset instruction structure includes, for example, a structure including an instruction type field, a parameter pointer, and an execution status field.
[0022] The first instruction structure is the data encapsulated according to a preset instruction structure. The first instruction structure contains at least instruction parameters. Instruction parameters refer to the operation-related data required to execute the instruction to be processed.
[0023] The first instruction structure may also include an instruction type field and an execution status field. The instruction type field is used to distinguish whether the current instruction is a computation instruction or a synchronous instruction, while the execution status field is used to record the completion status or exception information of the instruction during execution.
[0024] 102. In response to the asynchronous mode of the scheduling mode of the instruction to be processed, an idle parameter object is obtained from multiple preset parameter objects, the first instruction structure is carried by the idle parameter object, and the first instruction structure is written into the circular scheduling queue.
[0025] In the embodiments of this application, the scheduling mode refers to different execution paths preset for executing the instructions to be processed, including synchronous mode and asynchronous mode.
[0026] In asynchronous mode, the processor can process other tasks without waiting for the instructions to complete after submitting them. The scheduling and execution of instructions are handled by background threads.
[0027] Therefore, once the scheduling mode of the instruction to be processed is determined to be asynchronous, a cyclic scheduling process will be performed based on the parameter object. That is, a currently unoccupied free parameter object will be obtained from multiple pre-allocated parameter objects. After successfully obtaining a free parameter object, the first instruction structure will be carried by the free parameter object, that is, the first instruction structure will be filled into the memory space of the free parameter object. Subsequently, the parameter object carrying the first instruction structure will be written into the circular scheduling queue.
[0028] Here, multiple parameter objects refer to a fixed-size pool of parameter objects created by the system during the initialization phase. Each parameter object in the pool can be used to carry the first instruction structure. Idle parameter objects are those in the parameter object pool that are not currently occupied by any pending instruction. The process of acquiring an idle parameter object can be implemented through atomic operations or a lock-free Compare-And-Swap (CAS) mechanism.
[0029] A circular scheduling queue is an instruction buffer queue with a circular buffer structure. This circular scheduling queue is a contiguous memory space, and read / write positions are determined by head and tail pointers. Because the circular scheduling queue has a fixed capacity and circular reuse characteristics, its enqueue operation only involves the atomic movement of the tail pointer and the writing of data to the corresponding slot, eliminating the need for frequent dynamic memory allocation and deallocation.
[0030] Step 103: In response to the number of first instruction structures in the circular scheduling queue meeting the preset instruction processing conditions, the consumer thread is activated. The consumer thread is used to extract at least one first instruction structure from the circular scheduling queue and execute the operation corresponding to each of the at least one first instruction structure.
[0031] In the embodiments of this application, the preset instruction processing condition refers to the triggering condition for batch processing instructions in the circular scheduling queue. The preset instruction processing condition includes, but is not limited to: the number of first instruction structures that have been enqueued in the circular scheduling queue but have not yet been retrieved reaches or exceeds a preset quantity threshold; the time elapsed since the last time the consumer thread was awakened reaches a preset time threshold, even if the number of instructions in the queue has not reached the quantity threshold.
[0032] Consumer threads are background threads created during system initialization to retrieve instructions from the circular scheduling queue and drive the computation accelerator to execute them. When inactive, consumer threads are in a dormant state and do not consume processor computing resources. Once activated, a consumer thread retrieves at least one first instruction structure from the circular scheduling queue; specifically, it can retrieve multiple first instruction structures at once, forming a batch of instruction sequences.
[0033] The extraction operation is implemented by atomically moving the queue head pointer, ensuring data consistency and lock-free synchronization with the producer thread. After extraction, the consumer thread initiates the corresponding hardware operation to the computing accelerator based on the instruction parameters in each extracted first instruction structure. Due to the batch extraction and execution approach, the consumer thread can process multiple instructions continuously, reducing the frequency of thread switching.
[0034] Step 104: Re-identify the parameter object corresponding to the executed first instruction structure as an idle parameter object.
[0035] In the embodiments of this application, after the consumer thread completes the execution of the operation corresponding to any first instruction structure, the parameter object used by the first instruction structure is no longer occupied. At this time, the parameter object corresponding to the executed first instruction structure is re-marked as an idle parameter object and returned to the parameter object pool.
[0036] The re-identification process can also be accomplished using atomic operations or a CAS mechanism. After being re-identified as an idle parameter object, the parameter object can be retrieved and reused by subsequent newly arriving instructions. Through the circular reuse mechanism, the total amount of the parameter object pool remains constant during system operation, eliminating the need to dynamically allocate new memory with each instruction submission or to release memory after instruction execution.
[0037] Meanwhile, to facilitate understanding of the asynchronous scheduling mode process in the embodiments of this application, the embodiments of this application provide an overall block diagram of an asynchronous instruction execution system, such as... Figure 2 As shown, it includes: The instruction parameter management, instruction scheduling, and synchronization mechanisms are designed in a unified manner; Frequent dynamic allocation is eliminated through a parameter object recycling mechanism; Reduce thread switching overhead through batch scheduling mechanisms; Program-level synchronization is achieved through embedded synchronization instructions; It supports switching between synchronous and asynchronous execution paths through a unified encapsulation structure.
[0038] This application provides a low-latency, high-throughput instruction scheduling and execution method for computing accelerators in asynchronous mode by unifying instruction encapsulation, reusing parameter objects, using a circular scheduling queue buffer, and batch scheduling, thereby improving the collaborative working efficiency between the processor and the computing accelerator.
[0039] In one possible implementation of this application embodiment, the first instruction structure further includes a pointer field pointing to instruction parameters, a type identifier for distinguishing the instruction type to be processed, and a status field for indicating the instruction execution status; Instruction types include at least computation type instructions and synchronization type instructions used to implement synchronization control between the processor and the accelerator.
[0040] In the embodiments of this application, the pointer field is used to determine the starting address of the memory region where the instruction parameters are located. By setting the pointer field, the first instruction structure can be kept concise and of a fixed size, which is beneficial for performing efficient batch read and write operations in a circular scheduling queue.
[0041] The type identifier is a field in the first instruction structure used to indicate the instruction type. Defining the type identifier field in the first instruction structure allows the consumer thread to quickly determine the instruction type by reading the type identifier after obtaining the first instruction structure, thus improving instruction processing efficiency.
[0042] The status field is used to determine the current execution status of the instruction to be processed. The execution status can include, but is not limited to, different stages or results such as pending, processing, completed, and execution failed. In addition, the status field can be used for parameter object recycling. For example, when the consumer thread completes the instruction operation, it updates the status field corresponding to the first instruction structure to "completed". Before recycling parameter objects, the parameter object pool can determine whether the instruction has been executed based on the status field.
[0043] Furthermore, in this embodiment, the instruction type includes at least computation type instructions and synchronization type instructions for implementing synchronization control between the processor and the accelerator. Computation type instructions are instructions used to drive the computational resources of the computation accelerator. Synchronization type instructions are instructions with synchronization semantics.
[0044] This application, through the definition of the first instruction structure, including pointer fields, type identifiers, status fields, and the division between computation type instructions and synchronous type instructions, can achieve instruction unification, enabling the asynchronous instruction scheduling and execution system to have high efficiency and reliability when processing tasks.
[0045] In one possible implementation of this application embodiment, if the first instruction structure extracted and executed by the consumer thread from the circular scheduling queue is a synchronization type instruction, then it is determined that all computation type instructions inserted into the circular scheduling queue before the synchronization type instructions have been executed.
[0046] In the embodiments of this application, when the consumer thread starts processing synchronization type instructions, it can be determined that all computation type instructions inserted into the circular scheduling queue before the synchronization type instructions have been extracted by the consumer thread and submitted to the computation accelerator, and have been executed in the accelerator hardware.
[0047] The execution mechanism of synchronization type instructions can be viewed as a program-level synchronization mechanism based on special instructions. Specifically, this application provides a schematic diagram of a program synchronization mechanism, such as... Figure 3 As shown, instruction types with synchronization semantics are defined: synchronization instructions and computation instructions enter the same queue; they are executed sequentially; the completion of a synchronization instruction indicates that all preceding instructions have been completed. This achieves precise synchronization between the CPU and the accelerator.
[0048] This application reduces the frequency of synchronization interactions between the processor and accelerator, lowers the thread context switching rate, and improves the system's task throughput and processor utilization efficiency through a program-level synchronization mechanism based on special instructions.
[0049] In one possible implementation of this application embodiment, when obtaining an idle parameter object from a preset plurality of parameter objects, it can be implemented in the following manner, but not limited to: extracting the idle parameter object from the head of the plurality of parameter objects through atomic operations of the production thread; wherein the plurality of parameter objects are configured in the form of a linked list structure, and each parameter object includes a status identifier for indicating the idle state.
[0050] In the embodiments of this application, the process of obtaining a free parameter object from multiple parameter objects is implemented through a lock-free acquisition mechanism based on atomic operations. This acquisition operation is initiated and executed by a production thread. A production thread refers to an execution thread on the processor used to encapsulate instructions to be processed into a first instruction structure and write them into a circular scheduling queue.
[0051] Multiple parameter objects are organized into a linked list structure. A linked list structure refers to a sequence of parameter objects linked together. Each object contains data space to hold the first instruction structure, as well as a pointer field to the next parameter object.
[0052] Specifically, regarding the description of parameter objects, this application provides a CAS-based lock-free object pool structure diagram, such as... Figure 4 As shown, the system pre-allocates a fixed number of parameter objects: each object contains a status identifier; the producer thread acquires idle objects through atomic operations; the consumer thread releases the objects after execution; and the objects are reused in a fixed pool. This avoids dynamic allocation at runtime.
[0053] When a producer thread acquires a free parameter object, it directly extracts the target object from the headers of multiple parameter objects through atomic operations. To support the implementation of a lock-free acquisition mechanism, each parameter object contains a status flag indicating its idle state within its internal structure. The status flag distinguishes whether the parameter object is currently in an unoccupied idle state or in an occupied state carrying instructions awaiting processing.
[0054] This application configures multiple parameter objects into a linked list structure, and the production thread extracts the free parameter object from the head of the linked list through atomic operations. At the same time, the free or occupied status of each parameter object is identified by its status flag. This can achieve low latency performance in high-concurrency instruction scenarios.
[0055] In one possible implementation of this application embodiment, when re-identifying the parameter object corresponding to the executed first instruction structure as an idle parameter object, it can be implemented in the following way, but is not limited to: by using the atomic operation of the consumer thread, the parameter object corresponding to the executed first instruction structure is inserted back as an idle parameter object to the head of multiple parameter objects.
[0056] In the embodiments of this application, after the consumer thread completes the execution of the operation corresponding to any first instruction structure, the parameter object occupied by the first instruction structure no longer needs to be occupied. In order for the parameter object to be used by subsequent newly arrived pending instructions, the consumer thread will insert the parameter object back into the idle object pool.
[0057] This insertion operation is an atomic operation performed by the consuming thread. It re-inserts the parameter object corresponding to the first instruction structure after execution, as a free parameter object, back to the head of multiple parameter objects. Inserting back to the head means placing the parameter object at the very front of the list of free parameter objects, making it the new head node of the list.
[0058] Specifically, the process of obtaining a free parameter object and releasing a occupied parameter object can also be implemented in, but is not limited to, the following ways: When obtaining a free parameter object, the allocation process (objpool_alloc) is performed: atomically read the head; retrieve the current node; update the head → next node using CAS; Returns the node pointer.
[0059] When releasing the occupied parameter object (objpool_free): set node->next = old_head; update head = node using CAS; Features: Lock-free, no malloc / free, parameter lifecycle is strictly bound to instructions.
[0060] This application releases parameter objects through the consumer thread, rather than having the producer thread passively reclaim them after waiting for instructions to complete, thus achieving a unidirectional flow of parameter objects between the producer and consumer. The producer thread handles the acquisition and population of parameter objects, while the consumer thread handles execution and release, reducing the overhead of cross-thread communication and improving the smoothness of asynchronous instruction scheduling.
[0061] In one possible implementation of this application embodiment, when the consumer thread is activated in response to the number of the first instruction structure in the circular scheduling queue meeting the preset instruction processing conditions, it can also be implemented in the following ways, but not limited to: in response to the number of the first instruction structure in the circular scheduling queue reaching a preset number threshold, an activation signal for the consumer thread is triggered.
[0062] In the embodiments of this application, the preset quantity threshold is a value configured according to the actual application scenario during system initialization or runtime. For example, 16, 30, 60, etc.
[0063] An activation signal is a notification sent by the producer thread or the system to the consumer thread when a preset quantity threshold condition is met. Upon receiving the signal, the consumer thread is activated from its dormant state and enters the instruction fetching and execution process. After processing all instructions, the consumer thread will re-enter the dormant state, waiting for the next activation signal.
[0064] Furthermore, regarding the circular scheduling queue, this application embodiment provides a schematic diagram of a batch circular instruction queue structure, as shown below. Figure 5 As shown, the batch circular scheduling queue uses a circular structure to store instructions; it supports batch enqueueing; it supports batch dequeueing; and it triggers batch wake-up after reaching a threshold. This reduces the frequency of thread switching.
[0065] Furthermore, to facilitate understanding of the instruction submission process of the production thread and the instruction execution process of the consumer thread in the embodiments of this application, the embodiments of this application provide a schematic diagram of the production thread instruction submission process, as shown below. Figure 6 As shown, and a schematic diagram of a batch execution process for a consumer thread, as follows. Figure 7 As shown, Ring is a circular scheduling queue, slot is a parameter object, and TsmCommand is the constructed first instruction structure. Through the linkage mechanism of batch scheduling and thread wake-up, the number of thread switching can be reduced and the execution continuity can be improved.
[0066] This application controls the frequency at which consumer threads are awakened by setting a reasonable preset quantity threshold and triggering an activation signal for consumer threads when the number of the first instruction structure in the circular scheduling queue reaches the threshold, thereby reducing system overhead caused by frequent thread switching.
[0067] Corresponding to the instruction scheduling method described above, this application also proposes an instruction scheduling apparatus. Since the apparatus embodiment of this application corresponds to the method embodiment described above, details not disclosed in the apparatus embodiment can be referred to in the method embodiment described above, and will not be repeated here.
[0068] Figure 8 This is a schematic diagram of the structure of an instruction scheduling device provided in an embodiment of this application, as shown below. Figure 8 As shown, it includes: The encapsulation module 81 is used to encapsulate the instruction to be processed in the processor into a first instruction structure according to a preset instruction structure. The first instruction structure includes at least instruction parameters for performing the operation corresponding to the instruction to be processed. The acquisition module 82 is used to obtain an idle parameter object from a set of preset parameter objects in response to the asynchronous scheduling mode of the instruction to be processed. The idle parameter object carries the first instruction structure and writes the first instruction structure into the circular scheduling queue. The activation module 83 is used to activate the consumer thread in response to the number of first instruction structures in the circular scheduling queue meeting the preset instruction processing conditions. The consumer thread is used to extract at least one first instruction structure from the circular scheduling queue and execute the operation corresponding to each of the at least one first instruction structure. The reset module 84 is used to re-identify the parameter object corresponding to the first instruction structure after execution as an idle parameter object.
[0069] Furthermore, in one possible implementation of this application embodiment, the first instruction structure further includes a pointer field pointing to instruction parameters, a type identifier for distinguishing the instruction type to be processed, and a status field for indicating the instruction execution status; Instruction types include at least computation type instructions and synchronization type instructions used to implement synchronization control between the processor and the accelerator.
[0070] Furthermore, in one possible implementation of this application embodiment, if the first instruction structure extracted and executed by the consumer thread from the circular scheduling queue is a synchronization type instruction, then it is determined that all computation type instructions inserted into the circular scheduling queue before the synchronization type instructions have been executed.
[0071] Furthermore, in one possible implementation of this application embodiment, the acquisition module 82 is specifically used for: Idle parameter objects are extracted from the head of multiple parameter objects through atomic operations of the production thread; the multiple parameter objects are configured in the form of a linked list, and each parameter object includes a status flag to indicate the idle state.
[0072] Furthermore, in one possible implementation of this application embodiment, the reset module 84 is specifically used for: Through atomic operations of the consumer thread, the parameter object corresponding to the first instruction structure after execution is inserted back into the head of multiple parameter objects as an idle parameter object.
[0073] Furthermore, in one possible implementation of this application embodiment, the activation module 83 is specifically used for: In response to the number of the first instruction structure in the circular scheduling queue reaching a preset threshold, an activation signal is triggered for the consumer thread.
[0074] Furthermore, in one possible implementation of this application embodiment, the activation module 83 is also used for: The scheduling mode for the pending instruction is synchronous, and the pending instruction is executed directly through the preset driver interface.
[0075] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of the embodiments of this application, and the principle is the same. Therefore, the embodiments of this application are not limited thereto.
[0076] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.
[0077] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.
[0078] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 902 or loaded from storage unit 908 into RAM (Random Access Memory) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. I / O (Input / Output) interface 905 is also connected to bus 904.
[0079] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0080] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as instruction scheduling methods. For example, in some embodiments, the instruction scheduling method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the aforementioned instruction scheduling method by any other suitable means (e.g., by means of firmware).
[0081] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs, ASSPs (Application Specific Standard Products), SOCs (System on Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0082] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0083] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0085] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.
[0086] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.
[0087] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.
[0088] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.
[0089] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. An instruction scheduling method, characterized in that, include: The instruction to be processed in the processor is encapsulated into a first instruction structure according to a preset instruction structure. The first instruction structure includes at least instruction parameters for performing the operation corresponding to the instruction to be processed. In response to the asynchronous mode of the scheduling mode of the instruction to be processed, an idle parameter object is obtained from a plurality of preset parameter objects, the first instruction structure is carried by the idle parameter object, and the first instruction structure is written into the circular scheduling queue. In response to the number of the first instruction structures in the circular scheduling queue meeting a preset instruction processing condition, a consumer thread is activated. The consumer thread is used to extract at least one of the first instruction structures from the circular scheduling queue and execute the operation corresponding to each of the first instruction structures. The parameter object corresponding to the first instruction structure after execution will be re-identified as an idle parameter object.
2. The instruction scheduling method according to claim 1, characterized in that, The first instruction structure further includes a pointer field pointing to the instruction parameters, a type identifier for distinguishing the instruction type to be processed, and a status field for indicating the instruction execution status; The instruction types include at least computation type instructions and synchronization type instructions for implementing synchronization control between the processor and the accelerator.
3. The instruction scheduling method according to claim 2, characterized in that, If the first instruction structure extracted and executed by the consumer thread from the circular scheduling queue is the synchronization type instruction, then it is determined that all computation type instructions inserted into the circular scheduling queue before the synchronization type instruction have been executed.
4. The instruction scheduling method according to claim 1, characterized in that, The step of obtaining a free parameter object from a set of preset parameter objects includes: The idle parameter object is extracted from the head of the plurality of parameter objects through atomic operations of the production thread; wherein the plurality of parameter objects are configured in the form of a linked list, and each parameter object includes a status identifier for indicating the idle state.
5. The instruction scheduling method according to claim 1, characterized in that, The step of re-identifying the parameter object corresponding to the executed first instruction structure as an idle parameter object includes: Through the atomic operation of the consumer thread, the parameter object corresponding to the executed first instruction structure is inserted back into the head of the plurality of parameter objects as an idle parameter object.
6. The instruction scheduling method according to claim 1, characterized in that, The activation of the consumer thread in response to the fact that the number of the first instruction structures in the circular scheduling queue meets a preset instruction processing condition includes: In response to the number of the first instruction structures in the circular scheduling queue reaching a preset threshold, an activation signal is triggered for the consumer thread.
7. The instruction scheduling method according to claim 1, characterized in that, The method further includes: The scheduling mode for the instruction to be processed is synchronous mode, and the instruction to be processed is executed directly through a preset driver interface.
8. A command scheduling device, characterized in that, include: An encapsulation module is used to encapsulate the instruction to be processed in the processor into a first instruction structure according to a preset instruction structure. The first instruction structure includes at least instruction parameters for performing the operation corresponding to the instruction to be processed. The acquisition module is used to, in response to the asynchronous mode of the scheduling mode of the instruction to be processed, acquire an idle parameter object from a preset plurality of parameter objects, carry the first instruction structure through the idle parameter object, and write the first instruction structure into a circular scheduling queue. An activation module is used to activate a consumer thread in response to the number of the first instruction structures in the circular scheduling queue meeting a preset instruction processing condition. The consumer thread is used to extract at least one of the first instruction structures from the circular scheduling queue and execute at least one operation corresponding to each of the first instruction structures. The reset module is used to re-identify the parameter object corresponding to the first instruction structure after execution as an idle parameter object.
9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.