Programming method, device and equipment based on producer-consumer model
By encapsulating the producer and consumer model into a function for parallel scheduling, the programming errors and high overhead caused by manually dividing data blocks in traditional programming frameworks are solved, and efficient parallel programming of data sub-blocks is achieved.
Patent Information
- Application Number
- CN202511386808.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-25
- Publication Date
- 2026-01-09
AI Technical Summary
In traditional parallel computing programming frameworks, programmers need to manually divide data blocks and call loop instructions, which leads to frequent programming errors and high programming overhead, affecting efficiency.
The instructions for the producer to produce each data sub-block are encapsulated into a production function, and the instructions for the consumer to consume each data sub-block are encapsulated into a consumption function. The pipeline parallelism of data sub-blocks is realized through a scheduling function, reducing human error and programming overhead.
It effectively reduces programming errors, improves programming efficiency, simplifies the programming process, and lowers the technical requirements for programmers.
Smart Images

Figure CN121300773A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer software technology, and in particular to a programming method, apparatus, computer-readable storage medium, computer program product, and terminal device based on a producer-consumer model. Background Technology
[0002] When developing the kernel (parallel function, i.e., the function executed on the AI chip) of an AI chip based on the traditional parallel computing programming framework, the producer-consumer model often appears in the operator implementation process, that is, some data is first produced by the producer and then consumed by the consumer.
[0003] When producers and consumers are implemented by different hardware components, for example, the producer of a certain data refers to loading data from off-chip storage into on-chip registers (loading into on-chip registers signifies the completion of data production), and the consumer refers to performing calculations on the data in the on-chip registers using the ALU (Arithmetic Logic Unit). Another example is that the producer of a certain data refers to using an SFU (Special Function Unit) pipeline to produce data, and the consumer refers to using an ALU pipeline to consume data, and so on. In these production and consumption scenarios formed by different hardware divisions, a common optimization approach for operators is to parallelize data production and consumption. That is, to divide a complete data block into several fine-grained small data portions (i.e., data sub-blocks) and to parallelize the production and consumption of these data sub-blocks.
[0004] However, when implementing the above optimization ideas based on traditional parallel computing programming frameworks, programmers need to manually divide the data block into several data sub-blocks and call a loop to schedule several data sub-blocks according to the above optimization ideas. Although this programming method is relatively common, programmers need to manually write loop instructions every time, which is easy to make mistakes. Furthermore, if programmers use the loop method to write optimization scheduling instructions themselves, the calculation of coordinates for data sub-block memory access and the explicit retrieval of data sub-blocks from the data block are very cumbersome, resulting in a significantly higher amount of programming code. This greatly increases the possibility of programming errors caused by human error, and also increases the programmer's programming overhead, affecting programming efficiency. Summary of the Invention
[0005] The purpose of this invention is to provide a programming method, apparatus, computer-readable storage medium, computer program product, and terminal device based on a producer-consumer model. By encapsulating functions for data production and data consumption and calling functions for parallel production and consumption, it can effectively reduce programming errors caused by human error of the programmer, and can also effectively reduce the programmer's programming overhead and improve programming efficiency.
[0006] To achieve the above objectives, embodiments of the present invention provide a programming method based on a producer-consumer model, comprising:
[0007] The production instructions for the producer to produce each data sub-block are encapsulated into a production function; the original data blocks of the interaction between the producer-consumer model are pre-divided into several data sub-blocks.
[0008] The consumption instructions for each data sub-block are encapsulated into a consumption function;
[0009] Based on the preset initial batch size and batch processing sub-block size, the production and consumption pipelines of data sub-blocks are parallelized by calling the production function and the consumption function through the scheduling function.
[0010] Furthermore, the encapsulation of the production instructions for producing each data sub-block by the producer into a production function specifically includes:
[0011] Based on the dimension information of the original data block and the dimension information of each data sub-block, the total number of all data sub-blocks is obtained through calculation statements;
[0012] Using the total quantity as the loop count, each data sub-block is traversed through the first loop statement, and a corresponding production instruction is generated for each data sub-block.
[0013] A production function is formed based on the calculation statement and the first loop statement; wherein the output of the production function is a production instruction block containing production instructions for all data sub-blocks.
[0014] Furthermore, the encapsulation of the consumer's consumption instructions for each data sub-block into a consumption function specifically includes:
[0015] Based on the dimension information of the original data block and the dimension information of each data sub-block, the total number of all data sub-blocks is obtained through calculation statements;
[0016] Using the total quantity as the loop count, each data sub-block is traversed through the second loop statement to generate a corresponding consumption instruction for each data sub-block;
[0017] A consumption function is formed based on the calculation statement and the second loop statement; wherein the output of the consumption function is a consumption instruction block containing consumption instructions for all data sub-blocks.
[0018] Further, the step of obtaining the total number of all data sub-blocks through a calculation statement based on the dimensional information of the original data block and the dimensional information of each data sub-block specifically includes:
[0019] Based on the dimensional information of the original data block and the dimensional information of each data sub-block, calculate the number of single-dimensional sub-blocks after the original data block is divided in each dimension.
[0020] Calculate the product of the number of single-dimensional sub-blocks in all dimensions to obtain the total number of all data sub-blocks.
[0021] Furthermore, the process of using a scheduling function to call the production function and the consumption function based on a preset initial batch size and batch processing sub-block number to achieve pipeline parallelism in the production and consumption of data sub-blocks specifically includes:
[0022] Based on the preset number of batch processing sub-blocks, the production function is called through the scheduling function to generate a production instruction block containing n production stage units; where n = loopNum / instrNum, loopNum represents the total number of all data sub-blocks, instrNum represents the number of batch processing sub-blocks, and the number of batch processing sub-blocks represents the number of data sub-blocks contained in each production stage unit;
[0023] Based on the preset number of batch processing sub-blocks, the consumption function is called through the scheduling function to generate a consumption instruction block containing n consumption stage units; wherein, the consumption stage unit and the production stage unit have a one-to-one correspondence.
[0024] Based on the preset initial batch size of the pipeline, the total number of production stage units, and the total number of consumption stage units, the production instruction blocks and the consumption instruction blocks are scheduled in parallel to achieve pipeline parallelism in the production and consumption of data sub-blocks; wherein, the initial batch size of the pipeline represents the number of production stage units initially pre-produced in the pipeline.
[0025] Furthermore, the parallel scheduling of the production instruction blocks and the consumption instruction blocks based on the preset initial batch size of the pipeline, the total number of production stage units, and the total number of consumption stage units, to achieve pipeline parallelism in the production and consumption of data sub-blocks, specifically includes:
[0026] Execute S0 to S10 of the production instruction block in sequence. x-1The production instructions for the included data sub-blocks complete the initial data filling of the pipeline; where x represents the initial batch size of the pipeline, and n production stage units and n consumption stage units are both represented as S0 to S1. n-1 ;
[0027] The loop variable i is controlled to loop within the range of 0 ≤ i < nx, and in each loop, the S in the consumption instruction block is executed sequentially. i ~S n-x-1 The data sub-blocks contain consumption instructions, and simultaneously, the S instructions in the production instruction blocks are executed sequentially. i+x ~S n-1 Production instructions for the included data sub-blocks;
[0028] After the loop ends, the S instructions in the consumer instruction block are executed sequentially. n-x ~S n-1 The instruction to consume the included data sub-blocks completes the consumption of all data sub-blocks.
[0029] To achieve the above objectives, embodiments of the present invention also provide a programming apparatus based on a producer-consumer model, comprising:
[0030] The production function encapsulation module is used to encapsulate the production instructions of the producer for producing each data sub-block into a production function; wherein, the original data block of the interaction between the producer-consumer model is pre-divided into several data sub-blocks;
[0031] The consumer function encapsulation module is used to encapsulate the consumer's consumption instructions for each data sub-block into a consumer function;
[0032] The parallel scheduling programming module is used to realize pipeline parallelism of data sub-block production and consumption by calling the production function and the consumption function through the scheduling function based on the preset initial batch number and batch processing sub-block number of the pipeline.
[0033] This invention also provides a computer-readable storage medium including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the programming method based on the producer-consumer model described above.
[0034] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the programming method based on the producer-consumer model described above.
[0035] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the programming method based on the producer-consumer model described above when executing the computer program.
[0036] Compared with existing technologies, embodiments of the present invention provide a programming method, apparatus, computer-readable storage medium, computer program product, and terminal device based on a producer-consumer model. First, the production instructions for the producer to produce each data sub-block are encapsulated into a production function, and the consumption instructions for the consumer to consume each data sub-block are encapsulated into a consumption function. Then, based on a preset initial batch size and the number of sub-blocks in the pipeline, the production and consumption functions are called through a scheduling function to achieve pipelined parallelism of data sub-block production and consumption. By encapsulating data production and consumption functions and calling functions for parallel production and consumption, embodiments of the present invention can effectively reduce programming errors caused by human error and significantly reduce programming overhead, thereby improving programming efficiency. Attached Figure Description
[0037] Figure 1 This is a flowchart of a programming method based on a producer-consumer model provided in an embodiment of the present invention;
[0038] Figure 2 This is a schematic diagram of the structure of a general-purpose graphics processor provided in an embodiment of the present invention;
[0039] Figure 3 This is a structural block diagram of a programming device based on a producer-consumer model provided in an embodiment of the present invention;
[0040] Figure 4 This is a structural block diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] This invention provides a programming method based on a producer-consumer model. See [link to relevant documentation]. Figure 1 The diagram shows a flowchart of a programming method based on a producer-consumer model according to an embodiment of the present invention. The method includes steps S11 to S13:
[0043] Step S11: Encapsulate the production instructions for each data sub-block produced by the producer into a production function; wherein, the original data blocks of the interaction between the producer-consumer model are pre-divided into several data sub-blocks;
[0044] Step S12: Encapsulate the consumption instructions for each data sub-block consumed by the consumer into a consumption function;
[0045] Step S13: Based on the preset initial batch size and batch processing sub-block size of the pipeline, the production function and the consumption function are called through the scheduling function to realize pipeline parallelism of data sub-block production and consumption.
[0046] It should be noted that in the producer-consumer model, the original data block interacting between the producer thread and the consumer thread is pre-divided into several data sub-blocks. For example, assuming the original data block is represented as D, the resulting sub-blocks can be represented as D0, D1, D2, D3, ... The process requires producing D0, producing D1, consuming D0, producing D2, consuming D1, ... in the form of "Produce D0, produce D1, consume D0, produce D2, consume D1, ...". i+1 and consumption D i In parallel processing, ideally, the order of data production and consumption is: produce D0, produce D1 (consume D0), produce D2 (consume D1), produce D3 (consume D2)... However, in reality, due to limitations in the number of buffers in the data transfer station and the number of instructions issued by the producer engine, the instruction scheduling order will change depending on the number of buffers and instructions issued. Assuming the number of data sub-blocks that a producer can load / issue in one cycle is called a stage, the value of stage is limited by the number of buffers in the data transfer station and the number of instructions issued by the producer engine (i.e., the number of buffers in the data transfer station and the number of instructions issued by the producer engine determine the value of stage). Changes in the value of stage will cause changes in the instruction scheduling order. For example (in the following examples, "Produce" means "production," "Instr" is an abbreviation for "Instruction," "Consume" means "consumption," and "Produce D0 Instr" means "the production instruction to generate data sub-block D0," "Consume..."), etc. "D0Instr" means "the consumption instruction for consuming data subblock D0", and the same applies to others:
[0047] Assuming stage = 2, the corresponding instruction issue sequence is:
[0048] Produce D0 Instr,
[0049] Produce D1 Instr,
[0050] Consume D0 Instr,
[0051] Produce D2 Instr,
[0052] Consume D1 Instr,
[0053] Produce D3 Instr,
[0054] Consume D2 Instr,
[0055] …
[0056] Assuming stage = 4, the corresponding instruction issue sequence is:
[0057] Produce D0 Instr,
[0058] Produce D1 Instr,
[0059] Produce D2 Instr,
[0060] Produce D3 Instr,
[0061] Consume D0 Instr,
[0062] Produce D4 Instr,
[0063] Consume D1 Instr,
[0064] Produce D5 Instr,
[0065] Consume D2 Instr,
[0066] Produce D6 Instr,
[0067] …
[0068] By comparing the instruction scheduling order when stage=2 and stage=4, it can be seen that the instruction scheduling order is obviously different when the stage value is different.
[0069] For example, if the data transfer station has only 2 buffers, then stage = 2, meaning that the producer can load / launch 2 Production D records per cycle. i For example, when transmitting Produce D2 data, it is necessary to wait until D0 data has been consumed before continuing to transmit D2 data.
[0070] For example, suppose the number of buffers in the data transfer station is unlimited, but the number of instructions issued by the producer engine is limited; for example, a maximum of four Produce D instructions can be issued. i The data for step 5 requires waiting for the previous instruction data to be returned before it can be launched, so stage ≤ 4.
[0071] Understandably, the value of stage also needs to satisfy stage>1, because when stage=1, it is impossible to achieve parallel data production and data consumption.
[0072] As can be seen from the background technology, when implementing parallel optimization of data production and data consumption based on traditional parallel computing programming frameworks, programmers need to manually divide data blocks into several data sub-blocks and call a loop to schedule several data sub-blocks according to the above optimization idea. Here, we still take stage=2 and stage=4 as examples:
[0073] When stage=2, assuming there are m data sub-blocks, the programmer needs to write the following instructions:
[0074] Produce D0 Instr,
[0075] Produce D1 Instr,
[0076] Loop 0≤i<m-2{
[0077] Consume D i Instr
[0078] Produce D i+2 Instr
[0079] Consume D i+1 Instr
[0080] Produce D i+3 Instr
[0081] }
[0082] Consume D m-2 Instr
[0083] Consume D m-1 Instr.
[0084] When stage = 4, assuming there are m data sub-blocks, the programmer needs to write the following instructions:
[0085] Produce D0 Instr,
[0086] Produce D1 Instr,
[0087] Produce D2 Instr,
[0088] Produce D3 Instr,
[0089] Loop 0≤i<m-4{
[0090] Consume D i Instr,
[0091] Produce D i+4 Instr,
[0092] Consume D i+1 Instr,
[0093] Produce D i+5 Instr,
[0094] Consume D i+2 Instr,
[0095] Produce D i+6 Instr,
[0096] Consume D i+3 Instr,
[0097] Produce D i+7 Instr,
[0098] }
[0099] Consume D m-4 Instr,
[0100] Consume D m-3 Instr,
[0101] Consume D m-2 Instr,
[0102] Consume D m-1 Instr。
[0103] Therefore, this programming method requires the programmer to manually write each instruction every time, which is prone to errors. Furthermore, if the programmer uses a loop to optimize the scheduling instructions, the calculation of coordinates for data sub-block memory access and the explicit retrieval of data sub-blocks from the original data block become very cumbersome, resulting in a significantly higher amount of code. This greatly increases the possibility of programming errors caused by human error and also increases the programmer's overhead, affecting programming efficiency. Moreover, since changes in the value of the stage will cause changes in the instruction scheduling order, once the value of the stage changes, the programmer needs to rewrite each corresponding instruction, making the aforementioned technical problems even more prominent.
[0104] To address the aforementioned technical problems, this invention provides a programming method based on a producer-consumer model. In specific implementation, the production instructions corresponding to each data sub-block produced by the producer can be encapsulated into a production function (represented as MultiTiledPdc), and the consumption instructions corresponding to each data sub-block consumed by the consumer can be encapsulated into a consumption function (represented as MultiTiledCsm). Then, based on preset necessary parameters such as the initial batch size and the number of batch processing sub-blocks, the production and consumption pipelines of data sub-blocks can be executed in parallel through a scheduling function (represented as scheduler). For example, the scheduler that runs the scheduling function scheduler can call the encapsulated production function MultiTiledPdc and consumption function MultiTiledCsm.
[0105] It should be noted that existing technologies can also achieve parallel optimization of data production and consumption based on existing direct scheduling assembly instruction generators. When using this generator, programmers simply push (add) several production and consumption instructions into a container in an ideal order, and then arrange all the instructions in the container to generate assembly instructions. However, the optimized programming method provided in this invention, for code segments in the producer-consumer model where operators frequently appear, schedules production and consumption instructions separately according to the required loop method. Instead of immediately pushing these two instruction segments into the container, they are stored. Then, through another scheduling function, specifying necessary parameters such as stage (i.e., the initial batch size of the pipeline) and instrNum (i.e., the number of batch sub-blocks), the two instruction segments are pipelined. Simultaneously, the instructions that individually call producer and consumer instructions to generate multiple data sub-blocks are encapsulated into functions. At this point, the programmer only needs to write the following program and provide the corresponding parameter values to obtain the complete code for the parallel execution of the data sub-block production and consumption pipeline:
[0106] pdcInstrBlock=MultiTiledPdc(tileShape,Shape);
[0107] csmInstrBlock=MultiTiledCsm(tileShape,Shape);
[0108] scheduler(pdcInstrBlock, csmInstrBlock, stage, InstrNum).
[0109] It should be noted that the MultiTiledPdc function, MultiTiledCsm function, and scheduler function in the above code, as well as the various parameters involved in each function, will be explained in detail in the following embodiments, and will not be repeated here.
[0110] The producer-consumer model-based programming method provided in this invention significantly reduces the redundancy of coordinate calculations for data sub-block memory access and explicit data sub-block retrieval from the original data block by encapsulating the MultiTiledPdc and MultiTiledCsm functions and calling the scheduler function. It also significantly reduces the amount of code written by the programmer, effectively reducing programming errors caused by human error and improving programming efficiency. Furthermore, it provides programmers with a ready-made optimized programming paradigm. For example, traditional manual programming requires highly skilled programmers, necessitating experience to complete complex programs, and even then, correctness cannot be guaranteed. However, using this invention, even beginners can easily program the parallel execution logic of data production and consumption, as they only need to know the values of the parameters involved in the MultiTiledPdc, MultiTiledCsm, and scheduler functions, eliminating the need to manually write each instruction.
[0111] In one optional embodiment, the encapsulation of the producer's production instructions for producing each data sub-block into a production function specifically includes:
[0112] Based on the dimension information of the original data block and the dimension information of each data sub-block, the total number of all data sub-blocks is obtained through calculation statements;
[0113] Using the total quantity as the loop count, each data sub-block is traversed through the first loop statement, and a corresponding production instruction is generated for each data sub-block.
[0114] A production function is formed based on the calculation statement and the first loop statement; wherein the output of the production function is a production instruction block containing production instructions for all data sub-blocks.
[0115] Specifically, in conjunction with the above embodiments, when encapsulating the production instructions for each data sub-block produced by the producer into a production function, this embodiment of the invention can first obtain the total number of all data sub-blocks contained in the original data block through a calculation statement (i.e., a program statement used to calculate the total number of data sub-blocks) based on the dimension information of the original data block and the dimension information of each data sub-block; then, using the total number of data sub-blocks as a limit for the number of iterations, and following a certain traversal order (e.g., traversing in the order of D0, D1, D2, D3, ...), iterates sequentially through the first loop statement (i.e., a program statement used to generate production instructions through a loop). For each data sub-block, and in each loop, generate corresponding production instructions for each data sub-block traversed; finally, integrate the calculation statements used and the first loop statement together to form the complete program statements corresponding to the production function MultiTiledPdc, that is, obtain the production function MultiTiledPdc. The output of the production function is the production instruction block pdcInstrBlock containing the production instructions of all data sub-blocks, that is: pdcInstrBlock = MultiTiledPdc(tileShape, Shape).
[0116] Understandably, the parameters tileShape and Shape involved in the production function MultiTiledPdc correspond to the dimension information of the data sub-block and the dimension information of the original data block. For example, if the original data block is a two-dimensional matrix of size A×B, then its corresponding dimension information Shape is A×B. Similarly, if the data sub-block is a two-dimensional matrix of size a×b, then its corresponding dimension information tileShape is a×b, and a<A, b<B.
[0117] Understandably, the instruction returned by the MultiTiledPdc function is:
[0118] Produce D0 Instr,
[0119] Produce D1 Instr,
[0120] Produce D2 Instr,
[0121] Produce D3 Instr,
[0122] …
[0123] In one optional embodiment, encapsulating the consumption instructions for each data sub-block into a consumption function specifically includes:
[0124] Based on the dimension information of the original data block and the dimension information of each data sub-block, the total number of all data sub-blocks is obtained through calculation statements;
[0125] Using the total quantity as the loop count, each data sub-block is traversed through the second loop statement to generate a corresponding consumption instruction for each data sub-block;
[0126] A consumption function is formed based on the calculation statement and the second loop statement; wherein the output of the consumption function is a consumption instruction block containing consumption instructions for all data sub-blocks.
[0127] Specifically, in conjunction with the above embodiments, when encapsulating the consumption instruction for each data sub-block into a consumption function, this embodiment of the invention can first obtain the total number of all data sub-blocks contained in the original data block through a calculation statement (i.e., a program statement used to calculate the total number of data sub-blocks) based on the dimension information of the original data block and the dimension information of each data sub-block; then, using the total number of data sub-blocks as a limit for the number of iterations, and following a certain traversal order (e.g., traversing in the order of D0, D1, D2, D3, ...), iterates sequentially through a second loop statement (i.e., a program statement used to generate consumption instructions through a loop). For each data sub-block, and in each loop iteration, generate corresponding consumption instructions for each data sub-block traversed; finally, integrate the calculation statements and the second loop statement to form the complete program statements corresponding to the consumption function MultiTiledCsm, that is, obtain the consumption function MultiTiledCsm. The output of the consumption function is the consumption instruction block csmInstrBlock containing the consumption instructions of all data sub-blocks, that is: csmInstrBlock = MultiTiledCsm(tileShape, Shape).
[0128] It is understandable that the parameters tileShape and Shape involved in the consumption function MultiTiledCsm have the same meaning as those parameters tileShape and Shape involved in the production function MultiTiledPdc, so they will not be repeated here.
[0129] Understandably, the instruction returned by the MultiTiledCsm function is:
[0130] Consume D0 Instr,
[0131] Consume D1 Instr,
[0132] Consume D2 Instr,
[0133] Consume D3 Instr,
[0134] …
[0135] In one optional embodiment, the step of obtaining the total number of all data sub-blocks through a calculation statement based on the dimensional information of the original data block and the dimensional information of each data sub-block specifically includes:
[0136] Based on the dimensional information of the original data block and the dimensional information of each data sub-block, calculate the number of single-dimensional sub-blocks after the original data block is divided in each dimension.
[0137] Calculate the product of the number of single-dimensional sub-blocks in all dimensions to obtain the total number of all data sub-blocks.
[0138] Specifically, in conjunction with the above embodiments, when obtaining the total number of all data sub-blocks through calculation statements based on the dimensional information of the original data block and the dimensional information of each data sub-block, this embodiment of the invention can, for each dimension of the original data block and each dimension of the data sub-block, take the fact that both are in the same dimension as a benchmark, calculate the number of single-dimensional sub-blocks obtained after the original data block is divided in each dimension, and then calculate the product of the number of single-dimensional sub-blocks in all dimensions based on the number of single-dimensional sub-blocks corresponding to each dimension, and take the calculated product as the total number of all data sub-blocks.
[0139] For example, suppose the original data block is a two-dimensional matrix with a corresponding dimension information Shape of A×B, and suppose the data sub-block is a two-dimensional matrix with a corresponding dimension information tileShape of a×b, and a<A, b<B. Then, the number of single-dimensional sub-blocks under one dimension = A / a, the number of single-dimensional sub-blocks under the other dimension = B / b, and the total number of all data sub-blocks loopNum = (A / a)×(B / b).
[0140] Understandably, since the original data block is divided into several data sub-blocks, it is generally required that the results obtained by A / a and B / b are integers.
[0141] For example, the implementation of the MultiTiledPdc function can be represented as follows (where "idx" is an abbreviation for "index", representing the "current loop iteration number"):
[0142] loopNum = A / a × B / b;
[0143] for(int idx = 0; idx <loopNum;idx++){
[0144] Producer D i Instr;
[0145] }
[0146] Similarly, the implementation process of the MultiTiledCsm function can be represented as follows:
[0147] loopNum = A / a × B / b;
[0148] for(int idx = 0; idx <loopNum;idx++){
[0149] Consume D i Instr;
[0150] }
[0151] In one optional embodiment, the pipeline parallelism of data sub-block production and consumption is achieved by calling the production function and the consumption function through a scheduling function based on a preset initial batch size and batch processing sub-block number, specifically including:
[0152] Based on the preset number of batch processing sub-blocks, the production function is called through the scheduling function to generate a production instruction block containing n production stage units; where n = loopNum / instrNum, loopNum represents the total number of all data sub-blocks, instrNum represents the number of batch processing sub-blocks, and the number of batch processing sub-blocks represents the number of data sub-blocks contained in each production stage unit;
[0153] Based on the preset number of batch processing sub-blocks, the consumption function is called through the scheduling function to generate a consumption instruction block containing n consumption stage units; wherein, the consumption stage unit and the production stage unit have a one-to-one correspondence.
[0154] Based on the preset initial batch size of the pipeline, the total number of production stage units, and the total number of consumption stage units, the production instruction blocks and the consumption instruction blocks are scheduled in parallel to achieve pipeline parallelism in the production and consumption of data sub-blocks; wherein, the initial batch size of the pipeline represents the number of production stage units initially pre-produced in the pipeline.
[0155] Specifically, in conjunction with the above embodiments, when the present invention achieves pipeline parallelism for the production and consumption of data sub-blocks by calling the production function and consumption function through the scheduling function based on a preset initial batch size and batch processing sub-block size, it can first call the encapsulated production function MultiTiledPdc through the scheduling function scheduler based on the preset batch processing sub-block size instrNum to generate a production instruction block pdcInstrBlock containing n production stage units, and call the encapsulated consumption function MultiTiledCsm to generate a consumption instruction block csmInstrBlock containing n consumption stage units. Here, the consumption stage units and production stage units have a one-to-one correspondence; that is, n represents both the total number of production stage units contained in the production instruction block pdcInstrBlock and the total number of consumption stage units contained in the consumption instruction block csmInstrBlock, and n = 1. `oopNum` and `instrNum` represent the total number of data sub-blocks contained in the original data block, and `instrNum` represents the preset batch processing sub-block number. This batch processing sub-block number `instrNum` represents the number of data sub-blocks contained in each production stage unit / consumer stage unit. Based on the preset initial pipeline batch number `stage`, the total number of production stage units `n`, and the total number of consumer stage units `n`, the production instruction block `pdcInstrBlock` and the consumer instruction block `csmInstrBlock` are scheduled in parallel to achieve parallel execution of the pipeline for the production and consumption of data sub-blocks. Here, the preset initial pipeline batch number `stage` represents the number of production stage units initially pre-produced in the pipeline. This initial pipeline batch number `stage` determines the parallel depth of the pipeline. For example, assuming `stage = 2`, it means that 2 production stage units are initially pre-produced, and then the pipeline enters a parallel loop of "1 consumer + 1 production".
[0156] It should be noted that in the pipeline scheduling logic of the producer-consumer model, the production / consumption stage unit refers to the instruction set formed by grouping the production / consumption instructions of data sub-blocks according to a fixed granularity (i.e., InstrNum), which is the basic operation unit of pipeline scheduling.
[0157] Understandably, the `pdcInstrBlock`, `csmInstrBlock`, `stage`, and `InstrNum` involved in the scheduling function `scheduler` correspond to the production instruction block, the consumption instruction block, the initial batch size of the pipeline, and the number of batch processing sub-blocks, respectively. Here, `InstrNum` represents the number of data sub-blocks contained in each production stage unit, satisfying `InstrNum≥1`. That is, each production stage unit is not limited to only one data sub-block and can include one or more data sub-blocks. The initial batch size of the pipeline, `stage`, is here limited to "the number of production stage units initially pre-produced in the pipeline." However, in the above embodiment, it was already defined as "the number of data sub-blocks that the producer can load / emit within one cycle is called a stage." Although these two definitions of "stage" are not entirely consistent, they... It should be noted that when InstrNum = 1, that is, each production stage unit includes only one data sub-block, the number of production stage units initially pre-produced in the pipeline is the number of data sub-blocks initially pre-produced, which is the number of data sub-blocks that the producer can load / launch in one cycle. In this case, the meaning of "stage" in both places is essentially the same. When InstrNum > 1, that is, each production stage unit includes more than one data sub-block, the number of data sub-blocks initially pre-produced in the pipeline is the number of production stage units initially pre-produced (stage × InstrNum). Correspondingly, the number of data sub-blocks that the producer can load / launch in one cycle is also the number of production stage units initially pre-produced (stage × InstrNum). In this case, the meaning of "stage" in both places is also essentially the same (the number of data sub-blocks represented is the same).
[0158] In one optional embodiment, the parallel scheduling of the production instruction blocks and the consumption instruction blocks based on a preset initial batch size, the total number of production stage units, and the total number of consumption stage units, to achieve pipeline parallelism in the production and consumption of data sub-blocks, specifically includes:
[0159] Execute S0 to S10 of the production instruction block in sequence. x-1 The production instructions for the included data sub-blocks complete the initial data filling of the pipeline; where x represents the initial batch size of the pipeline, and n production stage units and n consumption stage units are both represented as S0 to S1. n-1 ;
[0160] The loop variable i is controlled to loop within the range of 0 ≤ i < nx, and in each loop, the S in the consumption instruction block is executed sequentially. i ~Sn-x-1 The data sub-blocks contain consumption instructions, and simultaneously, the S instructions in the production instruction blocks are executed sequentially. i+x ~S n-1 Production instructions for the included data sub-blocks;
[0161] After the loop ends, the S instructions in the consumer instruction block are executed sequentially. n-x ~S n-1 The instruction to consume the included data sub-blocks completes the consumption of all data sub-blocks.
[0162] Specifically, in conjunction with the above embodiments, when the present invention performs parallel scheduling of production instruction blocks and consumption instruction blocks based on a preset initial batch size, the total number of production stage units, and the total number of consumption stage units to achieve pipeline parallelism in the production and consumption of data sub-blocks, x can represent the preset initial batch size, and the n production stage units contained in the production instruction block and the n consumption stage units contained in the consumption instruction block are both defined as S0~S n-1 (That is, S0, S1, S2, S3, ..., S n-1 Based on this, first execute S0 to S10 in the production instruction block sequentially. x-1 The production instructions corresponding to the data sub-blocks (i.e., the data sub-blocks contained in the first x production stage units of the production instruction block) are used to complete the initial data filling of the pipeline; then the loop variable i is controlled to loop within the range of 0≤i<nx, and in each loop, the S in the consumer instruction block is executed sequentially. i ~S n-x-1 It contains the consumption instructions for data sub-blocks, and performs synchronization operations through a preset synchronization channel, sequentially executing the S instructions in the production instruction block. i+x ~S n-1 The production instructions for the included data sub-blocks are further configured in parallel within each loop using the following pattern: "Execute the consumption instructions for InstrNum data sub-blocks in one consumption stage unit + execute the production instructions for InstrNum data sub-blocks in one production stage unit + execute the consumption instructions for InstrNum data sub-blocks in the next consumption stage unit + execute the production instructions for InstrNum data sub-blocks in the next production stage unit +...". After the entire loop ends, the S instructions in the consumption instruction block are executed sequentially. n-x ~S n-1 The consumption instructions contain the data sub-blocks (i.e., the data sub-blocks contained in the remaining x consumption stage units in the consumption instruction block), thereby completing the consumption of all data sub-blocks.
[0163] It should be noted that the Scheduler function interweaves the Producer Instr and Consumer Instr according to the specified stage and InstrNum, thereby achieving the effect of parallel execution of the production and consumption pipeline of data sub-blocks.
[0164] Furthermore, the production instruction block pdcInstrBlock has obtained the production instructions corresponding to loopNum data sub-blocks, and the consumption instruction block csmInstrBlock has obtained the consumption instructions corresponding to loopNum data sub-blocks. The total number of production / consumption stage units has been calculated based on n = loopNum / instrNum. The preset initial batch size of the pipeline is stage = x. Therefore, the Scheduler function can schedule the production and consumption instructions according to the number of stages. The specific functionality implemented by the Scheduler function is as follows:
[0165] Produce S0 Instr,
[0166] Produce S1 Instr,
[0167] …
[0168] Produce S x-1 Instr
[0169] Loop 0≤i<nx{
[0170] Consume S i Instr
[0171] Produce S i+x Instr
[0172] Consume S i+1 Instr
[0173] Produce S i+x+1 Instr
[0174] …
[0175] Consume S n-x-1 Instr
[0176] Produce S n-1 Instr
[0177] }
[0178] Consume S n-xInstr
[0179] Consume S n-x+1 Instr
[0180] …
[0181] Consume S n-1 Instr.
[0182] It should be noted that the embodiments of the present invention are preferably applicable to the kernel development of artificial intelligence chips. The artificial intelligence chip can be a GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Processing Unit), etc. The embodiments of the present invention do not make specific limitations.
[0183] For example, see Figure 2 The diagram shown is a schematic representation of a general-purpose graphics processor according to an embodiment of the present invention. A GPGPU is actually an array of Streaming Processor Clusters (SPCs), for example, including... Figure 2 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a GPGPU, one streaming processor cluster processes one computational task, or multiple streaming processor clusters can jointly process a computational task, and multiple streaming processor clusters can share data through a global cache / memory. A streaming processor cluster includes multiple computational units. Taking streaming processor cluster 1 as an example, it includes... Figure 2 The diagram shows computational units 1, 2, ..., N, where N is a positive integer. Each computational unit (e.g., a stream processor) performs arithmetic and logical operations other than matrix calculations such as matrix multiplication and convolution. These operations include accumulation, reduction, and regular addition, subtraction, multiplication, and division. A computational unit contains multiple cores (also called computational kernels). Each core includes an arithmetic logic unit (ALU), a floating-point unit, etc. The cores are used to execute specific computational tasks. The computational unit also includes registers (e.g., ...). Figure 2 The register file (as shown) and shared cache are used for hierarchical storage of source and destination data related to computational tasks. A shared cache within a computational unit is used to share data between the cores of that unit. For example... Figure 2As shown, the streaming processor cluster 1 also includes a tensor operation unit, which performs tensor computations, such as matrix multiplication and convolution operations. Furthermore, as... Figure 2 As shown, an intermediate-level cache can also be set in the streaming processor cluster 1. The intermediate-level cache is used to share data between computing units within a streaming processor cluster. Of course, in some GPGPUs, an intermediate-level cache may not be set, and computing units can share data through a global cache / memory.
[0184] It should be noted that the data sub-blocks involved in the producer-consumer model can be different types of data such as text, images, speech, and video. For example, in a text processing scenario, the data sub-blocks can be text data used in tasks such as text generation and text recognition. In an image processing scenario, the data sub-blocks can be image data used in tasks such as image preprocessing, image segmentation, and object detection. In a speech processing scenario, the data sub-blocks can be speech data used in tasks such as speech enhancement, speech recognition, and speech synthesis. This embodiment of the invention does not impose specific limitations.
[0185] For example, in combination Figure 2 As shown, in an application scenario where the AI chip is a GPGPU and the data sub-blocks are image data, in the specific implementation of this embodiment, the original image data blocks to be processed can be pre-loaded into the global cache / memory of the GPGPU. Provided that the computing power of the GPGPU's streaming processor cluster and the capacity of the shared cache allow, the original image is divided into multiple appropriately sized image data sub-blocks, denoted as D0, D1, D2, D3, ... . Based on this, the following steps can be performed:
[0186] (1) Call the MultiTiledPdc function to generate production instructions corresponding to multiple image data sub-blocks. All production instructions are grouped into several production stage units according to instrNum and form a production instruction block pdcInstrBlock. Each production instruction corresponds to the loading logic of an image data sub-block. For example, "Produce D0 Instr" can be defined as the scheduler loading the D0 sub-block from the address 0x0000 of the global cache / memory according to the production instruction and storing it in the 0x00~0xFF area of the shared cache.
[0187] (2) Call the MultiTiledCsm function to generate consumption instructions for each image data sub-block. All consumption instructions are also grouped into several consumption stage units (corresponding one-to-one with the production stage units) according to instrNum, and form a consumption instruction block csmInstrBlock. Each consumption instruction corresponds to the calculation logic of an image data sub-block. For example, "Consume D0 Instr" can be defined as using the D0 sub-block in the shared cache to execute a specific image processing operator on the core of the calculation unit. For example, use the tensor operation unit to perform convolution operation and temporarily store the convolution result in the register file.
[0188] (3) The core logic of pipeline scheduling corresponding to the scheduler function is as follows:
[0189] (3.1) Parameter configuration stage: The scheduler receives the production instruction block pdcInstrBlock, the consumer instruction block csmInstrBlock, stage and InstrNum (e.g., stage=3, InstrNum=2);
[0190] (3.2) Initial pre-production stage: The scheduler prioritizes the execution of the first (stage×InstrNum) production instructions (e.g., loading the 6 image data sub-blocks from S0 to S2), pre-stores the corresponding image data sub-blocks in the shared buffer, and fills the pipeline buffer;
[0191] (3.3) Parallel Loop Phase: The parallel logic of "consuming old sub-blocks + producing new sub-blocks" is executed in a loop. For example, when one group of streaming processors is executing the instructions of a certain consumption stage unit (e.g., processing the D0 and D1 sub-blocks contained in S0), another group of streaming processors synchronously executes the instructions of the subsequent production stage unit (e.g., synchronously loading the D6 and D7 sub-blocks contained in S3). The synchronization mechanism provided by GPGPU ensures data dependency safety and avoids consuming unproduced sub-blocks. This ensures that when consuming a certain sub-block, the sub-block has been fully loaded into the shared cache. This maximizes the utilization of GPGPU's computing resources, reduces the idle time of computing units, and makes data loading and computing processing overlap as much as possible in time.
[0192] (3.4) Final consumption stage: After all image data sub-blocks have been produced, the scheduler executes the remaining consumption instructions in sequence (e.g., the last 3 stage units) to ensure that the computation of all image data sub-blocks is completed.
[0193] Understandably, after each image data sub-block undergoes computational processing, the corresponding computational results can be temporarily stored in the relevant storage units of their respective streaming processor clusters (e.g., Figure 2(As shown in the register file or shared cache), after all image data sub-blocks have been processed, all the corresponding calculation results will eventually be aggregated into the global cache / memory through the intermediate level cache, and stitched together into a complete processed image according to the layout of the original image for use by subsequent applications.
[0194] It should be noted that if we only consider "encapsulating some instructions into functions", the implementation method is the existing technology. However, the innovation of this invention does not lie in the encapsulation of functions itself, but in the specific instruction encapsulation method and instruction scheduling logic proposed for specific scenarios involving the producer-consumer model based on function encapsulation and calling technology. That is, the production / consumption instructions of data sub-blocks are encapsulated by MultiTiledPdc / MultiTiledCsm, and the production / consumption instructions are dynamically interleaved by the scheduler function in combination with parameters such as stage. This solves the pain point of manually writing loops and sub-block coordinates in the traditional way. This combination design of "encapsulation + dynamic scheduling" is fundamentally different from simple instruction encapsulation.
[0195] This invention also provides a programming apparatus based on a producer-consumer model, used to implement the programming method based on the producer-consumer model described in any of the above embodiments. See [link to relevant documentation]. Figure 3 The diagram shown is a structural block diagram of a programming device based on a producer-consumer model according to an embodiment of the present invention. The device includes:
[0196] The production function encapsulation module 11 is used to encapsulate the production instructions of the producer for producing each data sub-block into a production function; wherein, the original data block of the interaction between the producer-consumer model is pre-divided into several data sub-blocks;
[0197] The consumption function encapsulation module 12 is used to encapsulate the consumption instructions of the consumer for consuming each data sub-block into a consumption function;
[0198] The parallel scheduling programming module 13 is used to realize pipeline parallelism of data sub-block production and consumption by calling the production function and the consumption function through the scheduling function based on the preset initial batch number and batch processing sub-block number of the pipeline.
[0199] Preferably, the production function encapsulation module 11 specifically includes:
[0200] The first sub-block total quantity calculation unit is used to obtain the total quantity of all data sub-blocks by using calculation statements based on the dimension information of the original data block and the dimension information of each data sub-block.
[0201] The production instruction generation unit is used to traverse each data sub-block using the total quantity as the number of iterations, and generate a corresponding production instruction for each data sub-block by using a first loop statement.
[0202] A production function encapsulation unit is used to form a production function based on the calculation statement and the first loop statement; wherein the output of the production function is a production instruction block containing production instructions for all data sub-blocks.
[0203] Preferably, the consumption function encapsulation module 12 specifically includes:
[0204] The second sub-block total number calculation unit is used to obtain the total number of all data sub-blocks by using calculation statements based on the dimension information of the original data block and the dimension information of each data sub-block.
[0205] The consumption instruction generation unit is used to traverse each data sub-block using the total quantity as the loop count, and generate a corresponding consumption instruction for each data sub-block by using a second loop statement.
[0206] The consumption function encapsulation unit is used to form a consumption function based on the calculation statement and the second loop statement; wherein the output of the consumption function is a consumption instruction block containing consumption instructions for all data sub-blocks.
[0207] Preferably, the first sub-block total quantity calculation unit or the second sub-block total quantity calculation unit is specifically used for:
[0208] Based on the dimensional information of the original data block and the dimensional information of each data sub-block, calculate the number of single-dimensional sub-blocks after the original data block is divided in each dimension.
[0209] Calculate the product of the number of single-dimensional sub-blocks in all dimensions to obtain the total number of all data sub-blocks.
[0210] Preferably, the parallel scheduling programming module 13 specifically includes:
[0211] The production instruction block generation unit is used to generate a production instruction block containing n production stage units by calling the production function through a scheduling function based on a preset number of batch processing sub-blocks; where n = loopNum / instrNum, loopNum represents the total number of all data sub-blocks, instrNum represents the number of batch processing sub-blocks, and the number of batch processing sub-blocks represents the number of data sub-blocks contained in each production stage unit;
[0212] The consumption instruction block generation unit is used to generate a consumption instruction block containing n consumption stage units by calling the consumption function through a scheduling function based on a preset number of batch processing sub-blocks; wherein, the consumption stage unit and the production stage unit have a one-to-one correspondence.
[0213] The parallel scheduling programming unit is used to perform parallel scheduling of the production instruction block and the consumption instruction block based on the preset initial batch number of the pipeline, the total number of production stage units, and the total number of consumption stage units, so as to realize pipeline parallelism of data sub-block production and consumption; wherein, the initial batch number of the pipeline represents the number of production stage units initially pre-produced in the pipeline.
[0214] Preferably, the parallel scheduling programming unit is specifically used for:
[0215] Execute S0 to S10 of the production instruction block in sequence. x-1 The production instructions for the included data sub-blocks complete the initial data filling of the pipeline; where x represents the initial batch size of the pipeline, and n production stage units and n consumption stage units are both represented as S0 to S1. n-1 ;
[0216] The loop variable i is controlled to loop within the range of 0 ≤ i < nx, and in each loop, the S in the consumption instruction block is executed sequentially. i ~S n-x-1 The data sub-blocks contain consumption instructions, and simultaneously, the S instructions in the production instruction blocks are executed sequentially. i+x ~S n-1 Production instructions for the included data sub-blocks;
[0217] After the loop ends, the S instructions in the consumer instruction block are executed sequentially. n-x ~S n-1 The instruction to consume the included data sub-blocks completes the consumption of all data sub-blocks.
[0218] It should be noted that the programming device based on the producer-consumer model provided in this embodiment of the invention can implement all the processes of the programming method based on the producer-consumer model described in any of the above embodiments. The functions and technical effects of each module and unit in the device are the same as the functions and technical effects of the programming method based on the producer-consumer model described in the above embodiments, and will not be repeated here.
[0219] This invention also provides a computer-readable storage medium including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the programming method based on the producer-consumer model described in any of the above embodiments.
[0220] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the programming method based on the producer-consumer model described in any of the above embodiments.
[0221] This invention also provides a terminal device, see [link to relevant documentation]. Figure 4 The diagram shown is a structural block diagram of a terminal device according to an embodiment of the present invention. The terminal device includes a processor 10, a memory 20, and a computer program stored in the memory 20 and configured to be executed by the processor 10. When the processor 10 executes the computer program, it implements the programming method based on the producer-consumer model described in any of the above embodiments.
[0222] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 20 and executed by the processor 10 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.
[0223] The processor 10 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 10 may be any conventional processor. The processor 10 is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.
[0224] The memory 20 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., while the data storage area can store related data, etc. Furthermore, the memory 20 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), and a flash card, or other volatile solid-state storage devices.
[0225] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 4 The structural block diagram shown is merely a structural example of the terminal device described above and does not constitute a limitation on the structure of the terminal device. The terminal device may include more or fewer components than shown, or combine certain components, or use different components.
[0226] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A programming method based on a producer-consumer model, characterized in that, include: The production instructions for the producer to produce each data sub-block are encapsulated into a production function; the original data blocks of the interaction between the producer-consumer model are pre-divided into several data sub-blocks. The consumption instructions for each data sub-block are encapsulated into a consumption function; Based on the preset initial batch size and batch processing sub-block size, the production and consumption pipelines of data sub-blocks are parallelized by calling the production function and the consumption function through the scheduling function.
2. The programming method based on the producer-consumer model as described in claim 1, characterized in that, The process of encapsulating the production instructions for each data sub-block produced by the producer into a production function specifically includes: Based on the dimension information of the original data block and the dimension information of each data sub-block, the total number of all data sub-blocks is obtained through calculation statements; Using the total quantity as the loop count, each data sub-block is traversed through the first loop statement, and a corresponding production instruction is generated for each data sub-block. A production function is formed based on the calculation statement and the first loop statement; wherein the output of the production function is a production instruction block containing production instructions for all data sub-blocks.
3. The programming method based on the producer-consumer model as described in claim 1, characterized in that, The process of encapsulating the consumption instructions for each data sub-block into a consumption function specifically includes: Based on the dimension information of the original data block and the dimension information of each data sub-block, the total number of all data sub-blocks is obtained through calculation statements; Using the total quantity as the loop count, each data sub-block is traversed through the second loop statement to generate a corresponding consumption instruction for each data sub-block; A consumption function is formed based on the calculation statement and the second loop statement; wherein the output of the consumption function is a consumption instruction block containing consumption instructions for all data sub-blocks.
4. The programming method based on the producer-consumer model as described in claim 2 or 3, characterized in that, The step of obtaining the total number of all data sub-blocks through calculation statements based on the dimensional information of the original data block and the dimensional information of each data sub-block specifically includes: Based on the dimensional information of the original data block and the dimensional information of each data sub-block, calculate the number of single-dimensional sub-blocks after the original data block is divided in each dimension. Calculate the product of the number of single-dimensional sub-blocks in all dimensions to obtain the total number of all data sub-blocks.
5. The programming method based on the producer-consumer model as described in claim 1, characterized in that, The process, based on a preset initial batch size and batch processing sub-block number, uses a scheduling function to call the production and consumption functions to achieve pipelined parallelism in the production and consumption of data sub-blocks. Specifically, this includes: Based on the preset number of batch processing sub-blocks, the production function is called through the scheduling function to generate a production instruction block containing n production stage units; where n = loopNum / instrNum, loopNum represents the total number of all data sub-blocks, instrNum represents the number of batch processing sub-blocks, and the number of batch processing sub-blocks represents the number of data sub-blocks contained in each production stage unit; Based on the preset number of batch processing sub-blocks, the consumption function is called through the scheduling function to generate a consumption instruction block containing n consumption stage units; wherein, the consumption stage unit and the production stage unit have a one-to-one correspondence. Based on the preset initial batch size of the pipeline, the total number of production stage units, and the total number of consumption stage units, the production instruction blocks and the consumption instruction blocks are scheduled in parallel to achieve pipeline parallelism in the production and consumption of data sub-blocks; wherein, the initial batch size of the pipeline represents the number of production stage units initially pre-produced in the pipeline.
6. The programming method based on the producer-consumer model as described in claim 5, characterized in that, The method involves parallel scheduling of the production instruction blocks and the consumption instruction blocks based on a preset initial batch size, the total number of production stage units, and the total number of consumption stage units, thereby achieving pipeline parallelism in the production and consumption of data sub-blocks. Specifically, this includes: Execute S0 to S10 of the production instruction block in sequence. x-1 The production instructions for the included data sub-blocks complete the initial data filling of the pipeline; where x represents the initial batch size of the pipeline, and n production stage units and n consumption stage units are both represented as S0 to S1. n-1 ; The loop variable i is controlled to loop within the range of 0 ≤ i < nx, and in each loop, the S in the consumption instruction block is executed sequentially. i ~S n-x-1 The data sub-blocks contain consumption instructions, and simultaneously, the S instructions in the production instruction blocks are executed sequentially. i+x ~S n-1 Production instructions for the included data sub-blocks; After the loop ends, the S instructions in the consumer instruction block are executed sequentially. n-x ~S n-1 The instruction to consume the included data sub-blocks completes the consumption of all data sub-blocks.
7. A programming device based on a producer-consumer model, characterized in that, include: The production function encapsulation module is used to encapsulate the production instructions of the producer for producing each data sub-block into a production function; wherein, the original data block of the interaction between the producer-consumer model is pre-divided into several data sub-blocks; The consumer function encapsulation module is used to encapsulate the consumer's consumption instructions for each data sub-block into a consumer function; The parallel scheduling programming module is used to realize pipeline parallelism of data sub-block production and consumption by calling the production function and the consumption function through the scheduling function based on the preset initial batch number and batch processing sub-block number of the pipeline.
8. A computer-readable storage medium, characterized in that, The device includes a stored computer program that, when executed, controls the device containing the computer-readable storage medium to perform a programming method based on the producer-consumer model as described in any one of claims 1 to 6.
9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the programming method based on the producer-consumer model as described in any one of claims 1 to 6.
10. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the programming method based on the producer-consumer model as described in any one of claims 1 to 6.
