Tensor register management method and related product thereof

Abstract

The invention relates to a tensor register management method and related products thereof. The related products comprise a processing device, a computer readable storage medium and a computer program product. The apparatus may be included in a computing device of a combined processing device, which may include one or more data processing devices. The combined processing device can also comprise an interface device and other processing devices. And the computing device interacts with other processing devices to jointly complete the computing operation specified by the user. A compiler is loaded in the other processing device and is used for optimizing and compiling the source program executing the deep learning into a target program (machine language) which can be recognized and executed by the computing device. The combined processing device can further comprise a storage device, and the storage device is connected with the equipment and the other processing devices and used for storing data of the equipment and the other processing devices. Through the scheme of the invention, the state management overhead of the tensor register can be reduced.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computers. More specifically, the present disclosure relates to a tensor register management method, a processing device for executing the aforementioned method, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of AI technology, the requirements for software and hardware processing performance are getting higher and higher. In order to improve computing power and better process deep learning networks, AI-specific processors generally support tensor-based programming models, that is, users will create and use tensors for programming. The tensors created by users need to be bound to the tensor registers of the hardware. Due to the limitation of chip physical resources, the number of tensor registers is limited, so register allocation strategies are needed to manage the life cycle of tensor registers.

[0003] Unlike traditional scalar registers, tensor registers must be released before they are created again, so that they are in an initialized state. Traditional register allocation algorithms do not restore registers to their initialized state before using them. In addition, restoring the state of tensor registers (i.e., releasing tensor registers) is expensive. If the state is restored before each use, it will cause a large software overhead, and the state needs to be restored at the appropriate time.

[0004] In view of this, there is an urgent need for a tensor register management solution that only releases the tensor register when necessary to reduce the state management overhead of the tensor register. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a management solution for tensor registers in multiple aspects. By using the solution of the present disclosure, the state management overhead of the tensor registers can be reduced.

[0006] In a first aspect, the present disclosure provides a tensor register management method, the method comprising: counting tensor registers created by each basic block in a control flow graph; obtaining any basic block in the control flow graph as a first current basic block; based on the statistical result, comparing whether all the predecessor basic blocks of the first current basic block have created a specific tensor register; and based on the comparison result, determining the release position of the tensor register in the control flow graph. .

[0007] In a second aspect, the present disclosure provides a computer-readable storage medium having a computer program code of a tensor register management method stored thereon. When the computer program code is executed by a processing device, multiple embodiments discussed above and below are executed.

[0008] In a third aspect, the present disclosure provides a computer program product, comprising a computer program of a tensor register management method, and when the computer program instructions are executed by a processor, the above method and multiple embodiments thereof to be discussed below are performed.

[0009] In a fourth aspect, the present disclosure provides a processing device, the processing device comprising a compiler: the compiler is used to execute the above method and multiple embodiments thereof to be discussed below.

[0010] Through the solutions provided in the above aspects of the present disclosure, when analyzing the tensor registers in a computer program, the compiler abstracts the computer program into a control flow graph, each node of the flow graph is called a basic block, and each basic block corresponds to a section of linear code. Before deciding the release timing of the tensor registers, the tensor registers created by each basic block in the control flow graph are counted, and based on the statistical results, all the predecessor basic blocks of the basic block are compared to see whether they have created specific tensor registers, and then the release position of the tensor register in the control flow graph is determined based on the comparison results. Compared with the processing method of uniformly releasing tensor registers in each basic block, the specific tensor registers that need to be created by the basic block and its predecessor basic block in the embodiment of the present disclosure are compared, and the necessity of releasing tensor registers in each basic block is confirmed, which can save unnecessary release overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0012] Figure 1 is a schematic diagram showing the structure of a board according to an embodiment of the present disclosure;

[0013] Figure 2 is a structural diagram showing a combined processing device in a chip according to an embodiment of the present disclosure;

[0014] Figure 3 is a simplified flow chart schematically illustrating a tensor register management method in an embodiment of the present disclosure;

[0015] Figure 4 is a simple control flow chart schematically shown;

[0016] Figure 5 is a flowchart schematically illustrating a tensor register management method in an embodiment of the present disclosure;

[0017] Figure 6 is a state statistics diagram schematically showing a control flow graph before processing;

[0018] Figure 7 It is a state statistics diagram schematically showing a control flow graph after completing global statistics;

[0019] Figure 8 is a state statistics diagram schematically showing a control flow diagram configuration release instruction processing completion;

[0020] Fig. 9 is a flowchart schematically illustrating a tensor register management method in an embodiment of the present disclosure;

[0021] Fig.10 is a state statistics diagram schematically showing a control flow graph before processing;

[0022] Fig.11 It is a schematic diagram showing the state statistics of a control flow graph after completing the first round of traversal;

[0023] Fig.12 It is a state statistics diagram schematically showing a control flow graph after completing the second round of traversal;

[0024] Fig.13 It is a state statistics diagram schematically showing a control flow chart configuration release instruction processing completion. DETAILED DESCRIPTION

[0025] It should be understood that the terms "first", "second", "third", "fourth", etc. in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0026] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0027] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0028] As mentioned above, due to the limitation of chip physical resources, the number of tensor registers is limited, so the state of tensor registers needs to be managed. Tensor registers have two states: initialization state and creation state. Before the program is executed, the tensor register is in the initialization state; once a tensor register is created, the state of the tensor register changes to the creation state. If the tensor register is released, the state of the tensor register returns to the initialization state.

[0029] When analyzing the tensor registers in a computer program, the compiler abstracts the computer program into a control flow graph (CFG). Each node of the flow graph is called a basic block (BB). Each basic block corresponds to a linear code (i.e., there is no branch jump).

[0030] For two basic blocks in a control flow graph that have a front-to-back dependency relationship, the front basic block is called the predecessor basic block of the back basic block. If a basic block and its predecessor basic block both create the same tensor register, the tensor register must be released before the tensor register of the basic block is created again. However, releasing the tensor register has an overhead. If each basic block of the control flow graph performs a release operation before running, it will cause a large software overhead. In view of this, the present disclosure proposes a set of tensor register management solutions based on the analysis of the data flow in the control flow graph to reduce the state management overhead of the tensor register.

[0031] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0032] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1As shown, the board 10 includes a chip 101, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 10 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and a large amount of computing power.

[0033] The chip 101 is connected to an external device 103 via an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 103 to the chip 101 via the external interface device 102. The calculation result of the chip 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface.

[0034] The board 10 also includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected to the control device 106 and the chip 101 through a bus and transmits data. The control device 106 in the board 10 is configured to control the state of the chip 101. To this end, in an application scenario, the control device 106 may include a microcontroller (MCU).

[0035] Figure 2 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a DRAM 204 .

[0036] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0037] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into the storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into the control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data in the storage device of the computing device 201 and transmit it to the processing device 203.

[0038] The processing device 203 is a general processing device that performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, which include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0039] DRAM 204 is used to store data to be processed and is a DDR memory, which is usually 16G or larger in size and is used to store data of the computing device 201 and / or the processing device 203 .

[0040] In order for the computing device 201 to implement neural network operations, the developer must input the ideas, methods and means of the problem to be solved into the computing device 201 in a form that the computing device 201 can understand, so that the computing device 201 can execute in sequence according to the instructions. This channel of communication between people and computers is called programming. Programming languages ​​include machine languages ​​and high-level languages.

[0041] Machine language is an operation code that can be directly recognized by the machine. Each operation code has a corresponding circuit inside the computing device 201 to complete it. Generally, a series of 0 and 1 instructions are used to directly control the potential of each component of the computing device 201 to complete the expected task. Programs written in machine language occupy less memory and have high execution efficiency because each instruction corresponds to a specific basic action of the computing device 201. However, the disadvantages are that the programming workload is large, it is prone to errors, it is difficult to interpret, and it depends on the specific computer structure, so the versatility and portability of the program are poor.

[0042] High-level languages ​​are independent of the hardware structure and instruction system of the computing device 201. They are based on human logic and grammar, so they have stronger expressive power, are convenient for representing data operations and program control structures, can intuitively describe various algorithms, and are easy to learn and master. Currently popular programming languages ​​such as Bang C of Cambrian and CUDA of NVIDIA are all high-level languages. Since high-level languages ​​are programmed from a human perspective, they are relatively indirect to the computing device 201, so the opcodes generated after compilation are often longer than the program codes in machine language, and the execution speed is also slow. In addition, high-level languages ​​"cannot see" the hardware structure of the computing device 201, so they cannot directly control the system software that accesses hardware resources.

[0043] Codes written in high-level languages ​​need to be converted into machine codes through a compiler to drive the computing device 201. The processing device 203 is equipped with a compiler or a compiler, which optimizes the source program for deep learning and compiles it into a target program (machine language) that can be recognized and executed by the computing device 201. In order to make the compiler logically simpler and clearer, especially to make the optimization of the target code easier to implement, the processing device 203 will use an intermediate language with a complexity between the source program and the machine language to translate the source code into the intermediate code, so that its structure is logically simpler and clearer. The processing device 203 performs optimization based on the intermediate code, and after the optimization is completed, it is transformed into the target code that can be recognized and executed by the computing device 201.

[0044] Figure 3 3 is a simplified flowchart schematically showing a tensor register management method 300 in an embodiment of the present disclosure. In an embodiment of the present disclosure, the tensor register management method is executed by the processing device 203 for the structure of the computing device 201 .

[0045] In step S301, statistics are collected on tensor registers created by each basic block in the control flow graph.

[0046] The disclosed embodiment will analyze the data flow of the basic blocks in the control flow graph, determine the data dependency between the basic blocks (i.e., who is the predecessor basic block of whom, reflecting the connection relationship of the data flow direction in the control flow graph) and the tensor registers created by each basic block, and determine the release strategy of the tensor registers by whether the basic block and its predecessor basic block have created the same tensor registers. Specifically, the disclosed embodiment can count the tensor registers created in each basic block by simulating the code corresponding to each basic block in the control flow graph.

[0047] In deep learning, a tensor refers to a data structure that can be represented as a multidimensional array. In common deep learning frameworks, such as PyTorch and TensorFlow, tensors are basic data types. In a program, input data, model parameters, intermediate calculation results, etc. are usually represented as tensors. A tensor register is a data structure that can store multiple tensors, similar to a tensor array. In a program, tensor registers are usually used to store model parameters, calculation results, etc. Tensor registers can be easily used for batch operations, such as batch initialization, batch reading, batch assignment, etc.

[0048] Therefore, in the code corresponding to the basic block of the embodiment of the present application, if there is a demand for tensor use, a corresponding tensor register will be created (that is, there is an instruction "createtr trx" in the code, where "create" means creation, "tr" means tensor register, and "x" means the number corresponding to the tensor register).

[0049] In step S302, any basic block in the control flow graph is obtained as a first current basic block.

[0050] It should be noted that the embodiments of the present disclosure will analyze the basic blocks in the control flow graph one by one, and the basic block currently being analyzed is called the first current basic block. In the analysis process of determining the release position of the tensor register, if the current basic block has completed the analysis, the next basic block is updated to the first current basic block. In the subsequent embodiments of the present disclosure, the statistical process of the tensor register will be described. Similarly, the basic blocks of the control flow graph will be traversed one by one during the statistical process, and the basic block currently being statistically analyzed is called the second current basic block.

[0051] In step S303 , based on the statistical result, it is compared whether all the predecessor basic blocks of the first current basic block have created specific tensor registers.

[0052] When analyzing the first current basic block, first determine all its predecessor basic blocks, and then determine the tensor registers created by the first current basic block and all its predecessor basic blocks. If the first current basic block and any of its predecessor basic blocks create a same tensor register, then the same tensor register is a specific tensor register.

[0053] In step S304, the release position of the tensor register in the control flow graph is determined based on the comparison result.

[0054] In a release scenario, if all predecessor basic blocks of the first current basic block have created specific tensor registers, it means that each predecessor basic block of the current basic block needs to release the specific tensor register. Therefore, the instructions for releasing the specific tensor registers can be uniformly configured at the entrance of the first current basic block or inside the first current basic block. Then, multiple predecessor basic blocks only need to configure one tensor register release instruction, which can save the configuration overhead of releasing the tensor registers.

[0055] See also Figure 4 , Figure 4 4 is a schematic diagram of a simplified control flow chart 400. Assuming that basic block C is the first current basic block, according to Figure 4 The data flow relationship in , basic block A and basic block B are the predecessor basic blocks of basic block C. Among them, the line segment area where the entrance of basic block C is located is from endpoint c1 to endpoint c2, the line segment area where the exit of basic block A (the connection path between basic block A and basic block C) is located is from endpoint a to endpoint c1, and the line segment area where the exit of basic block B (the connection path between basic block B and basic block C) is located is from endpoint b to endpoint c1.

[0056] Specifically, Figure 4 As shown in the figure, the entry of a basic block in the control flow chart refers to: each predecessor basic block points to a section of the area that will flow through the first current basic block. The entry of a basic block is reflected in the execution of the program as follows: a basic block is a section of the program to be executed, and the connection relationship between basic blocks reflects the timing relationship of the execution of these subroutines. Therefore, the entry of a basic block can also be understood as a time window before the execution of the basic block. That is, the operation of releasing the tensor register is performed before the subroutine corresponding to the basic block is executed.

[0057] Specifically, releasing a specific tensor register inside a basic block refers to adding an instruction for releasing the specific tensor register in a program corresponding to the original basic block.

[0058] In another release scenario, if any predecessor basic block of the first current basic block does not create a specific tensor register, it means that not every predecessor basic block needs to release the specific tensor register. If the tensor register is released uniformly at the entrance of the first current basic block or inside the first current basic block, the tensor register needs to be released once each time the program corresponding to the first current basic block is executed. For some predecessor basic blocks that do not create specific tensor registers, this release operation is redundant, and simply increasing the release overhead does not produce a substantial effect. Therefore, an instruction to release a specific tensor register can be configured at the exit of each predecessor basic block that creates a specific tensor register, so that when running the program corresponding to the basic block, only the predecessor basic block with the release requirement needs to perform the release operation, which can save unnecessary release overhead.

[0059] Specifically, Figure 4 As shown in the figure, the exit of a basic block in the control flow chart refers to: the area before the entrance of the first current basic block outputted by the predecessor basic block. The exit of a basic block in program execution is reflected as follows: a basic block is a section of a program to be executed, and the connection relationship between basic blocks reflects the timing relationship of the execution of these subroutines. Therefore, the exit of a basic block can also be understood as a time window after the execution of the basic block. That is, after each execution of the subroutine corresponding to the predecessor basic block, the operation of releasing the tensor register is executed immediately.

[0060] It should be noted that, in actual applications, a basic block can serve as a predecessor basic block of multiple basic blocks, that is, a basic block can have multiple exits. The basic block "exit" in the disclosed embodiment refers only to the first current basic block (that is, it refers to the exit of the predecessor basic block located between the connection path between the first current basic block and the predecessor basic block).

[0061] The disclosed embodiment mainly decides the release timing of tensor registers. By counting the tensor registers created by each basic block in the control flow chart, based on the statistical results, it is compared whether all the predecessor basic blocks of the first current basic block have created specific tensor registers. If all the predecessor basic blocks of the first current basic block have created specific tensor registers, it means that each predecessor basic block needs to release the specific tensor register. Therefore, the instructions for releasing the specific tensor registers are uniformly configured at the entrance of the first current basic block or inside the first current basic block, which can save the configuration overhead of releasing the tensor registers (that is, multiple predecessor basic blocks only need to configure one tensor register release instruction); if any predecessor basic block of the first current basic block does not create a specific tensor register, then at the exit of each predecessor basic block that creates the specific tensor register, the specific tensor register release instruction is configured; and for the predecessor basic block that does not create a specific tensor register, it is not necessary to configure the tensor register release instruction, which can save unnecessary release overhead (that is, when running the program corresponding to the basic block, only the predecessor basic block with the release requirement needs to perform the release operation).

[0062] Figure 5 Schematically shows a flow chart of a tensor register management method 500 in an embodiment of the present disclosure. In the actual analysis scenario of the control flow graph, it is necessary to consider the status of all input tensors and output tensors of each basic block. If there is a loop operation between the basic blocks in the control flow graph (that is, the output of the subsequent basic block will be used as the input of the previous basic block), only one traversal of the basic blocks in the control flow graph cannot accurately know all the input tensors and output tensors of the basic blocks; if there is no loop operation between the basic blocks in the control flow graph (such as Figure 6 The control flow diagram shown in FIG. 10 shows a control flow diagram of FIG. 10 , and the statistics of the state flow information can be completed by one traversal. It should be understood that Figure 5 The tensor register management method shown may also include a state statistics process of the tensor register. Figure 5 The embodiment shown can be considered as Figure 3 One implementation of the embodiment, therefore regarding Figure 3 The description also applies to Figure 5 .

[0063] In step S501, a basic block of the control flow graph that has not yet executed state flow information statistics is obtained as the second current basic block. In the embodiment of the present disclosure, in the process of counting the tensor registers created by each basic block in the control flow graph, the basic blocks of the control flow graph are traversed one by one, and the basic block currently being counted is called the second current basic block. If the current basic block has completed the statistics, the next basic block is updated as the second current basic block.

[0064] Since it is necessary to release the tensor registers, the situation of the tensor registers in the predecessor basic block must be considered first. Therefore, in order to improve the analysis efficiency of the basic block, the reverse order can be used as the acquisition order of the basic block. Reverse order is a traversal order, which means that before accessing the current node, all the predecessor nodes of the current node will be visited first. Figure 4 For example, if you want to access basic block C, you need to first access basic block A and basic block B. Assuming that basic block A has a predecessor basic block, before accessing basic block A, you need to visit all the predecessor basic blocks of basic block A, and so on.

[0065] Specifically, the state flow information (tr_state_flow_info) in the embodiment of the present disclosure includes: an input tensor register set, an output tensor register set, and a tensor register creation instruction set.

[0066] The input tensor register set (in_tr_set) is used to record the status of all tensor registers at the entry of the basic block, and is updated as the status of the output tensor registers of the predecessor basic block of the basic block changes. In the embodiment of the present disclosure, the value of the input tensor register is determined according to the intersection tensor register set of the basic block. Specifically, the intersection tensor register set is the intersection of the output tensor register sets of all predecessor basic blocks in the basic block.

[0067] The output tensor register set (out_tr_set) is used to record the status of all tensor registers at the exit of the basic block, and is updated as the tensor register status inside the basic block is analyzed and calculated. In the embodiment of the present disclosure, the value of the output tensor register set is determined based on all tensor register creation instructions inside the basic block. For example, if the basic block has two tensor register creation instructions "createtr tr1" and "createtr tr2", the output tensor register set includes "tr1" and "tr2".

[0068] The tensor register creation instruction set is used to record the set of all tensor register creation instructions within the basic block (such as Figure 6 "createtr tr1" and createtr tr2" inside basic block A in .

[0069] Figure 6 FIG. 6 is a schematic diagram showing a state statistics diagram 600 before a control flow graph is processed. Figure 7It is a schematic diagram showing a state statistics diagram 700 after a control flow graph completes global statistics (global statistics means that each basic block in the control flow graph has completed a traversal of state flow information statistics). When the global statistics of state flow information are performed for the first time, each basic block in the control flow graph is initialized before obtaining the basic block for state flow information statistics. Specifically, the initialization operation includes: clearing the state flow information such as the input tensor register set, the output tensor register set, and the register set to be released of all basic blocks.

[0070] by Figure 6 and Figure 7 Taking an example for explanation, the predecessor basic block of basic block D is basic block C, the predecessor basic blocks of basic block C are basic block A and basic block B, and basic block A and basic block B have no predecessor basic blocks. According to the reverse order rule, basic block A or basic block B is obtained first, basic block C is obtained secondly, and basic block D is obtained last. State statistics diagram 600 is the initial state before processing, and all state flow information is an empty set. Assuming that the first basic block to perform state flow information statistics is basic block A, as shown in state statistics diagram 700, after basic block A completes the statistics of state flow information, its input tensor register set (in_tr_set) is an empty set (because it has no predecessor basic block), the output tensor register set (out_tr_set) is {tr1, tr2}, and the tensor register creation instruction set is {createtr tr1, createtr tr2}.

[0071] In step S502, it is determined whether all basic blocks in the current global statistics have completed the statistics of state flow information. In the embodiment of the present disclosure, global statistics means that the traversal of state flow information statistics is completed once for each basic block in the control flow graph. It should be noted that, depending on the actual situation (e.g., whether there are looping basic blocks in the control flow graph), the traversal of state flow information statistics can be performed once (e.g., Figure 5 exemplified embodiment) or twice or more (such as Fig. 9 Embodiment shown).

[0072] If there are basic blocks that have not been counted, the process returns to step S501 and continues to perform the step of counting the state flow information of the tensor registers for the basic blocks in the control flow graph that have not been counted.

[0073] Taking the state statistics diagram 600 as an example, according to the reverse order rule, the next basic block to be obtained is the basic block B. As shown in the state statistics diagram 700, after the basic block B completes the statistics, its input tensor register set (in_tr_set) is an empty set, the output tensor register set (out_tr_set) is {tr2}, and the tensor register creation instruction set is {createtr tr2}. By analogy, the basic block obtained by the next execution of step S501 is the basic block C. As shown in the state statistics diagram 700, after the basic block C completes the statistics, its intersection tensor register set (i.e., the intersection of the output tensor register sets in the predecessor basic block of the basic block C, all_in_tr_set) is {tr2}, therefore, its input tensor register set (in_tr_set) is {tr2}, the output tensor register set (out_tr_set) is {tr1, tr2}, and the tensor register creation instruction set is {createtr tr1, createtr tr2}. By analogy, the basic block obtained by executing step S501 next time is basic block D (basic block D is the basic block corresponding to the exit instruction, indicating the end of the process). As shown in the state statistics diagram 700, after the basic block D completes the statistics, its intersection tensor register set (all_in_tr_set) is {tr1, tr2}, therefore, its input tensor register set (in_tr_set) is {tr1, tr2}, and the output tensor register set (out_tr_set) is an empty set (because it corresponds to only an exit instruction and does not perform any tensor register creation operation), and the tensor register creation instruction set is an empty set.

[0074] If there is no uncounted basic block, the global counting ends and step S503 is executed to determine whether all the predecessor basic blocks of each basic block in the control flow graph have been determined.

[0075] In step S503, it is determined whether each basic block in the control flow graph is in a stable state. If not, the process returns to steps S501 to S502, and global statistics are performed again on each basic block in the control flow graph. If yes, the process of state flow information statistics ends, and step S504 is performed to analyze the first current basic block in the control flow graph based on the statistical results of the state flow information. Among them, each basic block is in a stable state means that the predecessor basic block of each basic block does not change in two adjacent global statistics.

[0076] In the embodiment of the present disclosure, because there is no loop between basic blocks, only one traversal needs to be performed. Therefore, the embodiment of the present disclosure has only one global statistics, that is, the state flow information statistics of the control flow graph are completed after one global statistics is completed, and all the predecessor basic blocks of each basic block are determined. Specifically, it can be determined whether each basic block is in a stable state by judging whether the basic block is stable (e.g., the output tensor register set of the basic block remains unchanged in two consecutive global statistics). It can be understood that in actual applications, there are other ways to judge whether each basic block in the control flow graph is in a stable state, which is only an example here.

[0077] In step S504, the first current basic block in the control flow graph is analyzed based on the statistical results of the state flow information.

[0078] In the analysis process, the order of obtaining the current basic block can also adopt the reverse order rule.

[0079] Specifically, if the first current basic block and any of its predecessor basic blocks create an identical tensor register, the identical tensor register is a specific tensor register. If the predecessor basic blocks of the first current basic block all create specific tensor registers, then step S505 is executed to configure an instruction to release the specific tensor register at the entrance of the first current basic block or inside the first current basic block; if any of the predecessor basic blocks of the first current basic block does not create a specific tensor register, then step S506 is executed to configure an instruction to release the specific tensor register at the exit of each predecessor basic block that creates a specific tensor register.

[0080] Taking the state statistics diagram 600 as an example, the state statistics diagram 700 is shown after the control flow chart completes the statistics of each basic block. According to the reverse order rule, basic block A and basic block B have no predecessor basic blocks, so the first current basic block analyzed is basic block C. Specifically, the intersection tensor register set is the intersection of the output tensor register sets of all the predecessor basic blocks of the basic block. Therefore, the "intersection tensor register set (all_in_tr_set) of basic block C is {tr2}, the input tensor register set (in_tr_set) is {tr2}, and the output tensor register set (out_tr_set) is {tr1, tr2}". The output tensor register sets (out_tr_set) of basic block A and basic block B are {tr1, tr2} and {tr2} respectively. Since the input tensor register set (in_tr_set) of basic block C is equal to the output tensor register set (out_tr_set) of basic block B, and the input tensor register set (in_tr_set) of basic block C is not equal to the output tensor register set (out_tr_set) of basic block A, basic block A is determined to be the designated predecessor basic block, tr1 is the first tensor register (i.e., the tensor register in the output tensor register set of the designated predecessor basic block that is more than the input tensor register set of the second current basic block), wherein the first tensor register is a specific tensor register that does not exist in the output tensor register set of each predecessor basic block. tr2 is determined to be the second tensor register, wherein the second tensor register is a specific tensor register that exists in the output tensor register set of each predecessor basic block.

[0081] In step S505 , if the predecessor basic blocks of the first current basic block all create specific tensor registers, an instruction to release the specific tensor registers is configured at the entry of the first current basic block or inside the first current basic block.

[0082] Taking the state statistics diagrams 600 and 700 as examples, Figure 8 FIG. 8 is a schematic diagram showing a state statistics diagram 800 of a control flow diagram configuration release instruction processing completion. Figure 8As shown, after determining that tr2 is the second tensor register, in one embodiment, a new basic block is created at the entrance of the first current basic block, and a release instruction (releasetr tr2) of tr2 (the second tensor register) is inserted into the new basic block. In another embodiment, a release instruction (releasetr tr2) of tr2 (the second tensor register) can also be inserted inside the first current basic block (i.e., basic block C) before executing the relevant instructions of the second tensor register (e.g., createtr tr2). By configuring the instruction to release a specific tensor register at the entrance of the first current basic block or inside the first current basic block, it is only necessary to configure the release instruction of the tensor register at one place, so that the second tensor register can be released each time the program corresponding to the basic block is run, saving the configuration overhead of releasing the tensor register.

[0083] In step S506 , if any predecessor basic block of the first current basic block does not create a specific tensor register, an instruction to release the specific tensor register is configured at the exit of each predecessor basic block that creates the specific tensor register.

[0084] Take the state statistics diagrams 600, 700 and 800 as examples. Figure 8 As shown, since the input tensor register set (in_tr_set) of basic block C is not equal to the output tensor register set (out_tr_set) of basic block A, basic block A is determined to be the designated predecessor basic block, and tr1 is determined to be the first tensor register. Between the connection path between the second current basic block (basic block C) and the designated predecessor basic block (basic block A), a new basic block (releasetr tr1) is created at the exit of the designated predecessor basic block (basic block A), and a release instruction (releasetrtr1) of the first tensor register (tr1) is inserted into the new basic block. If there is a first tensor register, it means that not every predecessor basic block needs to release the first tensor register. If the processing method of releasing the tensor register at the entrance of the unified current basic block or the internal tensor register of the current basic block is followed, the tensor register needs to be released once for each execution of the program corresponding to the basic block. For some predecessor basic blocks that do not create specific tensor registers, this release operation is redundant, and simply increases the release overhead without producing a substantial effect. Therefore, an instruction to release a specific tensor register can be configured at the exit of each predecessor basic block that creates a specific tensor register, so that when the program corresponding to the basic block is run, only the predecessor basic block that requires the release needs to perform the release operation, which can save unnecessary release overhead.

[0085] Figure 5The embodiment describes a specific implementation method for performing state flow information statistics and release analysis on basic blocks in a scenario where there is no loop operation between basic blocks in a control flow graph, thereby saving unnecessary release overhead (for example, for tensor register tr1, the operation of releasing tensor register tr1 is only required when running from basic block A to basic block C; and when running from basic block B to basic block C, there is no need to execute the operation of releasing tensor register tr1), and also saves the configuration overhead of releasing tensor registers (the running path from basic block A to basic block C, and the running path from basic block B to basic block C, only need to configure a release instruction once at the entrance of basic block C or inside basic block C), thereby improving the state management efficiency of tensor registers.

[0086] Fig. 9 Schematically shows a flow chart of a tensor register management method 900 in an embodiment of the present disclosure. In the actual analysis scenario of the control flow graph, it is necessary to consider all input tensors and output tensors of each basic block. If there is a loop operation between the basic blocks of the control flow graph (that is, the output of the subsequent basic block will be used as the input of the previous basic block), it is impossible to accurately know all the input tensors and output tensors of the basic block by only traversing the basic blocks in the control flow graph once; the embodiment of the present disclosure proposes a solution, please refer to Fig. 9 It should be understood that Fig. 9 The tensor register management method shown may include a state statistics process of the tensor register, Fig. 9 The embodiment shown can be considered as Figure 3 One implementation of the embodiment, therefore regarding Figure 3 The description also applies to Fig. 9 .

[0087] In step S901, the first basic block in the control flow graph is obtained as the second current basic block.

[0088] Since it is necessary to release the tensor registers, it is necessary to first consider the situation of the tensor registers in the predecessor basic block. Therefore, in order to improve the analysis efficiency of the basic block, the reverse order can be used as the acquisition order of the basic block.

[0089] Fig.10 1 is a schematic diagram showing a state statistics diagram 1000 before a control flow graph is processed. The state statistics diagram 1000 is the initial state of the control flow graph before processing, and all state flow information is an empty set. Taking the state statistics diagram 1000 as an example, the predecessor basic block of basic block D is basic block C, and the predecessor basic blocks of basic block C are basic block A, basic block B and basic block C. Basic block A and basic block B have no predecessor basic blocks. According to the reverse order rule, basic block A or basic block B is obtained first, basic block C is obtained second, and basic block D is obtained last.

[0090] When the global statistics of state flow information are performed for the first time, each basic block in the control flow graph is initialized before obtaining the basic block for state flow information statistics. Specifically, the initialization operation includes: clearing the state flow information such as the input tensor register set, output tensor register set, and register set to be released of all basic blocks, and initializing the state flag bits of all basic blocks to the "invalid" state.

[0091] Specifically, the state flow information (tr_state_flow_info) in the embodiment of the present disclosure includes: an input tensor register set, an output tensor register set, an intersection tensor register set (all_in_tr_set) and a register set to be released (insert_releasetr_stmt_bef).

[0092] The input tensor register set (in_tr_set) is used to record the status of all tensor registers at the entry of the basic block, and is updated as the status of the output tensor registers of the predecessor basic block of the basic block changes. In the embodiment of the present disclosure, the value of the input tensor register is determined according to the intersection tensor register set of the basic block. Specifically, the intersection tensor register set is the intersection of the output tensor register sets of all predecessor basic blocks in the basic block.

[0093] The output tensor register set (out_tr_set) is used to record the status of all tensor registers at the exit of the basic block, and is updated as the tensor register status inside the basic block is analyzed and calculated. In the embodiment of the present disclosure, the value of the output tensor register set is determined based on all tensor register creation instructions inside the basic block. For example, if the basic block has two tensor register creation instructions "createtr tr1" and "createtr tr2", the output tensor register set includes "tr1" and "tr2".

[0094] The register set to be released (insert_releasetr_stmt_bef) is used to record the statements of the code that need to insert tensor register release instructions.

[0095] The status flag is used to mark whether the basic block is in a stable state (i.e., the intersection tensor register set of the basic block no longer changes in two adjacent global statistics), wherein "invalid" indicates stability, and "valid" indicates instability. Specifically, whether the second current basic block is stable can be determined based on the intersection tensor register set of the second current basic block in two adjacent global statistics; if the intersection tensor register sets in the two adjacent global statistics are equal, the second current basic block is determined to be stable, and the status flag is marked as "invalid"; if the intersection tensor register sets in the two adjacent global statistics are not equal, the second current basic block is determined to be unstable, and the status flag is marked as "valid"; when the basic blocks are all stable, all predecessor basic blocks of each basic block are determined.

[0096] In step S902, it is determined whether all basic blocks in the current global statistics have completed the statistics of state flow information.

[0097] If there are basic blocks that have not been counted, step S903 is executed to determine the intersection tensor register set of the second current basic block.

[0098] If there is no uncounted basic block, the global counting ends and step S907 is executed to determine whether the status flags of all basic blocks are invalid.

[0099] Fig.111100 schematically shows a state statistics diagram after a control flow graph completes the first round of traversal. After the first round of traversal (i.e., the first global statistics are completed), basic block A completes the statistics and obtains: its intersection tensor register set (all_in_tr_set) is an empty set (because it has no predecessor basic block), the input tensor register set (in_tr_set) is an empty set, the output tensor register set (out_tr_set) is {tr1, tr2}, and the register set to be released (insert_releasetr_stmt_bef) is an empty set. Basic block B completes the statistics and obtains: its intersection tensor register set (all_in_tr_set) is an empty set (because it has no predecessor basic block), the input tensor register set (in_tr_set) is an empty set, the output tensor register set (out_tr_set) is {tr2}, and the register set to be released (insert_releasetr_stmt_bef) is an empty set. The statistics of basic block C show that its intersection tensor register set (all_in_tr_set) is an empty set (because there is a loop, basic block C has an output tensor register set of a predecessor basic block that has not been obtained, so the intersection cannot be obtained), the input tensor register set (in_tr_set) is an empty set, the output tensor register set (out_tr_set) is {tr1, tr2}, and the register set to be released (insert_releasetr_stmt_bef) is an empty set. The statistics of basic block D show that its intersection tensor register set (all_in_tr_set) is {tr1, tr2}, the input tensor register set (in_tr_set) is {tr1, tr2}, the output tensor register set (out_tr_set) is an empty set (because it corresponds to only an exit instruction and does not perform any tensor register creation operation), and the register set to be released (insert_releasetr_stmt_bef) is an empty set.

[0100] Fig.121200 schematically shows a state statistics diagram after a control flow graph completes the second round of traversal. After the second round of traversal (i.e., the second global statistics are completed), basic block A completes the statistics and obtains: its intersection tensor register set (all_in_tr_set) is an empty set (because it has no predecessor basic block), the input tensor register set (in_tr_set) is an empty set, the output tensor register set (out_tr_set) is {tr1, tr2}, and the register set to be released (insert_releasetr_stmt_bef) is an empty set. Basic block B completes the statistics and obtains: its intersection tensor register set (all_in_tr_set) is an empty set (because it has no predecessor basic block), the input tensor register set (in_tr_set) is an empty set, the output tensor register set (out_tr_set) is {tr2}, and the register set to be released (insert_releasetr_stmt_bef) is an empty set. The statistics of basic block C are as follows: its intersection tensor register set (all_in_tr_set) is {tr2}, its input tensor register set (in_tr_set) is {tr2}, its output tensor register set (out_tr_set) is {tr1, tr2}, and its register set to be released (insert_releasetr_stmt_bef) is {createtr tr2}. The statistics of basic block D are as follows: its intersection tensor register set (all_in_tr_set) is {tr1, tr2}, its input tensor register set (in_tr_set) is {tr1, tr2}, its output tensor register set (out_tr_set) is an empty set (because it corresponds to only an exit instruction and does not perform any tensor register creation operation), and its register set to be released (insert_releasetr_stmt_bef) is an empty set.

[0101] In step S903 , the intersection tensor register set of the second current basic block is determined.

[0102] Specifically, the intersection of the output tensor register sets of all predecessor basic blocks in the second current basic block is calculated as the intersection tensor register set of the second current basic block.

[0103] by Fig.10 and Fig.11 For example, assume that the first basic block to perform state flow information statistics is basic block A. As shown in state statistics diagram 1100, after basic block A completes the statistics, its intersection tensor register set (all_in_tr_set) is an empty set (because it has no predecessor basic block).

[0104] by Fig.11For example, if the second current basic block is updated to basic block C, the predecessor basic blocks of basic block C are basic block A, basic block B and basic block C (such as Fig.11 As shown in the figure, the output of basic block C is also used as the input of one of the branches of basic block C, so basic block C is also used as its own predecessor basic block). Because there is a loop at basic block C, basic block C is not stable after only one global statistics, so the intersection of all its predecessor basic blocks cannot be obtained after the first round of traversal.

[0105] by Fig.12 Take an example to illustrate, if the second current basic block is updated to basic block C, the predecessor basic blocks of basic block C are basic block A, basic block B and basic block C. In the second round of traversal, it is known that the output tensor register set (out_tr_set) of basic block C is {tr1, tr2}, and the output tensor register sets (out_tr_set) of basic block A and basic block B are {tr1, tr2} and {tr2} respectively, therefore, the intersection tensor register set (all_in_tr_set) of basic block C can be obtained as {tr2}.

[0106] In step S904 , it is determined whether the second current basic block performs state flow information statistics of tensor registers for the first time, or whether the intersection tensor register set of the second current basic block is not equal to its input tensor register set.

[0107] If so, the state flag bit of the second current basic block is set to valid, and step S905 is executed to perform state flow information statistics of the tensor registers of the second current basic block.

[0108] If not, the state flag bit of the second current basic block is set to invalid, and the process returns to step S902 to determine whether all basic blocks have completed the statistics of state flow information.

[0109] When performing the first round of traversal, assuming that the second current basic block is basic block C, since basic block C is not stable after only one global statistics, the intersection tensor register set of basic block C is an empty set at this time, and step S905 has not been executed in the first round of traversal (that is, the value of the intersection tensor register set has not been assigned to the input tensor register set), so the input tensor register set of basic block C is also an empty set at this time. Therefore, at this time, the intersection tensor register set of basic block C is equal to the input tensor register set, but since this is the first time that the state flow information statistics of the tensor registers are executed for the second current basic block, the state flag bit of basic block C is set to valid when performing the first round of traversal, and step S905 is executed.

[0110] When performing the second round of traversal, it is assumed that the second current basic block is basic block C, and the intersection tensor register set of basic block C is {tr2} at this time, and the first round of traversal has ended (that is, when the first round of traversal executes step S905, the null value of the intersection tensor register set of basic block C at that time is assigned to the input tensor register set of basic block C), so the input tensor register set of basic block C is an empty set. Therefore, at this time, the intersection tensor register set of basic block C is not equal to the input tensor register set, and the state flow information statistics of the tensor registers are executed for the first time by the non-second current basic block, so the state flag bit of basic block C is set to valid when performing the second round of traversal, and step S905 is executed.

[0111] When performing the third round of traversal, it is assumed that the second current basic block is basic block C. At this time, the intersection tensor register set of basic block C is {tr2}, and the second round of traversal has ended (that is, when the second round of traversal executes step S905, {tr2} of the intersection tensor register set of basic block C at that time is assigned to the input tensor register set of basic block C), so the input tensor register set of basic block C at this time is {tr2}. Therefore, at this time, the intersection tensor register set of basic block C is equal to the input tensor register set, and the state flow information statistics of the tensor registers are executed for the first time for the non-second current basic block, so when performing the third round of traversal, the state flag bit of the second current basic block is set to invalid, and the execution returns to step S902 (that is, the third round of traversal no longer performs the state flow information statistics of basic block C).

[0112] In the actual analysis scenario of the control flow graph, it is necessary to consider all the input tensors and output tensors of each basic block. If there are loop operations between the basic blocks of the control flow graph (such as Fig.10 The output of basic block C in the control flow graph is also used as one of its own inputs), then only one traversal is performed on the basic blocks in the control flow graph, and all input tensors and output tensors of the basic blocks cannot be accurately known. Therefore, the embodiment of the present disclosure sets a state flag for each basic block, marked as "invalid" to indicate stability, and marked as "valid" to indicate instability. Among them, a basic block in a stable state indicates that the intersection tensor register set of the basic block no longer changes in two adjacent global statistics. In the control flow graph, if a basic block is in an unstable state, it means that in the next global statistics, in addition to the basic block in the unstable state, the intersection tensor register sets of other basic blocks in the control flow graph will also change, resulting in inaccurate analysis of the control flow graph. Therefore, in the embodiment of the present disclosure, it will iterate until all basic blocks are in a stable state before confirming that the statistical process of the state flow information in the control flow graph is completed.

[0113] In step S905 , statistics of state flow information of tensor registers are performed on the second current basic block.

[0114] Specifically, the temporary register set and the to-be-released register set of the second current basic block are initialized, the intersection tensor register set of the second current basic block is obtained, and the number values ​​in the intersection tensor register set are assigned to the input tensor register set and the temporary register set (tmp_in_tr_set) of the second current basic block.

[0115] Get the tensor register creation instructions (createtr tr) inside the second current basic block one by one, and determine whether the currently obtained tensor register creation instruction (the number corresponding to the tensor register) is in the temporary register set (tmp_in_tr_set). If so, add the number value (such as tr1 or tr2) in the tensor register creation instruction set to the register set to be released (insert_releasetr_stmt_bef) of the second current basic block; if not, add the number value (such as tr1 or tr2) in the tensor register creation instruction set to the temporary register set (tmp_in_tr_set) of the second current basic block until all tensor register creation instructions inside the second current basic block are processed, assign the value of the temporary register set (tmp_in_tr_set) in the second current basic block to the output tensor register set (out_tr_set) of the second current basic block, and the state flow information statistics of the second current basic block are completed.

[0116] In step S906, the next basic block of unexecuted state flow information statistics in the control flow graph is obtained and updated as the second current basic block.

[0117] After the current basic block statistics are completed, the next basic block in the control flow graph that has not yet executed state flow information statistics is obtained and updated as the second current basic block. After the update, the execution returns to step S902 to determine whether all basic blocks have completed the state flow information statistics.

[0118] In step S907, it is determined whether the status flags of all basic blocks are invalid.

[0119] If so, the process of performing statistics on the state flow information of the tensor registers for each basic block in the control flow graph ends, and step S908 is executed to analyze the first current basic block in the control flow graph based on the statistical results of the state flow information.

[0120] If not, the process returns to step S901 to obtain the first basic block in the control flow graph and perform global statistics again.

[0121] In the steps of the embodiment of the present disclosure, it is determined whether all basic blocks are in a stable state by judging whether the status flags of all basic blocks are invalid. Specifically, if the status flag of a basic block is invalid, it means that the predecessor basic block of the basic block does not change in two adjacent global statistics (such as, the intersection tensor register sets of the basic block in two adjacent global statistics are equal), then it can be determined that the basic block is in a stable state. If the status flag of a basic block is valid, it means that the predecessor basic block of the basic block has changed in two adjacent global statistics (such as, the intersection tensor register sets of the basic block in two adjacent global statistics are not equal), then it can be determined that the basic block is in an unstable state.

[0122] In the control flow graph, if a basic block is in an unstable state, it means that in the next global statistics, in addition to the basic block in the unstable state, the intersection tensor register sets of other basic blocks in the control flow graph will also change, resulting in inaccurate analysis of the control flow graph. Therefore, in the embodiment of the present disclosure, it will be iterated until all basic blocks are in a stable state before confirming that the statistical process of the state flow information in the control flow graph is completed, and then executing the subsequent analysis steps.

[0123] In step S908, the first current basic block in the control flow graph is analyzed based on the statistical results of the state flow information.

[0124] Specifically, if the first current basic block and any of its predecessor basic blocks create an identical tensor register, the identical tensor register is a specific tensor register. If the predecessor basic blocks of the first current basic block all create specific tensor registers, then step S909 is executed to configure an instruction to release the specific tensor register at the entrance of the first current basic block or inside the first current basic block; if any of the predecessor basic blocks of the first current basic block does not create a specific tensor register, then step S910 is executed to configure an instruction to release the specific tensor register at the exit of each predecessor basic block that creates a specific tensor register.

[0125] Taking the state statistics diagram 1200 as an example, after the second traversal is completed, each basic block is statistically analyzed as shown in the state statistics diagram 1200. According to the reverse order rule, basic block A and basic block B have no predecessor basic blocks, so the first current basic block analyzed is basic block C, and the "intersection tensor register set (all_in_tr_set) of basic block C is {tr2}, the input tensor register set (in_tr_set) is {tr2}, and the output tensor register set (out_tr_set) is {tr1, tr2}". The predecessor basic blocks of basic block C are basic block A, basic block B and basic block C, and the output tensor register sets (out_tr_set) of these three basic blocks are {tr1, tr2}, {tr2} and {tr1, tr2} respectively. Since the input tensor register set (in_tr_set) of basic block C is not equal to the output tensor register set (out_tr_set) of basic block A and basic block C, basic block A and basic block C are determined to be designated predecessor basic blocks, and tr1 is the first tensor register (i.e., the tensor register in the output tensor register set of the designated predecessor basic block that is more than the input tensor register set of the first current basic block), wherein the first tensor register is a specific tensor register in the output tensor register set of the designated predecessor basic block that is more than the input tensor register set of the first current basic block, and the designated predecessor basic block is a predecessor basic block whose output tensor register set is not equal to the input tensor register set of the first current basic block. Since the input tensor register set (in_tr_set) of basic block C is equal to the output tensor register set (out_tr_set) of basic block B, tr2 is determined to be the second tensor register (i.e., a tensor register in the input tensor register set of the first current basic block, which exists in the output tensor register set of each predecessor basic block of the first current basic block), wherein the second tensor register is a specific tensor register that exists in the output tensor register set of each predecessor basic block of the first current basic block.

[0126] In step S909 , if the predecessor basic blocks of the first current basic block all create specific tensor registers, an instruction to release the specific tensor registers is configured at the entry of the first current basic block or inside the first current basic block.

[0127] Fig.13 1300 is a schematic diagram showing a state statistics of a control flow diagram configuration release instruction processing completion. Fig.13 As shown, after determining that tr2 is the second tensor register, in one embodiment, a release instruction (releasetr tr2) of tr2 (the second tensor register) can be inserted before executing related instructions of the second tensor register (such as createtr tr2) inside the first current basic block (i.e., basic block C).

[0128] In step S910 , if any predecessor basic block of the first current basic block does not create a specific tensor register, an instruction for releasing the specific tensor register is configured at the exit of each predecessor basic block that creates the specific tensor register.

[0129] Taking the state statistics diagram 1300 as an example, since the input tensor register set (in_tr_set) of basic block C is not equal to the output tensor register set (out_tr_set) of basic block A and basic block C, basic block A and basic block C are determined to be designated predecessor basic blocks, and tr1 is determined to be the first tensor register. Fig.13 As shown, for the specified predecessor basic block A, between the connection path between the first current basic block (basic block C) and the specified predecessor basic block A, a new basic block (releasetr tr1) is created at the exit of the specified predecessor basic block A, and a release instruction (releasetr tr1) of the first tensor register (tr1) is inserted into the new basic block. In addition, for the specified predecessor basic block C, between the path from the basic block C to itself, a new basic block (releasetr tr1) is created, and a release instruction (releasetr tr1) of the first tensor register (tr1) is inserted into the new basic block.

[0130] The embodiment of flowchart 900 describes a specific implementation method for performing state flow information statistics and release analysis on basic blocks in a scenario where there are loop operations between basic blocks in a control flow graph. When the basic blocks in the control flow graph are traversed and iterated using the method of flowchart 900, if the intersection tensor register set of any basic block does not change in two consecutive global statistics, it means that the basic block is in a stable state (the state flag is marked as invalid); when all basic blocks in the control flow graph are in a stable state, all state flow information of each basic block can be accurately counted, thereby providing effective data basis for subsequent analysis and decision-making.

[0131] The compiler of the processing device 203 of the aforementioned multiple embodiments performs compilation optimization based on the tensor register management method, thereby reducing the computing program of the state management overhead of the tensor register, and obtaining optimized machine code. The machine code is executed by the computing device 201, so that the computing device 201 can accelerate the operation of reasoning such as computer vision, speech, natural language processing, data mining, etc.

[0132] Another embodiment of the present disclosure is a computer-readable storage medium having a computer program code of a tensor register management method stored thereon. When the computer program code is executed by a processing device, the following is performed: Figure 3 , 5Another embodiment of the present disclosure is a computer program product, including a computer program of a tensor register management method, which implements the following when the computer program is executed by a processor: Figure 3 , 5 Another embodiment of the present disclosure is a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement Figure 3 , 5 and 9 are steps of the method shown.

[0133] The disclosed solution is based on the analysis of the data flow in the control flow graph, and proposes a set of management solutions for tensor registers, which can reasonably configure the release strategy of tensor registers, and can save the configuration overhead of releasing tensor registers while also saving unnecessary release overhead.

[0134] According to different application scenarios, the electronic equipment or device disclosed in the present invention may include a server, a cloud server, a server cluster, a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an automatic driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric light, a gas stove, and a range hood; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument and / or an electrocardiograph. The electronic equipment or device disclosed in the present invention can also be applied to the Internet, the Internet of Things, a data center, energy, transportation, public administration, manufacturing, education, power grid, telecommunications, finance, retail, construction sites, medical treatment and other fields. Further, the electronic equipment or device disclosed in the present invention can also be used in the cloud, edge end, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0135] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art will appreciate that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0136] In terms of specific implementation, based on the disclosure and teaching of the present disclosure, those skilled in the art can understand that several embodiments disclosed in the present disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them on the basis of considering the logical functions, and there may be other splitting methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0137] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located at the same location or distributed to multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0138] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs, etc. Further, the aforementioned storage unit or storage device may be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which may be, for example, a variable resistive memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

[0139] The foregoing content can be better understood in accordance with the following terms:

[0140] Clause A1. A tensor register management method, the method comprising: counting the tensor registers created by each basic block in a control flow graph; obtaining any basic block in the control flow graph as the current basic block; based on the statistical results, comparing whether all the predecessor basic blocks of the first current basic block have created specific tensor registers; and based on the comparison results, determining the release position of the tensor register in the control flow graph.

[0141] Clause A2. The method according to Clause 1, wherein the release position of the tensor register in the control flow graph is determined based on the comparison result, including: if all predecessor basic blocks of the first current basic block do not create a specific tensor register, then an instruction to release the specific tensor register is configured at the entry of the first current basic block or inside the first current basic block; and if any predecessor basic block of the first current basic block does not create the specific tensor register, then an instruction to release the specific tensor register is configured at the exit of each predecessor basic block that creates the specific tensor register; wherein the entry of the first current basic block and the exit of the predecessor basic block are both located between the connection path between the first current basic block and the predecessor basic block.

[0142] Clause A3. According to the method described in Clause A2, the statistics of the tensor registers created by each basic block in the control flow graph include: statistics of the state flow information of the tensor registers for each basic block in the control flow graph, and the state flow information includes: an input tensor register set and an output tensor register set.

[0143] Clause A4. The method described in Clause A3, wherein the state flow information statistics of tensor registers are performed on each basic block in the control flow graph, comprising: obtaining an unexecuted basic block of the control flow graph in reverse order as the second current basic block; and performing state flow information statistics on tensor registers of the second current basic block.

[0144] Clause A5. According to the method described in Clause A4, after the state flow information statistics of the tensor registers of the second current basic block are performed, it includes: judging whether all basic blocks have completed the state flow information statistics in the current global statistics; if there are still basic blocks that have not completed the statistics, continuing to execute the step of performing state flow information statistics of the tensor registers of the basic blocks that have not completed the statistics in the control flow chart; and if all basic blocks have completed the statistics, the current global statistics ends, and it is judged whether each basic block in the control flow chart is in a stable state. If not, global statistics are performed again for each basic block in the control flow chart. If so, the process of state flow information statistics ends, wherein each basic block is in a stable state when the predecessor basic block of each basic block does not change in two adjacent global statistics.

[0145] Clause A6. According to the method described in Clause A5, before performing state flow information statistics of the tensor register of the second current basic block, it also includes: determining whether the second current basic block is stable based on whether the predecessor basic block of the second current basic block changes in two adjacent global statistics; if the predecessor basic block of the second current basic block has not changed in two adjacent global statistics, then determining that the second current basic block is stable; if the predecessor basic block of the second current basic block has changed in two adjacent global statistics, then determining that the second current basic block is unstable.

[0146] Clause A7. According to the method described in Clause A6, the state flow information also includes: an intersection tensor register set, which is the intersection of the output tensor register sets of all predecessor basic blocks in a basic block; the method of determining whether the second current basic block is stable based on whether the predecessor basic block of the second current basic block changes in two adjacent global statistics includes: determining whether the second current basic block is stable based on the intersection tensor register set and the input tensor register set of the second current basic block; wherein, if the intersection tensor register set of the second current basic block is not equal to its input tensor register set, or the second current basic block executes the state flow information statistics of the tensor registers for the first time, then the second current basic block is determined to be unstable; if the intersection tensor register set of the second current basic block is equal to its input tensor register set, and it is not the first time that the second current basic block executes the state flow information statistics of the tensor registers, then the second current basic block is determined to be stable; the input tensor register set is equal to the intersection tensor register set of the second current basic block in the previous global statistics.

[0147] Clause A8. The method according to Clause A6 further comprises: marking whether the basic block is stable by setting a status flag in the basic block; wherein, if the status flag is invalid, the corresponding basic block is stable; and if the status flag is valid, the corresponding basic block is unstable.

[0148] Clause A9. According to the method described in Clause A3, the state flow information also includes: a register set to be released, the register set to be released is used to record the number value of the tensor register to be released; the state flow information statistics of the tensor register of a basic block include: obtaining the intersection tensor register set and the tensor register creation instruction set of the second current basic block; wherein the tensor register creation instruction set is a set of all tensor register creation instructions within the current basic block; assigning the number value in the intersection tensor register set to the input tensor register set and the temporary register set of the second current basic block; and comparing the number of each tensor register creation instruction in the tensor register creation instruction set with the number of the temporary register set; if the tensor register creation instruction and the temporary register set have the same number value, then record the same number value in the register set to be released; if the tensor register creation instruction and the temporary register set do not have the same number value, then assign the number value in the tensor register creation instruction to the output tensor register set of the second current basic block.

[0149] Clause A10. The method according to any one of clauses A3 to A8, wherein at the exit of each predecessor basic block that creates the specific tensor register, an instruction for releasing the specific tensor register is configured, including: if the input tensor register set of the first current basic block is not equal to the output tensor register set of any of its predecessor basic blocks, then a new basic block is created at the exit of the specified predecessor basic block, and a release instruction for the first tensor register is inserted into the new basic block, wherein the first tensor register is a specific tensor register in the output tensor register set of the specified predecessor basic block that is more than the input tensor register set of the first current basic block, and the specified predecessor basic block is a predecessor basic block whose output tensor register set is not equal to the input tensor register set of the first current basic block.

[0150] Clause A11. The method according to any one of clauses A3 to A8, wherein the instruction for releasing the specific tensor register is configured at the entry of the first current basic block or inside the first current basic block, including: if a tensor register in the input tensor register set of the first current basic block exists in the output tensor register set of each predecessor basic block of the first current basic block, then determining to create a new basic block at the entry of the first current basic block, and inserting the release instruction of the second tensor register into the new basic block; the second tensor register is a specific tensor register that exists in the output tensor register set of each predecessor basic block of the first current basic block.

[0151] Clause A12. The method according to any one of clauses A3 to A8, wherein the instruction for releasing the specific tensor register is configured at the entrance of the first current basic block or inside the first current basic block, including: if a tensor register in the input tensor register set of the first current basic block exists in the output tensor register set of each predecessor basic block of the first current basic block, then it is determined that the release instruction of the second tensor register is inserted before the relevant instructions of the second tensor register are executed inside the first current basic block; the second tensor register is a specific tensor register that exists in the output tensor register set of each predecessor basic block of the first current basic block.

[0152] Item A13. A computer-readable storage medium having stored thereon a computer program code of a tensor register management method, wherein when the computer program code is executed by a processing device, the method described in any one of Items A1 to A12 is executed.

[0153] Clause A14. A computer program product, comprising a computer program storing a tensor register management method, wherein when the computer program is executed by a processor, the steps of the method described in any one of Clauses A1 to A12 are implemented.

[0154] Clause A15. A processing device, comprising: a compiler, wherein the compiler is used to execute the steps of the method described in any one of Clauses A1 to A12.

[0155] The embodiments of the present disclosure are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A tensor register management method, characterized in that: The method comprises: Statistics of tensor registers created by each basic block in the control flow graph; Obtain any basic block in the control flow graph as a first current basic block; Based on the statistical results, comparing whether all predecessor basic blocks of the first current basic block have created specific tensor registers; and Based on the comparison results, the release position of the tensor register in the control flow graph is determined.

2. The method according to claim 1, characterized in that The step of determining the release position of the tensor register in the control flow graph based on the comparison result includes: If all predecessor basic blocks of the first current basic block have created a specific tensor register, configuring an instruction to release the specific tensor register at the entry of the first current basic block or inside the first current basic block; and If any predecessor basic block of the first current basic block does not create the specific tensor register, an instruction to release the specific tensor register is configured at the exit of each predecessor basic block that creates the specific tensor register; wherein the entry of the first current basic block and the exit of the predecessor basic block are both located between the connection path between the first current basic block and the predecessor basic block.

3. The method according to claim 2, characterized in that The statistics of tensor registers created by each basic block in the control flow graph include: State flow information statistics of tensor registers are performed for each basic block in the control flow graph, and the state flow information includes: an input tensor register set and an output tensor register set.

4. The method according to claim 3, characterized in that The performing state flow information statistics of tensor registers for each basic block in the control flow graph includes: Acquire a basic block of the control flow graph that has not been executed statistics in reverse order as a second current basic block; Perform state flow information statistics of tensor registers on the second current basic block.

5. The method according to claim 4, characterized in that After performing statistics on the state flow information of the tensor register of the second current basic block, the method includes: Determine whether all basic blocks have completed the statistics of state flow information in the current global statistics; If there are still basic blocks that have not completed statistics, continue to execute the step of performing statistics of state flow information of tensor registers for the basic blocks in the control flow graph that have not completed statistics; and If all basic blocks have completed statistics, the current global statistics ends, and it is determined whether each basic block in the control flow graph is in a stable state. If not, global statistics are performed again on each basic block in the control flow graph. If so, the process of state flow information statistics ends, wherein each basic block is in a stable state when the predecessor basic block of each basic block does not change in two adjacent global statistics.

6. The method according to claim 5, characterized in that Before performing state flow information statistics of tensor registers on the second current basic block, the method also includes: determining whether the second current basic block is stable according to whether a predecessor basic block of the second current basic block changes in two adjacent global statistics; if the predecessor basic block of the second current basic block does not change in two adjacent global statistics, determining that the second current basic block is stable; if the predecessor basic block of the second current basic block changes in two adjacent global statistics, determining that the second current basic block is unstable.

7. The method according to claim 6, characterized in that The state flow information also includes: an intersection tensor register set, the intersection tensor register set being the intersection of output tensor register sets of all predecessor basic blocks in a basic block; The step of determining whether the second current basic block is stable according to whether a predecessor basic block of the second current basic block changes in two adjacent global statistics comprises: determining whether the second current basic block is stable according to the intersection tensor register set and the input tensor register set of the second current basic block; Among them, if the intersection tensor register set of the second current basic block is not equal to its input tensor register set, or the second current basic block executes the state flow information statistics of the tensor registers for the first time, then the second current basic block is determined to be unstable; if the intersection tensor register set of the second current basic block is equal to its input tensor register set, and it is not the first time that the second current basic block executes the state flow information statistics of the tensor registers, then the second current basic block is determined to be stable; the input tensor register set is equal to the intersection tensor register set of the second current basic block in the last global statistics.

8. The method according to claim 6, characterized in that The method further comprises: Whether the basic block is stable is marked by setting a status flag in the basic block; if the status flag is invalid, the corresponding basic block is stable; if the status flag is valid, the corresponding basic block is unstable.

9. The method according to claim 3, characterized in that: The state flow information further includes: a register set to be released, the register set to be released being used to record the serial number value of the tensor register to be released; Statistics of tensor register state flow for a basic block, including: Obtaining an intersection tensor register set and a tensor register creation instruction set of the second current basic block; wherein the tensor register creation instruction set is a set of all tensor register creation instructions within the basic block; Assign the numbered values ​​in the intersection tensor register set to the input tensor register set and the temporary register set of the second current basic block; and The number of each tensor register creation instruction in the tensor register creation instruction set is compared with the number of the temporary register set; if the tensor register creation instruction and the temporary register set have the same number value, the same number value is recorded in the register set to be released; if the tensor register creation instruction and the temporary register set do not have the same number value, the number value in the tensor register creation instruction is assigned to the output tensor register set of the second current basic block.

10. The method according to any one of claims 3 to 8, characterized in that: The step of configuring, at the exit of each of the predecessor basic blocks that creates the specific tensor register, an instruction for releasing the specific tensor register comprises: If the input tensor register set of the first current basic block is not equal to the output tensor register set of any of its predecessor basic blocks, a new basic block is created at the exit of the specified predecessor basic block, and a release instruction of the first tensor register is inserted into the new basic block, wherein the first tensor register is a specific tensor register in the output tensor register set of the specified predecessor basic block that is more than the input tensor register set of the first current basic block, and the specified predecessor basic block is a predecessor basic block whose output tensor register set is not equal to the input tensor register set of the first current basic block.

11. The method according to any one of claims 3 to 8, characterized in that: The step of configuring, at the entry of the first current basic block or inside the first current basic block, an instruction for releasing the specific tensor register comprises: If a tensor register in the input tensor register set of the first current basic block exists in the output tensor register set of each predecessor basic block of the first current basic block, it is determined to create a new basic block at the entrance of the first current basic block, and insert a release instruction of a second tensor register in the new basic block; the second tensor register is a specific tensor register that exists in the output tensor register set of each predecessor basic block of the first current basic block.

12. The method according to any one of claims 3 to 8, characterized in that: The step of configuring, at the entry of the first current basic block or inside the first current basic block, an instruction for releasing the specific tensor register comprises: If a tensor register in the input tensor register set of the first current basic block exists in the output tensor register set of each predecessor basic block of the first current basic block, it is determined that a release instruction of the second tensor register is inserted before executing related instructions of the second tensor register inside the first current basic block; the second tensor register is a specific tensor register that exists in the output tensor register set of each predecessor basic block of the first current basic block.

13. A computer-readable storage medium having stored thereon a computer program code of a tensor register management method, wherein when the computer program code is executed by a processing device, the method according to any one of claims 1 to 12 is executed.

14. A computer program product, comprising a computer program storing a tensor register management method, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A processing device, characterized in that: The processing device comprises: a compiler, wherein the compiler is used to execute the steps of the method according to any one of claims 1 to 12.

Images