Performance optimization method and device of conditional branch instruction, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a conditional branch instruction performance optimization method and device, equipment and a medium, and the method comprises the steps: obtaining the instruction number of each branch in a conditional branch statement; under the condition that the instruction number of any branch is larger than a preset threshold value, a jump instruction is inserted into any branch of a conditional branch instruction, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold value is determined based on hardware characteristics of an artificial intelligence chip. According to the method, the device, the equipment and the medium provided by the invention, the jump instruction is inserted into the branch of which the instruction quantity exceeds the preset threshold value, so that a large number of invalid instructions can be effectively prevented from being executed, the performance overhead of instruction execution is remarkably reduced, and the instruction execution efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a performance optimization method, device, equipment and medium for conditional branch instructions. Background Art

[0002] Artificial Intelligence (AI) chips usually use a Single Instruction Multiple Threads (SIMT) architecture, in which the most basic execution unit is called a thread bundle, and each thread bundle contains multiple threads working in parallel. This design allows the chip to process multiple data points or computing tasks at the same time, significantly improving computing efficiency. In order to efficiently manage the execution status of these threads, AI chips introduce a mask register mechanism.

[0003] In the design and application of AI chips, conditional branch instructions are the key to supporting complex algorithm logic and efficient data processing. When an AI chip encounters a conditional branch instruction, it controls the activation state of each thread in the thread bundle according to the state of each bit in the mask register. If a thread meets the branch condition, the corresponding mask register bit is set to 1 (indicating that the thread is activated), otherwise it is set to 0 (indicating that the thread is inactive). This mechanism enables the chip to flexibly handle different execution paths and support complex logical judgment and data processing.

[0004] However, although the mask register mechanism provides AI chips with powerful conditional branch processing capabilities, there are still certain defects in practical applications, among which the most prominent is the performance loss caused by invalid instruction execution. Specifically, when all bits of the mask register are 0, all threads are inactive, but instructions are still sent to these threads for processing, but the processing results are not written to memory or other registers. Although these instructions do not actually change any data or state (because no threads are activated), they still consume valuable computing resources and time, causing unnecessary performance losses. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for optimizing the performance of a conditional branch instruction, so as to solve the defect of performance loss caused by the execution of invalid instructions by a chip when encountering a conditional branch instruction in the related art.

[0006] The present invention provides a performance optimization method for conditional branch instructions, comprising: Get the number of instructions for each branch in the conditional branch statement; When the number of instructions of any branch is greater than a preset threshold, a jump instruction is inserted into any branch of the conditional branch instruction, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0007] According to a performance optimization method for conditional branch instructions provided by the present invention, obtaining the number of instructions for each branch in the conditional branch statement includes: Get the conditional branch statements in the pseudocode; The conditional branch statement is parsed to obtain the number of instructions for each branch in the conditional branch statement.

[0008] According to a performance optimization method for conditional branch instructions provided by the present invention, obtaining the number of instructions for each branch in the conditional branch statement includes: Obtaining a conditional branch statement in the pseudocode, and generating a conditional branch instruction based on the conditional branch statement; The conditional branch instruction is parsed to obtain the number of instructions for each branch in the conditional branch instruction, and the number of instructions for each branch in the conditional branch instruction is used as the number of instructions for each branch in the conditional branch statement.

[0009] According to a performance optimization method for a conditional branch instruction provided by the present invention, the step of determining the preset threshold comprises: Based on the hardware characteristics of the artificial intelligence chip, an initial threshold is determined, and the initial threshold is fine-tuned to obtain the preset threshold.

[0010] According to a performance optimization method for a conditional branch instruction provided by the present invention, the initial threshold is fine-tuned to obtain the preset threshold, including: Obtaining a benchmark test program, wherein the benchmark test program includes branch statements with different numbers of instructions, wherein the instructions include jump instructions and common instructions; Run the benchmark program on the artificial intelligence chip and collect performance indicators when the program is running; Based on the performance indicator, determining the performance overhead of the jump instruction and the performance overhead of the common instruction; Based on the performance overhead of the jump instruction and the performance overhead of the common instruction, the initial threshold is adjusted to obtain the preset threshold.

[0011] According to a performance optimization method for conditional branch instructions provided by the present invention, the hardware characteristics include instruction execution delay and instruction cache size, and the instruction execution delay includes jump instruction execution delay and common instruction execution delay.

[0012] According to a performance optimization method for a conditional branch instruction provided by the present invention, each branch in the conditional branch statement includes a first branch and a second branch, and when the number of instructions in any branch is greater than a preset threshold, inserting a jump instruction in any branch of the conditional branch instruction comprises: When the number of instructions of the first branch is greater than a preset threshold, inserting a first jump instruction into the first branch of the conditional branch instruction, wherein the first jump instruction is used to instruct the program to jump to the second branch; When the number of instructions of the second branch is greater than a preset threshold, a second jump instruction is inserted into the second branch of the conditional branch instruction, and the second jump instruction is used to instruct the program to jump to a post-processing instruction, and the post-processing instruction is located after the first branch and the second branch.

[0013] The present invention also provides a performance optimization device for conditional branch instructions, comprising: An acquisition unit, used to acquire the number of instructions of each branch in a conditional branch statement; An insertion unit is used to insert a jump instruction into any branch of a conditional branch instruction when the number of instructions of any branch is greater than a preset threshold, wherein the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a performance optimization method for conditional branch instructions as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a performance optimization method for conditional branch instructions as described in any one of the above.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the performance optimization method for the conditional branch instruction described above is implemented.

[0017] The performance optimization method, device, equipment and medium of the conditional branch instruction provided by the present invention obtains the number of instructions of each branch in the conditional branch statement, and intelligently determines whether it is necessary to insert a jump instruction in the branch based on the comparison result of these numbers with the preset threshold value. When the number of instructions of any branch exceeds the preset threshold value, a jump instruction can be inserted in the branch. The jump instruction can effectively avoid the execution of a large number of invalid instructions, thereby significantly reducing the performance overhead and improving the efficiency of instruction execution. It not only solves the performance bottleneck problem caused by the excessive number of instructions in the branch, but also realizes the significant improvement and optimization of the performance of the chip when executing the conditional branch instruction. In addition, the preset threshold value is determined based on the hardware characteristics of the artificial intelligence chip, so that the present invention can be applied to chips with different hardware characteristics and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is one of the flow charts of the performance optimization method of the conditional branch instruction provided by the present invention; Figure 2 It is a schematic diagram of inserting jump instructions when the number of instructions provided by the present invention is large; Figure 3 It is a schematic diagram of generating conditional branch statement instructions provided by the present invention; Figure 4 This is the second flow chart of the performance optimization method of the conditional branch instruction provided by the present invention; Figure 5 It is a structural schematic diagram of a performance optimization device for conditional branch instructions provided by the present invention; Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] With the rapid development of artificial intelligence technology, the demand for computing performance is growing, and traditional processors can no longer meet the needs of large-scale data processing and complex model training. Therefore, chips designed specifically for artificial intelligence applications have emerged, such as graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), tensor processing units (TPUs), etc. These AI chips have been deeply optimized in terms of instruction set architecture, computing units, execution units, memory management, etc. to support efficient execution of artificial intelligence tasks.

[0022] The hardware structure of an AI chip usually includes an instruction emission unit and an execution unit. The emission unit is responsible for taking instructions from the instruction cache and sending them to the execution unit, while the execution unit is responsible for actually executing these instructions and processing data. Among them, the most basic execution unit is called a thread bundle, and each thread bundle contains multiple threads working in parallel. When AI chips handle complex tasks, such as the training and reasoning processes in deep learning algorithms, they need to perform a lot of calculations and data processing. These tasks often involve multiple steps and conditional judgments, so conditional branch instructions are needed to control the execution flow of the program. In order to support the execution of conditional branch instructions, AI chips introduce the key component of mask registers.

[0023] When the AI ​​chip encounters a conditional branch instruction, the emission unit sends it to the execution unit, which first checks the state of the mask register. The mask register is a bit vector corresponding to the number of threads in the thread bundle, and each bit represents the execution state of a thread (activated or inactivated). According to the conditional calculation result of the conditional branch instruction, the corresponding thread will be marked as activated or inactivated, that is, the corresponding bit in the mask register is set to 1 or 0. According to the value of the mask register, it can be determined which threads should execute the current instruction. If the mask register bit corresponding to a thread is 1, the thread will execute the current instruction; if it is 0, the thread will skip the execution of the current instruction. In this way, the AI ​​chip can flexibly control the execution flow of each thread in the thread bundle and implement complex conditional branch logic.

[0024] Although no thread will be activated to execute the instruction when all the bits in the mask register are 0, the instruction itself will still be sent to the execution unit and go through a series of processing flows. This is because the emission unit does not know the specific state of the mask register in the execution unit when issuing the instruction. The emission unit is only responsible for sending instructions to the execution unit according to the program sequence or instruction scheduling strategy. Therefore, even if no thread will execute the instruction in the end, the emission and processing of the instruction will still consume a certain amount of time and resources.

[0025] In addition, in some chip architectures, even if the bit of the mask register is 0, the thread will still execute instructions, but will not write the results to the memory or register, which can maintain the utilization of the execution unit, but will ignore the calculation results of the inactive threads. In this case, although these instructions do not actually change any data or state (because no thread is activated), they still consume valuable computing resources and time, resulting in unnecessary performance loss.

[0026] To this end, the present invention provides a method for optimizing the performance of conditional branch instructions to overcome the above-mentioned defects. Figure 1 FIG. 1 is one of the flow charts of the performance optimization method of the conditional branch instruction provided by the present invention. Figure 1 As shown, the method includes: Step 110, obtaining the number of instructions for each branch in the conditional branch statement; It should be noted that in the field of AI chips, conditional branch statements refer to statements in programming languages ​​(such as C, C++, etc.) or domain-specific languages ​​(such as model descriptions in deep learning frameworks) that are used to select different execution paths based on specific conditions, such as if-else statements in C language, which allow different code blocks to be executed based on the truth or falsity of conditional expressions. The instruction generator can convert conditional branch statements into conditional branch instructions. It first parses the conditional branch statements in the code, extracts the conditional expressions and the code blocks of each branch, and then converts the conditional expressions into corresponding comparison instructions based on the instruction set architecture of the artificial intelligence chip, and converts the code blocks of each branch into corresponding conditional branch instructions.

[0027] Here, instruction generator refers to a tool used to convert algorithms or logic in a programming language or domain-specific language into a sequence of instructions that can be directly executed by the chip. Conditional branch instructions are a type of instruction in the processor instruction set that allows different code paths to be selectively executed based on a certain condition (such as a comparison result).

[0028] Specifically, in a conditional branch statement, each branch refers to a different code block that is selected for execution based on the result of a conditional expression. For example, in an if-else structure, there is usually an if branch and an else branch. In order to obtain the number of instructions for each branch in a conditional branch statement, it can be performed in the instruction generation phase or the static analysis phase. In the static analysis phase, the conditional branch statements in the code can be directly traversed to calculate the number of instructions corresponding to each branch. It is also possible to calculate the number of instructions contained in the code block corresponding to each branch in the instruction generation phase by analyzing the conditional branch statements in the code and tracking the instruction sequence under each branch.

[0029] It is understandable that the number of instructions for each branch in a conditional branch statement refers to the number of instructions contained in each branch after the conditional branch statement is converted into conditional branch instructions. By obtaining the number of instructions for each branch in the conditional branch statement, it is possible to prepare for subsequent performance optimization analysis. If the number of instructions for a branch exceeds a preset threshold, a jump instruction can be inserted into the branch to optimize the performance of instruction execution.

[0030] Step 120, when the number of instructions of any branch is greater than a preset threshold, insert a jump instruction into any branch of the conditional branch instruction, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0031] It should be noted that a jump instruction refers to an instruction used to change the execution order of a program. It allows the program to skip the execution of certain instructions and directly jump to another address to continue execution. When the number of instructions in a branch is small, adding a jump instruction to the branch often leads to performance degradation. This is because the execution of jump instructions also brings additional performance overhead. Under normal circumstances, the instruction issuance unit maintains a smooth pipeline operation, that is, while issuing the current instruction, it reads subsequent instructions from the instruction cache. This read-while-issuing mechanism ensures an efficient instruction execution flow. However, when a jump instruction is encountered, this process is interrupted because the program needs to jump to a new address to execute instructions, which requires the issuance unit to wait for the instructions at the new address to be read into the cache before continuing to execute. This interruption and waiting process is particularly significant when the number of instructions is small, compared with the sequential execution of instructions, and will increase the overall execution time. Therefore, when the number of instructions is small, if a jump instruction is added to the branch, although invalid instructions can be avoided, the pipeline of instruction execution is disrupted, resulting in a performance overhead greater than the normal execution of instructions in sequence, thereby causing performance degradation.

[0032] On the contrary, in branches with a large number of instructions, although jump instructions also introduce a certain amount of additional overhead, the overhead of executing jump instructions becomes relatively small compared to the sequential execution of these unnecessary invalid instructions. Through jumps, the program can quickly locate and execute key code segments, avoiding the processing of a large number of invalid instructions, thereby improving the execution efficiency as a whole. Therefore, when the number of instructions is large, the reasonable use of jump instructions helps to improve program performance. In this regard, the embodiment of the present invention determines whether to insert a jump instruction in a branch by setting a reasonable preset threshold and comparing the number of instructions of the branch with the preset threshold.

[0033] Specifically, when the number of instructions in any branch is greater than the preset threshold, it indicates that the branch contains more instructions. If each thread in the thread bundle does not meet the branch condition, theoretically, it is not necessary to execute each instruction in the branch. However, in practice, these instructions in the branch will still be sent to each thread for processing. In this process, the chip consumes valuable resources such as computing resources, memory bandwidth, and cache space to execute these instructions that actually have no effect.

[0034] In order to avoid the performance loss caused by executing these instructions, when the number of instructions in a branch exceeds a preset threshold, a jump instruction can be inserted into the branch, so that when each thread in the thread bundle does not meet the branch condition, the jump instruction can skip the execution of these unnecessary instruction operations in the branch, thereby reducing the performance overhead of instruction execution and improving the efficiency of instruction execution. It should be understood that in conditional branch instructions, jump instructions are usually used in conjunction with conditional judgment instructions to determine whether to jump based on the result of the conditional judgment. In an embodiment of the present invention, by inserting a jump instruction in any branch when the number of instructions in the branch exceeds a preset threshold, this optimization strategy is targeted, and it only processes those branches that will have a significant impact on performance, avoiding unnecessary optimization overhead.

[0035] It is understandable that the above-mentioned preset threshold is a pre-set numerical limit, which is used to determine whether the number of instructions in the branch is too many, so as to determine whether a jump instruction needs to be inserted for performance optimization. Here, the hardware characteristics of the artificial intelligence chip, such as the architecture, processing power, and cache size, will affect the execution efficiency and performance bottleneck of the instructions. Therefore, the preset threshold can be determined according to the hardware characteristics of the artificial intelligence chip. It is also possible to further combine actual tests and historical data to analyze the impact of different numbers of branch instructions on program performance, so as to set a reasonable threshold. It should be understood that different chips have different processing capabilities for conditional branch instructions due to their different hardware characteristics such as architecture, processing power, and cache size. By setting the threshold according to the hardware characteristics of the specific chip, it can be ensured that the optimization scheme can maximize the performance potential of the chip.

[0036] The method provided by the embodiment of the present invention obtains the number of instructions in each branch of the conditional branch statement, and intelligently determines whether to insert a jump instruction in the branch according to the comparison result between these numbers and a preset threshold. When the number of instructions in any branch exceeds the preset threshold, a jump instruction can be inserted in that branch. By means of the jump instruction, a large number of invalid instructions can be effectively avoided, thereby significantly reducing the performance overhead and improving the instruction execution efficiency. It not only solves the performance bottleneck problem caused by too many instructions in the branch, but also significantly improves and optimizes the performance of the chip when executing conditional branch instructions. In addition, the preset threshold is determined based on the hardware characteristics of the AI chip, enabling the present invention to be applicable to chips with different hardware characteristics and having strong adaptability.

[0037] Based on any of the above embodiments, when the number of instructions in any branch is greater than the preset threshold, the performance can be optimized by inserting a jump instruction in that branch. Specifically, first, the instruction sequence of the branch can be analyzed, a suitable jump point can be selected in the branch, and then the corresponding jump instruction can be inserted at the jump point. Figure 2 It is a schematic diagram of inserting a jump instruction when the number of instructions provided by the present invention is large. As Figure 2 shown, in the pseudocode in the figure, it can be seen that if the condition m < n is satisfied, instructions A, B, C, D, E, F, G, and H are executed in sequence; otherwise, instructions a, b, c, d, e, f, g, and h are executed in sequence. Regardless of which branch is executed, the execution will ultimately reach the place where instruction X is executed. Here, pseudocode is an informal, programming language-like way to describe an algorithm. It does not follow the syntax rules of any specific programming language but is clear enough to express the logical structure of the algorithm. The main purpose of pseudocode is to help users understand and design algorithms rather than directly execute them on a computer.

[0038] In an AI chip, assuming that a warp includes 32 threads, usually these 32 threads execute instructions synchronously. When executing a conditional branch instruction, there are three cases: one is that all threads satisfy the condition m < n, and at this time, all 32 bits of the mask register are 1; another is that all threads do not satisfy the condition m < n, and at this time, all 32 bits of the mask register are 0; and there is also a case where some threads satisfy the condition m < n and some threads do not satisfy the condition m < n, and at this time, the 32 bits of the mask register include 0 and 1.

[0039] When converting pseudocode to conditional branch instructions, it is default to use masked instructions to implement. It should be understood that when executing conditional branch instructions, a mask register can be used to control the execution status of all threads in a warp. If the bit in the mask register corresponding to a thread is 0, it is considered "masked". Even if the corresponding instruction is executed, the thread will not produce any effect. As Figure 2 shown, when implementing with masked instructions, first a comparison operation (compare m<n) is performed to determine whether m is less than n. According to the comparison result, a 32-bit mask mask is generated. Each bit in the mask represents whether a thread meets the condition m<n. For a certain thread, if m<n is true, the corresponding mask bit for that thread is set to 1; if it is false, it is set to 0. In this way, the mask mask reflects which threads in the warp meet the condition m<n. After the mask is generated, the program will execute sequentially. First, the instruction set in label 1 is executed. In label 1, instructions A~E will be determined whether to execute according to the mask mask. Specifically, if the bit corresponding to a thread in the mask is 1 (i.e., the thread meets m<n), the thread is in an active state and will sequentially execute the corresponding instructions A~E and obtain the corresponding execution results; if the mask bit is 0 (i.e., does not meet the condition m<n), the thread is in an inactive state, and even if these instructions are executed, the corresponding execution results will not be saved. After executing the instructions in label 1, the program reaches label 2 and starts to execute. In label 2, first the previously generated mask mask is inverted (i.e., all 1s in the mask are changed to 0s and all 0s are changed to 1s). The inverted mask represents all threads that do not meet the condition m<n. Subsequently, according to the inverted mask, the threads in the warp will sequentially execute instructions a~h. Here, inverting the mask mask is to activate the threads that do not meet the condition m<n, so that these threads can execute another set of different instructions and obtain the corresponding execution results. Finally, in label 3, the mask mask is reset (it can be set to all 0s or all 1s, or according to the specific implementation), and instruction X is executed. This step is usually used to clean up the state or prepare for the next round of calculations. The execution of instruction X is not affected by the mask mask and can perform the same operation on all threads.

[0040] It should be noted that when using masked instructions, if all 32 bits of the mask are 0, that is, all threads in the thread bundle are inactive, then executing the instruction set in label 1 will not have any effect, but these instructions will still consume valuable computing resources and time, causing unnecessary performance loss. Therefore, when there are a large number of instructions in label 1, performance optimization can be performed by inserting jump instructions in the branch. Similarly, if all 32 bits of the mask are 1, that is, all threads in the thread bundle are active, then all threads in the thread bundle will execute the instructions in label 1 in sequence, but when the instruction execution reaches label 2, since the mask is inverted, all its bits change from 0 to 1, and all threads in the thread bundle are switched to an inactive state. At this time, executing the instruction set in label 2 will not have any effect, but these unnecessary instructions will still cause a certain performance overhead. Therefore, when there are a large number of instructions in label 2, performance optimization can also be performed by inserting jump instructions in the branch.

[0041] like Figure 2 As shown in the figure, when using the branch jump instruction to implement, a comparison operation will be performed first to determine whether m is less than n. According to the comparison result, a 32-bit mask mask is generated, and then the program continues to execute to label 1. At this time, if all bits of the mask are 0 (that is, the mask values ​​are all false), the program will execute the jump instruction and jump directly to label 2 without executing instructions A~H in label 1; after executing instructions a~h in label 2, the program will continue to execute instructions in label 3. When the program executes to label 1, if all bits of the mask are 1 (that is, the mask values ​​are all true), the instructions A~H in label 1 will be executed in sequence based on the mask; after the execution is completed, the program reaches label 2. Since the mask values ​​are all true, the program will execute the jump instruction in label 2 and jump directly to label3 without executing instructions a~h in label 2. It should be understood that when the number of instructions in label 1 and label 2 is large, by inserting jump instructions in label 1 and label 2 respectively, it is possible to effectively avoid executing a large number of invalid instructions when mask is all false or all true, thereby reducing performance overhead and improving the efficiency of instruction execution.

[0042] For the case where the mask value is neither all false nor all true, the program will execute the instructions in label 1, label 2, and label 3 in order. First, in label 1, instructions A~H are executed in sequence based on the mask, which means that instructions A~H will only be executed on threads corresponding to bits 1 in the mask and the results will be saved; in label 2, the mask value is inverted, and then instructions a~h are executed in sequence based on the inverted mask, which allows threads that have not effectively executed instructions A~H before to perform corresponding operations and save the results; in label 3, the mask is reset to an initial state, and then an instruction X that is unrelated to the mask is executed. This instruction can be a general operation, cleanup work, or preparation for the next operation that needs to be performed after the previous conditional branch is executed.

[0043] Based on any of the above embodiments, step 110 specifically includes: Get the conditional branch statements in the pseudocode; The conditional branch statement is parsed to obtain the number of instructions for each branch in the conditional branch statement.

[0044] Specifically, in order to obtain the number of instructions of each branch in the conditional branch statement, it can be directly implemented by parsing the conditional branch statement in the pseudocode. Before parsing the conditional branch statement, the conditional branch statement in the pseudocode can be obtained by lexical analysis, syntax analysis and other technologies. For example, regular expressions or special lexical analyzer tools (such as Flex) can be used to identify tokens in the pseudocode, such as keywords (such as if, else), operators, identifiers, etc.; then write or use existing syntax analyzers (such as Bison, ANTLR, etc.) to convert the token sequence into an abstract syntax tree (Abstract Syntax Tree, AST) according to the grammatical rules of the pseudocode. In the AST, the conditional branch statement will be clearly represented as a node in the tree; finally, by traversing the AST and finding the node representing the conditional branch, the conditional branch statement in the pseudocode can be obtained.

[0045] In order to obtain the number of instructions in each branch of a conditional branch statement, we can start from the conditional branch node in the AST and traverse its direct child nodes, which usually represent conditional expressions, if branches, and else branches. For each branch, we further traverse its child nodes and calculate the number of all instructions in the branch. After completing the traversal of all branches, we can get the number of instructions in each branch of the conditional branch statement.

[0046] Based on any of the above embodiments, step 110 specifically includes: Obtaining a conditional branch statement in the pseudocode, and generating a conditional branch instruction based on the conditional branch statement; The conditional branch instruction is parsed to obtain the number of instructions for each branch in the conditional branch instruction, and the number of instructions for each branch in the conditional branch instruction is used as the number of instructions for each branch in the conditional branch statement.

[0047] Specifically, in order to obtain the number of instructions for each branch in the conditional branch statement, the conditional branch statement can also be converted into conditional branch instructions first, and then obtained by parsing the conditional branch instructions. Here, an instruction generator can be used to convert the conditional branch statement into a conditional branch instruction. Specifically, the pseudocode string can be first divided into a series of tokens, such as keywords (if, else), operators, identifiers, etc., by a lexical analyzer; then these tokens are combined into AST according to the grammatical rules of the pseudocode using a syntax analyzer; from the AST, an intermediate representation (IR) can be further generated. IR is a representation between source code and target code. It is closer to the structure of machine code, but has not yet been optimized for specific hardware. In IR, the conditional branch statement is converted into the corresponding control flow instruction. Finally, the instruction generator converts the IR into appropriate machine instructions according to the instruction set architecture of the target chip, and arranges and optimizes them in the correct order to generate the final conditional branch instruction.

[0048] After the conditional branch instruction is generated, the conditional branch instruction can be parsed to obtain the number of instructions for each branch. Specifically, the generated machine instruction sequence can be traversed to identify the conditional branch instruction and its target address, and the instruction sequence can be divided into different execution paths (i.e. branches) according to the target address of the conditional branch instruction; then, the instructions in each branch can be counted to obtain the number of instructions for each branch.

[0049] It is understandable that after obtaining the number of instructions for each branch in the conditional branch instruction, it can be directly used as the number of instructions for each branch in the conditional branch statement, because the number of instructions for each branch in the conditional branch instruction corresponds to the number of instructions for each branch in the conditional branch statement. The conditional branch statement describes the control flow of the program in pseudocode, and the conditional branch instruction implements the same control flow in machine code. When converting the pseudocode into machine instructions, the instruction generator will maintain the integrity of the control flow and ensure that the machine instructions can correctly reflect the logic in the pseudocode.

[0050] Based on any of the above embodiments, the step of determining the preset threshold includes: Step 210, based on the hardware characteristics of the artificial intelligence chip, determine the initial threshold, and fine-tune the initial threshold to obtain the preset threshold.

[0051] Specifically, the preset threshold for determining whether to insert a jump instruction in a branch can be determined based on the hardware characteristics of the artificial intelligence chip. For an artificial intelligence chip with a certain hardware characteristic, the threshold is relatively fixed. Specifically, an initial threshold can be preliminarily set by comprehensively considering hardware characteristics such as instruction execution delay and instruction cache size, through theoretical analysis, empirical data or experimental testing. Here, the initial threshold refers to a threshold for the number of instructions that is preliminarily set based on the hardware characteristics of the artificial intelligence chip and general experience. When the number of instructions in a conditional branch exceeds the threshold, consider inserting a jump instruction in the branch to optimize performance. However, since this threshold is preliminarily set and may not be completely accurate, it can be further fine-tuned through subsequent steps.

[0052] It is understandable that the hardware characteristics of artificial intelligence chips include instruction execution delay and instruction cache size. Here, instruction execution delay refers to the time required from the processor receiving the instruction to the completion of the instruction execution and the generation of results. This time interval includes the time required for instruction decoding, execution, and possible data transmission. Lower instruction execution delay means that the processor can process data faster, thereby improving overall performance. The instruction cache size refers to the size of the cache area inside the processor used to store recently accessed instructions. Among them, the above-mentioned instruction execution delay includes jump instruction execution delay and ordinary instruction execution delay. Jump instruction execution delay specifically refers to the delay when the processor executes a jump instruction, and ordinary instruction execution delay refers to the execution delay of all instructions except jump instructions. These instructions include arithmetic operations, logical operations, data transmission, etc.

[0053] When determining the initial threshold value based on the hardware characteristics of the artificial intelligence chip, it can be achieved by comprehensively considering the delay in executing the jump instruction, the delay in executing the ordinary computing instruction, and the size of the instruction cache. If the jump instruction execution delay is low and the instruction cache is small, then the initial threshold value will be smaller. For example, the jump instruction delay is about 10 clock cycles, the instruction cache can cache 8 instructions, and a computing instruction is issued every two clock cycles. In this case, the initial threshold value can be determined as 10. It should be understood that the threshold value can be determined based on the specific application scenario, taking into account factors such as the size of the instruction cache and the delay of the instruction.

[0054] After determining the initial threshold, the threshold can be further fine-tuned to obtain a more accurate instruction number threshold. Although executing jump instructions will bring certain performance overhead, executing a certain number of meaningless instructions also has a large overhead. If the overhead of executing a certain number of meaningless instructions is greater than the overhead of executing jump instructions, then using jump instructions can reduce the overhead to a certain extent and improve the efficiency of instruction execution. Therefore, the initial threshold can be fine-tuned by the following steps: First, a series of benchmark programs containing branch statements with different numbers of instructions are designed. These programs cover a variety of possible scenarios to comprehensively evaluate the performance overhead of jump instructions and ordinary instructions. Here, ordinary instructions refer to ordinary computing instructions that are masked and can be directly recognized and executed by computer processors, such as data operation instructions, data transfer instructions, etc. Then, run these benchmark programs on the target artificial intelligence chip and collect performance indicators when the program is running, such as execution time, cache hit rate, etc. According to the collected performance indicators, the performance overhead of jump instruction execution and ordinary instruction execution is analyzed. Finally, based on the analysis results, the initial threshold can be adjusted. For example, if the initial threshold is 10, when the performance overhead of executing 10 ordinary instructions is less than the performance overhead of executing a jump instruction, the threshold needs to be appropriately increased; conversely, if the performance overhead of executing a jump instruction is relatively small, the threshold can be appropriately lowered.

[0055] In the embodiment of the present invention, by determining the preset threshold value according to the actual hardware characteristics and the performance evaluation result, not only a more accurate threshold value can be obtained, but also the threshold value has higher flexibility and adaptability.

[0056] Based on any of the above embodiments, in step 210, the fine-tuning of the initial threshold to obtain the preset threshold includes: Step 211, obtaining a benchmark test program, wherein the benchmark test program includes branch statements of different numbers of instructions, wherein the instructions include jump instructions and common instructions; Step 212, running the benchmark program on the artificial intelligence chip and collecting performance indicators when the program is running; Specifically, a benchmark program refers to a tool used to measure the actual operating performance of chip hardware and the performance improvement effect of software optimization. Benchmark programs can be obtained through open source platforms or written by users. The benchmark program contains conditional branch statements with different numbers of instructions. For example, if the initial threshold is 10, the benchmark program can include conditional branches with different numbers of instructions such as 9, 10 or 11. By running these programs on the target artificial intelligence chip and testing their execution time and performance, the initial threshold can be fine-tuned according to the test results to obtain a more reasonable threshold, so that in most cases, better performance can be obtained using jump instructions.

[0057] It is understandable that before running the benchmark program on the artificial intelligence chip, you can prepare the test environment first, that is, ensure that the artificial intelligence chip and its supporting software and hardware environment have been built, including the operating system, driver, necessary library files, etc. Then, select a suitable benchmark program according to the requirements, or write the test program yourself according to the test objectives. Configure the parameters of the benchmark program according to the test requirements, such as test time, test data set, etc., then run the benchmark program on the artificial intelligence chip and record the test results. Here, the recorded test results are the performance indicators when the program is running, which may include instruction execution time, cache hit rate, etc. It should be understood that when collecting performance indicator data when the benchmark program is running on the artificial intelligence chip, it should be collected separately for branches containing jump instructions and branches that do not contain jump instructions.

[0058] Step 213, determining the performance overhead of the jump instruction and the performance overhead of the common instruction based on the performance indicator; Step 214: Based on the performance overhead of the jump instruction and the performance overhead of the common instruction, the initial threshold is adjusted to obtain the preset threshold.

[0059] Specifically, the collected performance indicator data is analyzed in depth, and the performance overhead of jump instruction execution is calculated based on the execution time, cache hit rate, resource consumption and other data of jump instructions. At the same time, the performance overhead of ordinary instruction execution is calculated based on the execution time, cache hit rate, resource consumption and other data of ordinary instructions. By comparing the performance overhead of jump instructions and ordinary instructions, the initial threshold can be adjusted to obtain the preset threshold.

[0060] It is understandable that the performance balance between jump instructions and ordinary instructions should be considered when adjusting the threshold. Specifically, if the performance overhead of executing 10 ordinary instructions is less than the performance overhead of executing jump instructions, it means that under the hardware characteristics of the current chip, it is more efficient to execute these instructions sequentially. At this time, jump instructions should not be inserted, so the threshold needs to be increased appropriately. When the threshold is adjusted to 12, if the performance overhead of executing 12 ordinary instructions is greater than the performance overhead of executing jump instructions, it means that jump instructions can bring performance improvement in the current scenario. At this time, the threshold can be retained as the preset threshold so that jump instructions can be inserted in branches with more than 12 instructions in the future.

[0061] Based on any of the above embodiments, each branch in the conditional branch statement includes a first branch and a second branch. Accordingly, step 120 specifically includes: When the number of instructions of the first branch is greater than a preset threshold, inserting a first jump instruction into the first branch of the conditional branch instruction, wherein the first jump instruction is used to instruct the program to jump to the second branch; When the number of instructions of the second branch is greater than a preset threshold, a second jump instruction is inserted into the second branch of the conditional branch instruction, and the second jump instruction is used to instruct the program to jump to a post-processing instruction, and the post-processing instruction is located after the first branch and the second branch.

[0062] Specifically, Figure 3 It is a schematic diagram of the conditional branch statement instruction generation provided by the present invention, such as Figure 3 As shown, in artificial intelligence chips, conditional branch instructions are usually two-way branches, including a first branch and a second branch, that is, Figure 3 Assume that the preset threshold is 5. In the first branch (i.e., the if branch), since the number of instructions exceeds the preset threshold, the first jump instruction can be inserted into the first branch when the conditional branch instruction is generated, i.e. Figure 3 The "mask is all false jump to label 2" instruction inserted in label 1 shown in the figure can avoid executing a large number of meaningless instructions when the mask values ​​are all false. It should be understood that the first jump instruction refers to the jump instruction inserted in the first branch of the conditional branch instruction, which is used to change the execution flow of the program under specific conditions. When a specific condition (such as mask is all false) is met, the first jump instruction instructs the processor to jump directly to the specified address (i.e., the instruction address in the second branch) to continue execution, rather than sequentially executing subsequent instructions in the first branch.

[0063] In the second branch (i.e., the else branch), since the number of instructions does not exceed the preset threshold, there is no need to insert a jump instruction to avoid the impact of the jump instruction on the instruction reading and instruction execution pipeline. In the second branch, the masked instructions are continued to be executed, that is, the thread executes the corresponding instructions according to the value of the mask register bit and saves the execution result.

[0064] like Figure 2 As shown, when the number of instructions of the second branch also exceeds the preset threshold, when generating the conditional branch instruction, a second jump instruction can be inserted into the second branch for optimization. Figure 2The "mask is all true jump to label 3" instruction inserted in label 2 shown in the figure can avoid executing a large number of meaningless instructions when the mask values ​​are all true. It should be understood that the second jump instruction refers to the jump instruction inserted in the second branch of the conditional branch instruction, which has a similar function to the first jump instruction, but has a different application scenario. The second jump instruction will instruct the processor to jump to another specified address (such as the address of the post-processing instruction in label 3) to continue execution when a specific condition (such as mask is all true) is met.

[0065] When the second jump instruction is triggered, it instructs the processor to start execution directly at the post-processing instruction, rather than continuing to sequentially execute subsequent instructions in the second branch. Here, the post-processing instruction is usually located after the two branches of the conditional branch instruction, and is used to process the common logic after the branch or perform necessary cleanup work. The post-processing instruction is a sequence of instructions executed after the conditional branch instruction, and is used to process subsequent work after the branch logic is completed, such as data aggregation, status update, resource release and other operations, to ensure the correctness and efficiency of the program. It should be understood that even if the second jump instruction is not triggered, after the first branch and the second branch are executed, the program will still reach the post-processing instruction and start execution.

[0066] Based on any of the above embodiments, Figure 4 FIG. 2 is a flow chart of the performance optimization method of the conditional branch instruction provided by the present invention. Figure 4 As shown, the method includes: Step S1, when using an instruction generator to generate operator instructions for an artificial intelligence chip, for a conditional branch instruction therein, the number of instructions for each branch in the conditional branch instruction may be first obtained; Step S2, if the number of instructions of a branch is greater than an empirical value (i.e., a preset threshold), a jump instruction can be inserted when a specific condition is met to avoid executing a large number of meaningless instructions; Step S3: If the number of instructions of a branch is not greater than the empirical value, there is no need to insert a jump instruction, so as to avoid the problem of greater performance overhead caused by the jump instruction.

[0067] The method provided by the embodiment of the present invention can avoid executing a large number of meaningless instructions by inserting jump instructions when there are a large number of instructions in the branch, and no jump instructions are inserted when there are a small number of instructions in the branch. By deciding whether to insert jump instructions based on the number of instructions in the conditional branch to optimize performance and improve the execution efficiency of instructions, the execution efficiency of operators is improved, and the training and reasoning efficiency of artificial intelligence models is further improved.

[0068] The performance optimization device for conditional branch instructions provided by the present invention is described below. The performance optimization device for conditional branch instructions described below and the performance optimization method for conditional branch instructions described above can be referenced to each other.

[0069] Based on any of the above embodiments, Figure 5 : is a schematic diagram of the structure of the performance optimization device for conditional branch instructions provided by the present invention, such as Figure 5 As shown, the device comprises: An acquisition unit 510 is used to acquire the number of instructions of each branch in a conditional branch statement; The insertion unit 520 is used to insert a jump instruction into any branch of the conditional branch instruction when the number of instructions of any branch is greater than a preset threshold, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0070] The device provided by the embodiment of the present invention obtains the number of instructions of each branch in the conditional branch statement, and intelligently determines whether it is necessary to insert a jump instruction in the branch based on the comparison result of these numbers with the preset threshold value. When the number of instructions of any branch exceeds the preset threshold value, a jump instruction can be inserted in the branch. The jump instruction can effectively avoid the execution of a large number of invalid instructions, thereby significantly reducing the performance overhead and improving the efficiency of instruction execution. It not only solves the performance bottleneck problem caused by the excessive number of instructions in the branch, but also realizes the significant improvement and optimization of the chip performance when executing conditional branch instructions. In addition, the preset threshold value is determined based on the hardware characteristics of the artificial intelligence chip, so that the present invention can be applied to chips with different hardware characteristics and has strong adaptability.

[0071] Based on any of the above embodiments, the acquiring unit 510 is specifically used for: Get the conditional branch statements in the pseudocode; The conditional branch statement is parsed to obtain the number of instructions for each branch in the conditional branch statement.

[0072] Based on any of the above embodiments, the acquiring unit 510 is specifically used for: Obtaining a conditional branch statement in the pseudocode, and generating a conditional branch instruction based on the conditional branch statement; The conditional branch instruction is parsed to obtain the number of instructions for each branch in the conditional branch instruction, and the number of instructions for each branch in the conditional branch instruction is used as the number of instructions for each branch in the conditional branch statement.

[0073] Based on any of the above embodiments, the device further includes a determining unit, where the determining unit is configured to: Based on the hardware characteristics of the artificial intelligence chip, an initial threshold is determined, and the initial threshold is fine-tuned to obtain the preset threshold.

[0074] Based on any of the above embodiments, the determining unit is specifically configured to: Obtaining a benchmark test program, wherein the benchmark test program includes branch statements with different numbers of instructions, wherein the instructions include jump instructions and common instructions; Run the benchmark program on the artificial intelligence chip and collect performance indicators when the program is running; Based on the performance indicator, determining the performance overhead of the jump instruction and the performance overhead of the common instruction; Based on the performance overhead of the jump instruction and the performance overhead of the common instruction, the initial threshold is adjusted to obtain the preset threshold.

[0075] Based on any of the above embodiments, the hardware characteristics include instruction execution delay and instruction cache size, and the instruction execution delay includes jump instruction execution delay and common instruction execution delay.

[0076] Based on any of the above embodiments, each branch in the conditional branch statement includes a first branch and a second branch. Accordingly, the insertion unit 520 is specifically configured to: When the number of instructions of the first branch is greater than a preset threshold, inserting a first jump instruction into the first branch of the conditional branch instruction, wherein the first jump instruction is used to instruct the program to jump to the second branch; When the number of instructions of the second branch is greater than a preset threshold, a second jump instruction is inserted into the second branch of the conditional branch instruction, and the second jump instruction is used to instruct the program to jump to a post-processing instruction, and the post-processing instruction is located after the first branch and the second branch.

[0077] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the performance optimization method of the conditional branch instruction, the method comprising: obtaining the number of instructions of each branch in the conditional branch statement; when the number of instructions of any branch is greater than a preset threshold, inserting a jump instruction in any branch of the conditional branch instruction, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0078] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the performance optimization method of conditional branch instructions provided by the above methods, and the method includes: obtaining the number of instructions for each branch in the conditional branch statement; when the number of instructions for any branch is greater than a preset threshold, inserting a jump instruction into any branch of the conditional branch instruction, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0080] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a performance optimization method for conditional branch instructions provided by the above methods is implemented. The method includes: obtaining the number of instructions for each branch in the conditional branch statement; when the number of instructions for any branch is greater than a preset threshold, inserting a jump instruction into any branch of the conditional branch instruction, wherein the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

[0081] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A performance optimization method for conditional branch instructions, characterized in that: include: Get the number of instructions for each branch in the conditional branch statement; When the number of instructions of any branch is greater than a preset threshold, a jump instruction is inserted into any branch of the conditional branch instruction, the conditional branch instruction is generated based on the conditional branch statement, and the preset threshold is determined based on the hardware characteristics of the artificial intelligence chip.

2. The performance optimization method of conditional branch instructions according to claim 1, characterized in that: The obtaining of the number of instructions for each branch in the conditional branch statement includes: Get the conditional branch statements in the pseudocode; The conditional branch statement is parsed to obtain the number of instructions for each branch in the conditional branch statement.

3. The performance optimization method of conditional branch instruction according to claim 1, characterized in that: The obtaining of the number of instructions for each branch in the conditional branch statement includes: Obtaining a conditional branch statement in the pseudocode, and generating a conditional branch instruction based on the conditional branch statement; The conditional branch instruction is parsed to obtain the number of instructions for each branch in the conditional branch instruction, and the number of instructions for each branch in the conditional branch instruction is used as the number of instructions for each branch in the conditional branch statement.

4. The performance optimization method of conditional branch instruction according to claim 1, characterized in that: The step of determining the preset threshold comprises: Based on the hardware characteristics of the artificial intelligence chip, an initial threshold is determined, and the initial threshold is fine-tuned to obtain the preset threshold.

5. The performance optimization method of conditional branch instruction according to claim 4, characterized in that: The fine-tuning of the initial threshold to obtain the preset threshold includes: Obtaining a benchmark test program, wherein the benchmark test program includes branch statements with different numbers of instructions, wherein the instructions include jump instructions and common instructions; Run the benchmark program on the artificial intelligence chip and collect performance indicators when the program is running; Based on the performance indicator, determining the performance overhead of the jump instruction and the performance overhead of the common instruction; Based on the performance overhead of the jump instruction and the performance overhead of the common instruction, the initial threshold is adjusted to obtain the preset threshold.

6. The performance optimization method of conditional branch instruction according to claim 4, characterized in that: The hardware characteristics include instruction execution delay and instruction cache size, and the instruction execution delay includes jump instruction execution delay and common instruction execution delay.

7. The performance optimization method of the conditional branch instruction according to any one of claims 1 to 5, characterized in that: Each branch in the conditional branch statement includes a first branch and a second branch, and when the number of instructions in any branch is greater than a preset threshold, inserting a jump instruction in any branch of the conditional branch instruction includes: When the number of instructions of the first branch is greater than a preset threshold, inserting a first jump instruction into the first branch of the conditional branch instruction, wherein the first jump instruction is used to instruct the program to jump to the second branch; When the number of instructions of the second branch is greater than a preset threshold, a second jump instruction is inserted into the second branch of the conditional branch instruction, and the second jump instruction is used to instruct the program to jump to a post-processing instruction, and the post-processing instruction is located after the first branch and the second branch.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the performance optimization method of the conditional branch instruction as claimed in any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the performance optimization method of the conditional branch instruction as claimed in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the performance optimization method of the conditional branch instruction as claimed in any one of claims 1 to 7 is implemented.

Images