Instruction jump prediction method and device, computer device, readable storage medium and program product

Abstract

The application relates to an instruction jump prediction method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: obtaining a to-be-processed instruction obtained through decoding; in the case that the to-be-processed instruction is identified as a conditional jump instruction, obtaining structural behavior information corresponding to the conditional jump instruction; performing feature extraction based on the structural behavior information to obtain a structural feature; obtaining a sensor corresponding to the conditional jump instruction; inputting the structural feature into the sensor to perform jump prediction and obtain a predicted jump result. The method can improve the accuracy of instruction jump prediction, thereby reducing instruction execution conflicts and reducing the waste of computer resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an instruction jump prediction method and device, computer equipment, readable storage medium and program product. BACKGROUND

[0002] In order to improve the efficiency of computer execution instructions, pipeline technology is introduced, which splits a task into multiple instructions and stagger their execution time to improve execution efficiency. However, pipeline data is very large, and there are instruction execution conflicts. In order to avoid instruction execution conflicts, the execution path of the instruction needs to be predicted in advance.

[0003] However, the prediction mechanism in the conventional technology (such as BHT, TAGE) usually relies on historical behavior or PC template. If there is no historical behavior or PC template of the model, the correctness of the predicted execution path of the instruction is very low, which will lead to the occurrence of instruction execution conflict, so a large number of clock cycles are needed to make up for it, which wastes a lot of computing resources. SUMMARY

[0004] Therefore, it is necessary to provide an instruction jump prediction method, device, computer equipment, readable storage medium and program product which can improve the accuracy of instruction jump prediction, reduce instruction execution conflicts and reduce the waste of computer resources.

[0005] In a first aspect, the present application provides an instruction jump prediction method, which comprises:

[0006] Obtaining a to-be-processed instruction obtained by decoding;

[0007] In the case where the to-be-processed instruction is identified as a conditional jump instruction, obtaining structural behavior information corresponding to the conditional jump instruction;

[0008] Performing feature extraction based on the structural behavior information to obtain a structural feature;

[0009] Obtaining a perceiver corresponding to the conditional jump instruction;

[0010] Inputting the structural feature into the perceiver to perform jump prediction and obtain a predicted jump result.

[0011] In one of the embodiments, the structural behavior information comprises at least one of a loop count recorded by a loop counter, a proportion of zero values in a current tensor corresponding to an instruction, a proportion of target instructions in a historical target number instruction, and a storage location of the instruction; and the structural feature comprises at least one of a critical jump point feature obtained based on the loop count, a sparsity feature obtained based on a proportion of zero values in a current tensor corresponding to an instruction and a jump threshold, an instruction type feature obtained based on a proportion of target instructions in a historical target number instruction and a distribution threshold, and a hash feature obtained by hashing the storage location of the instruction.

[0012] In one of the embodiments, the structural feature comprises a multi-level encoded feature for representing a jump probability; and the inputting the structural feature into the perceptron to perform jump prediction to obtain a predicted jump result comprises:

[0013] inputting the structural feature into the perceptron to obtain a jump probability;

[0014] taking the jump probability as the predicted jump result.

[0015] In one of the embodiments, the inputting the structural feature into the perceptron to perform jump prediction to obtain a predicted jump result comprises:

[0016] obtaining a weight vector and a bias vector of the perceptron;

[0017] calculating a dot product result based on the structural feature, the weight vector, and the bias vector;

[0018] obtaining a predicted jump result based on the dot product result.

[0019] In one of the embodiments, the method further comprises:

[0020] in a case where the predicted jump result is different from an actual jump result, correcting the weight vector based on the dot product result and the structural feature so that the dot product result of the structural feature approaches a dot product corresponding to the actual jump result, and correcting the bias vector based on the dot product result to adjust a classification boundary.

[0021] In one of the embodiments, the method further comprises:

[0022] correcting the perceptron in a case where the perceptron meets a correction condition.

[0023] The correction condition comprises at least one of the following conditions:

[0024] a number of times that a predicted jump result of the perceptron is different from an actual jump result is greater than a number threshold;

[0025] Or, no condition jump instruction corresponding to the sensor is identified within the preset time length.

[0026] Or, the sensor is an invalid sensor.

[0027] Or, the weight vector and the bias vector of the sensor are updated with overflow.

[0028] In one embodiment, the obtaining the sensor corresponding to the condition jump instruction comprises:

[0029] Calculating a hash value corresponding to the storage address of the condition jump instruction;

[0030] Querying the sensor corresponding to the condition jump instruction based on the hash value.

[0031] In one embodiment, the obtaining the sensor corresponding to the condition jump instruction comprises:

[0032] In the case of a multi-core processor, calculating a hash value corresponding to the storage address of the condition jump instruction;

[0033] XORing the hash value and the identifier of the core corresponding to the condition jump instruction to obtain a sensor index;

[0034] Querying the sensor corresponding to the condition jump instruction based on the sensor index.

[0035] In one embodiment, the method further comprises:

[0036] Obtaining the number of valid sensors corresponding to each core;

[0037] Replacing the sensors of the core whose number of valid sensors is greater than the valid threshold with the sensors of the core whose number of valid sensors is less than the valid threshold.

[0038] In one embodiment, the method further comprises:

[0039] In the case of a main branch predictor failure, continuing to execute the step of identifying the condition jump instruction; or

[0040] Obtaining the prediction result of the main branch predictor obtained in parallel with the predicted jump result, and obtaining a target jump result based on the prediction result of the main branch predictor and the predicted jump result.

[0041] In a second aspect, the present application also provides an instruction branch prediction device, which comprises:

[0042] An instruction obtaining module, configured to obtain a to-be-processed instruction obtained through decoding;

[0043] The identification module is configured to acquire structural behavior information corresponding to the conditional jump instruction in a case where the to-be-processed instruction is identified as a conditional jump instruction.

[0044] The feature extraction module is configured to perform feature extraction based on the structural behavior information to obtain structural features.

[0045] The perceiver acquisition module is configured to acquire a perceiver corresponding to the conditional jump instruction.

[0046] The prediction module is configured to input the structural features into the perceiver to perform jump prediction to obtain a predicted jump result.

[0047] In a third aspect, the present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0048] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method in any one of the above embodiments when executed by a processor.

[0049] In a fifth aspect, the present application also provides a computer program product, which includes a computer program, and the computer program implements the steps of the method in any one of the above embodiments when executed by a processor.

[0050] The instruction jump prediction method, device, computer device, readable storage medium and program product described above acquire a to-be-processed instruction obtained through decoding, acquire structural behavior information corresponding to the conditional jump instruction in a case where the to-be-processed instruction is identified as a conditional jump instruction, perform feature extraction based on the structural behavior information to obtain structural features, acquire a perceiver corresponding to the conditional jump instruction, input the structural features into the perceiver to perform jump prediction to obtain a predicted jump result. Since each structural layer has a determined start and end mode, and feature extraction is performed based on the structural behavior information, the jump prediction accuracy can be greatly improved, and thus the instruction execution conflict can be reduced and the waste of computer resources can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0052] Figure 1An application environment diagram of the instruction jump prediction method in an embodiment;

[0053] Figure 2 A flowchart of the instruction jump prediction method in an embodiment;

[0054] Figure 3 A flowchart of the instruction jump prediction method in another embodiment;

[0055] Figure 4 A structural block diagram of the instruction jump prediction device in an embodiment;

[0056] Figure 5 An internal structural diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0057] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0058] It should be noted that the terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two and more than two.

[0059] The instruction jump prediction method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 . In the application environment, an instruction decoding unit decodes an input instruction to obtain a to-be-processed instruction, and inputs the to-be-processed instruction into a jump predictor to determine whether to jump by prediction through the jump predictor. The jump predictor is used to optimize the cycle exit and layer boundary prediction in neural network model inference. In this way, since each structural layer has a determined start and end mode, feature extraction is performed based on structural behavior information, which can greatly improve the jump prediction accuracy, thereby reducing instruction execution conflicts and wasting computer resources.

[0060] In other embodiments, a main branch predictor can also be included, and the jump predictor serves as an auxiliary path of the main branch predictor. The main branch predictor performs jump prediction based on historical behavior or PC templates through existing prediction mechanisms (such as BHT, TAGE, etc.). The jump predictor performs jump prediction based on structural features to improve the accuracy of overall jump prediction.

[0061] After obtaining the jump prediction result, instruction scheduling is performed based on the jump prediction result to avoid conflicts in the instruction execution process, thereby improving the efficiency of instruction execution.

[0062] It should be noted that the instruction jump prediction method of the present application can be applied to the jump prediction of the structure of each neural network layer in the AI program, or can be applied to the ordinary program, and the specific limitation is not made herein. In the present application, the above instruction jump prediction method is applied to the jump prediction of the structure of each neural network layer in the AI program.

[0063] In an exemplary embodiment, as shown in Figure 2 , an instruction jump prediction method is provided, which is taken as an example of the jump predictor in Figure 1 , and includes the following steps S202 to S210. Among them:

[0064] S202: obtaining the decoded to-be-processed instruction.

[0065] The instruction decoding unit decodes the input instruction to obtain the to-be-processed instruction, and the decoding process can include the instruction address and / or decoding process, which is not specifically limited herein.

[0066] S204: in the case that the to-be-processed instruction is identified as a conditional jump instruction, obtaining the structural behavior information corresponding to the conditional jump instruction.

[0067] The conditional jump instruction is obtained by the instruction decoding unit decoding the to-be-processed instruction, and the instruction decoding unit can identify the instruction type to determine whether the to-be-processed instruction is a conditional jump instruction. Optionally, the instruction decoding unit can also output the jump PC (Program Counter, a memory address for storing the next to-be-executed instruction to ensure the continuous operation of the instruction pipeline) to the feature extractor and the perceiver.

[0068] In some optional embodiments, the structural behavior information includes at least one of the number of cycles recorded by the cycle counter, the proportion of zero values in the current tensor corresponding to the instruction, the proportion of target instructions in the historical target number instructions, and the instruction storage location.

[0069] Among them, each layer structure of the neural network model has a certain start and end mode, for example, the convolution layer performs fixed iterations in a loop, the activation function such as ReLU usually appears at a specific position, the execution contains a large number of CMP operations, in the quantization or sparse model, part of the layers have high sparsity, and the program execution address (PC) presents the characteristics of stage clustering. That is, there is a clear jump at these structural jump points. Therefore, the structural behavior information is obtained, which includes at least one of the number of cycles recorded by the cycle counter, the proportion of zero values in the current tensor corresponding to the instruction, the proportion of target instructions in the historical target number instructions, and the instruction storage location.

[0070] When a layer structure is unrolled by a for-loop, a jump usually occurs at the end of the loop. Therefore, the number of loops recorded by the loop counter can be used as the structural behavior information.

[0071] For some model structure layers, such as deep convolution or pruned MLP layers, there are sparse input tensors, and a jump can occur when the proportion of zero values is higher than a threshold. Therefore, the proportion of zero values in the current tensor corresponding to the instruction can be used as the structural behavior information.

[0072] Different structure layers of a neural network model have different distributions of instructions. For example, in an activation layer (such as ReLU), there are a significant proportion of comparison instructions such as CMP and MAX. The change in the type of such instructions can be used as an implicit signal of structural jumps to predict whether the current is entering or about to exit a certain phase. Therefore, the proportion of target instructions in the historical target number of instructions can be used as the structural behavior information.

[0073] The low bits of the instruction address can distinguish the structure blocks. For example, different Conv layers have different instruction addresses. Therefore, the instruction storage location can be used as the structural behavior information.

[0074] S206: Feature extraction based on the structural behavior information to obtain structural features.

[0075] The structural features are obtained by processing the structural behavior information by a feature extractor. The structural features include at least one of a critical jump point feature based on the number of loops, a sparsity feature based on the proportion of zero values in the current tensor corresponding to the instruction and a jump threshold, an instruction type feature based on the proportion of target instructions in the historical target number of instructions and a distribution threshold, and a hash feature obtained by hashing the instruction storage location.

[0076] Specifically, a conditional jump instruction corresponds to a group of counters recording the number of for-loop executions, such as 4, 5, 6, 7, 8, etc. The number of loops based on the counter can obtain the loop count value, and then obtain the corresponding critical jump point feature. Optionally, the width of the counter can include 8, and therefore the critical jump point feature can be represented as .

[0077] The sparsity feature can be obtained by comparing the proportion of zero values in the current tensor with the jump threshold. The jump threshold can be pre-set. In some optional embodiments, since the jump thresholds of different model structure layers are not the same, the sparsity feature can be obtained by multi-level encoding.

[0078] The instruction type feature is obtained based on the proportion of target instructions in the historical target quantity instruction and the distribution threshold, for example, the proportion of CMP and / or ACT type instructions in the past target quantity instruction, and the instruction type feature obtained by comparing the proportion with the distribution threshold. In some optional embodiments, since the distribution thresholds of different model structure layers are different, the instruction type feature can be obtained by multi-level coding.

[0079] The hash feature is obtained based on the hash of the instruction storage location. In actual neural network execution, especially when running the model on a CPU or general AI accelerator, each neural network layer will be compiled / generated as a specific code block (such as a loop, matrix multiplication, activation function, etc.):

[0080]

[0081] Although these layers have similar semantic behaviors, the addresses of conditional jump instructions are completely different. If a traditional branch predictor (such as TAGE) is used, a separate state needs to be maintained for each address, which lacks generalization. Using the hash value of the instruction can make the perceptron map different addresses to a unified structure behavior prediction object, realizing cross-layer semantic generalization prediction.

[0082] In the present application, the structural feature extraction is integrated in the processor, and the structural features are extracted in real time in the pipeline stage.

[0083] S208: Obtain a perceptron corresponding to the conditional jump instruction.

[0084] S210: Input the structural features into the perceptron to perform jump prediction to obtain a predicted jump result.

[0085] Each conditional jump instruction corresponds to a unique perceptron. When the conditional jump instruction is identified, the structural features are input into the perceptron to perform jump prediction to obtain a predicted jump result.

[0086] Each perceptron includes a weight vector and a bias vector, and each perceptron can further include a perceptron identifier. The perceptron identifier can be a hash value based on the storage address of the conditional jump instruction, and the hash value is used as the index of the perceptron array. Therefore, after the conditional jump instruction is identified, the storage address of the conditional jump instruction can be obtained based on the program counter of the conditional jump instruction, and then the hash value is calculated, and the corresponding perceptron is obtained based on the hash value.

[0087] The perceptron includes a weight vector and a bias vector, and the jump prediction can be performed based on the structural features, the weight vector, and the bias vector to obtain a predicted jump result.

[0088] In addition, it should be noted that after obtaining the predicted jump result, subsequent instruction execution can be scheduled based on the predicted jump result to avoid instruction conflicts, thereby reducing the additional occupation of computer resources due to instruction conflicts and improving resource utilization. Moreover, the instruction jump is predicted in advance, and the corresponding storage register can be allocated for the execution of the instruction in advance to reduce the waste of resources caused by fixed register allocation.

[0089] The instruction jump prediction method described above obtains the decoded instruction to be processed, and in the case where the instruction to be processed is identified as a conditional jump instruction, obtains the structural behavior information corresponding to the conditional jump instruction; based on the structural behavior information, feature extraction is performed to obtain structural features; a sensor corresponding to the conditional jump instruction is obtained; the structural features are input into the sensor to perform jump prediction to obtain a predicted jump result. Since each structural layer has a determined start and end mode, feature extraction based on structural behavior information can significantly improve the accuracy of jump prediction, thereby reducing instruction execution conflicts and reducing waste of computer resources.

[0090] In one of the optional embodiments, inputting the structural features into the sensor to perform jump prediction to obtain a predicted jump result includes: obtaining a weight vector and a bias vector of the sensor; calculating a dot product result based on the structural features, the weight vector, and the bias vector; obtaining the predicted jump result based on the dot product result.

[0091] The dot product result z = w · x + b, and the predicted jump result , wherein the sign operator is used to return the sign of an integer, that is The value is ±1.

[0092] Taking the 8-bit loop count value as an example , wherein x = [+1, +1, +1, +1, +1, +1, +1, -1], w = [+2, +1, 0, -1, -1, 0, +1, -2], and b = -5, so z = 4 + (-5) = -1, and thus = -1, wherein -1 represents no jump, and +1 represents jump.

[0093] In some optional embodiments, the method further includes: in the case where the predicted jump result is different from the actual jump result, correcting the weight vector based on the dot product result and the structural features to make the dot product result of the structural features close to the dot product corresponding to the actual jump result, and correcting the bias vector based on the dot product result to adjust the classification boundary.

[0094] Wherein, if the prediction is wrong (i.e. y ≠ ), the sensor weight is updated online as w + y · x, and the bias weight is updated as b + y.

[0095] Still with the above example, if the actual result is y = +1 (should jump but prediction does not jump), then the w is updated to [ +3, +2, +1, 0, 0, +1, +2, -3 ], and the b is updated to -4.

[0096] where the update of the weight vector is based on the dot product result and the structural feature, where if the prediction is wrong, it means that the dot product direction is deviated, so y · x is added to w to make the future dot product direction of x closer to the direction of y; that is, the feature weight of the "correct direction" is strengthened, and the feature weight of the "misleading direction" is weakened. The update of the bias weight is used to adjust the classification boundary as a whole; if the jump behavior fluctuates near the boundary, b will gradually deviate to the correct classification.

[0097] where when a layer structure is expanded by a for loop, a jump usually occurs at the end of the loop. Take i = 254 as the jump critical point as an example:

[0098]

[0099] The perceptron learns to identify the structural jump condition of the loop termination point using this feature. Through weight accumulation and bias adjustment, the perceptron can capture the critical point in training.

[0100] Some layers, such as deep convolution or pruned MLP layers, have sparse input tensors, and a jump may occur when the proportion of zero values is higher than a threshold:

[0101]

[0102] This feature participates in prediction by the sparsity of input data in the structural layer (such as the proportion of zero values), which can assist the perceptron in identifying the behavior feature that may produce a structural jump in the pruning and sparse stage. To simplify hardware implementation, the current version uses to represent whether the sparsity threshold (such as 60%) is exceeded; the perceptron will learn the relevance of the sparse stage and the jump behavior.

[0103] Different structural layers of a neural network model have different types of instruction distribution. For example, in an activation layer (such as ReLU), there will be a significant proportion of comparison type instructions such as CMP, MAX, etc. The change of this type of instruction can be used as an implicit signal of structural jump to predict whether the current is entering or about to exit a certain stage.

[0104]

[0105] This feature can be used to represent whether the current structural layer belongs to the activation or comparison stage, and in training, the perceptron will automatically learn the relevance of these stage changes and jump behavior. In some cases, only this feature can identify the jump trend, which is suitable for structural boundary judgment.

[0106] The jump instruction address low bit distinguishable structure block, such as different Conv layers, the instruction position is different, using PC[7:4] value:

[0107]

[0108] The feature as a lightweight indicator of program execution semantics, different structure positions can be distinguished by Hash. Although the PC address of the jump instruction has program locality, the sensor can generalize across layers through the Hash feature, that is, if the structure behaviors corresponding to different addresses are similar, the sensor will obtain consistent response through weight training.

[0109] In the above embodiment, the low-overhead hardware logic can be used to detect the path in real time before instruction decoding or jumping, and the online learning jump behavior can be realized. Even for new models, after a few training, accurate prediction can be realized.

[0110] In addition, in order to facilitate understanding, when multiple structural features participate in judgment together, the semantic expression ability of the sensor can be improved, for example, when the input structural features include:

[0111]

[0112] The input vector is merged: x=[+1,+1,+1,+1,+1,+1,+1,-1,+1,+1,+1,+1,-1,+1], the initial state of the weight vector is w=[+3,+2,+1,0,0,+1,+2,3,+1,0,+2,-2,0,0], and the bias vector b=-5.

[0113] Dot product calculation is performed: the first 8 items dot product: 3+2+1+0+0+1+2+(-3)=6, the last 6 items dot product: +1+0+2-2+0+0=+1, the bias vector: -5, z=6+1-5=2, so =+1. In other embodiments, the dot product calculation can also be calculated together without distinguishing the first 8 items and the last 6 items.

[0114] If the prediction is wrong (actual y=-1): then update: w is w-x (item by item subtraction), b is b-1, and the update example (first 3 items) is:

[0115]

[0116] The 14-bit is updated together, the perceiver can capture the structural jump rule in real time by fusing multi-source structural features, the online training has fast convergence speed, the hardware implementation cost is low, and the AI model structure prediction is suitable. In addition, if the perceiver has weight overflow, high misjudgment times, replace or zero management strategy is added, the replace or zero management strategy can be specifically seen below, the update frequency of the perceiver in this application is controlled within 5-10 times of misprediction convergence, which can adapt to the structural behavior change of the neural network at different stages.

[0117] In some optional embodiments, the structural features include multi-level encoded features for representing jump probability; the structural features are input into the perceiver to perform jump prediction to obtain a predicted jump result, including: inputting the structural features into the perceiver to obtain the jump probability; and taking the jump probability as the predicted jump result.

[0118] In this application, in order to improve the accuracy of the structural features, the jump probability is represented by multi-level encoding, for example, in the sparsity feature, if higher expression capability is needed, multi-level sparsity grading coding can be introduced, which can adapt to the sparsity features of different model structure layers, for example, the sparsity jump point of a certain model structure layer is 50%, and the sparsity jump point of a certain model is 75%. Therefore, multiple different levels of sparsity features can be introduced, for example, 2-bit is used to represent four levels: low, medium low, medium high, and high, to enhance the discrimination ability of the jump boundary and support a more detailed weight learning process. In other embodiments, other numbers of levels can also be introduced, but not limited to 2-bit.

[0119] For instruction type features, if higher expression capability is needed, multi-occupancy grading coding can be introduced, for example, 2-bit is used to represent four levels: low, medium low, medium high, and high. In other embodiments, other numbers of levels can also be introduced, but not limited to 2-bit.

[0120] Therefore, in this embodiment, the multi-level encoded structural features can be calculated, and then the perceiver processes the structural features to obtain the jump probability, thereby improving the accuracy of jump prediction.

[0121] In some optional embodiments, the method further includes: modifying the perceiver when the perceiver meets a modification condition; wherein the modification condition includes at least one of the following conditions: the number of times that the predicted jump result of the perceiver is different from the actual jump result is greater than a number threshold; or, a conditional jump instruction corresponding to the perceiver is not identified within a preset time period; or, the perceiver is an invalid perceiver; or, there is an overflow when the weight vector and the bias vector of the perceiver are updated.

[0122] The modification condition includes a replacement condition and / or a clear condition, which is used to replace the perceptron and / or clear the value of the corresponding register in the perceptron. The corresponding register in the perceptron can include a cycle counter, a weight vector of the perceptron, a bias vector, and the clear condition needs to clear the value of the register. When the perceptron is replaced, the clear condition needs to be cleared, and the state of the perceptron and the identification (such as the hash value described above) are updated.

[0123] In order to control resource consumption, the application also uses a perceptron pool to manage each perceptron, and each perceptron in the perceptron pool includes a perceptron state, an identification, a cycle counter, a weight vector, a bias vector, and a usage frequency, wherein the perceptron state is used to indicate whether the perceptron is valid, the identification is the hash value described above, and the usage frequency is represented by not identifying the condition jump instruction corresponding to the perceptron within a preset time period.

[0124] The total resource estimation can be based on the perceptron array, the number, and the bit width, wherein the size of the perceptron array is 2 k , k is determined by the hash bit width of the condition jump instruction, such as PC[7:4], which corresponds to 16 perceptrons, the perceptron array includes 16 perceptrons, the number of perceptrons is 16 groups (PC Hash = 4bit), and the total resource is: 16 groups * (16 (weight) x (5bit) + 5bit (bias item) + 4bit (PC hash) ≈ 1.4KB. In order to facilitate understanding, an embodiment of a weight is given, wherein the feature vector x = [Loop8, Sparsity2, Instr2, PCHash4], that is, the feature vector has a total of 16 bits ± 1, so the weight vector also has 16 bits, which corresponds to the number of bits of the feature vector. If more feature dimensions or hash bit widths of the condition jump instruction need to be supported (such as 32 or 64 perceptrons), there will be a linear increase. In other embodiments, the number of perceptrons in the perceptron array can be dynamically adjusted, such as supporting 16, 32, 64 perceptron instances, etc., which are not specifically limited here.

[0125] The cycle counter in the perceptron is incremented by 1 when the jump return starts, and is cleared when the jump offset is no longer jumped or the predictor is not hit. Optionally, a maximum counter can be set to prevent overflow, and the count value of the maximum counter can be 255.

[0126] For the dynamic detection type feature in the perceptron, the last state does not need to be reserved, only whether the current round exceeds the threshold is reserved; the instruction type feature is based on a sliding window (such as 64), and is updated once per cycle.

[0127] In the case where the above modification condition is met, the corresponding state is cleared or the perceptron is replaced.

[0128] In order to illustrate the clearing and replacing of the perceptron, an embodiment is given, in which embodiment, after the conditional jump instruction is identified, the perceptron corresponding to the conditional jump instruction is obtained, if not obtained, the existing perceptron in the perceptron pool is replaced, for example, whether the existing perceptron in the perceptron pool triggers the replacement condition (the replacement condition is the above-mentioned correction condition), if so, one of the perceptrons is selected for replacement, the replacement process can be referred to the above, so that the perceptron corresponding to the conditional jump instruction is obtained, then the jump prediction is performed, and the actual jump condition is recorded, if the two are inconsistent, the number of failures is recorded, wherein if the number of failures is greater than the number threshold, the perceptron can be cleared, in addition, if the weight overflows in the perceptron updating process, the perceptron can also be cleared.

[0129] In other embodiments, the perceptron pool can execute a management thread alone to periodically determine whether the perceptron in the perceptron pool satisfies the above-mentioned correction condition, if so, the corresponding perceptron is cleared or replaced.

[0130] In some optional embodiments, obtaining the perceptron corresponding to the conditional jump instruction comprises: calculating a hash value corresponding to the storage address of the conditional jump instruction; and querying the perceptron corresponding to the conditional jump instruction based on the hash value.

[0131] In the case of identifying the to-be-processed instruction as the conditional jump instruction, the hash value is calculated based on the storage address of the conditional jump instruction, that is, the identifier of the above-mentioned perceptron, so that the corresponding perceptron can be queried based on the identifier.

[0132] If no conditional jump instruction corresponding to the perceptron is identified within a preset time length, the perceptron can be replaced. For example, if no corresponding perceptron is queried based on the hash value, a replacement operation can be triggered to replace the perceptron which has not identified the conditional jump instruction corresponding to the perceptron within the preset time length. The replacement process needs to clear the corresponding vector, counter state, hash value, etc.

[0133] In some optional embodiments, obtaining the perceptron corresponding to the conditional jump instruction comprises: in the case of a multi-core processor, calculating a hash value corresponding to the storage address of the conditional jump instruction; performing an exclusive or operation on the hash value and the identifier of the core corresponding to the conditional jump instruction to obtain a perceptron index; and querying the perceptron corresponding to the conditional jump instruction based on the perceptron index.

[0134] In the case of a multi-core shared perceptron pool, the perceptron index is determined in combination with the identifier of the perceptron and the identifier of the core.

[0135] The multi-core processor can share a common perceptron for centralized training. The perceptron access of each core is mixed with the core identifier CORE ID through an XOR operation, so that the same hash value on different cores is mapped to different table entries, preventing conflicts. Cross-core migration is supported, which means filling the current core CORE's perceptron content into the shared common perceptron. The content remains unchanged, and the condition adjustment instruction identifier (hash value) needs to be XORed with the core identifier CORE ID.

[0136] In some optional embodiments, the method further comprises: obtaining the number of valid perceptrons corresponding to each core; and replacing the perceptron of the core whose number of valid perceptrons is greater than the valid threshold with the perceptron of the core whose number of valid perceptrons is less than the valid threshold.

[0137] In this application, due to limited resources, the number of shared common perceptrons occupied by different cores can be managed. For example, up to M valid perceptrons are retained per core, and those exceeding the number are replaced, for example, the perceptron of the core whose number of valid perceptrons is greater than the valid threshold is replaced with the perceptron of the core whose number of valid perceptrons is less than the valid threshold. This can balance the number of perceptrons corresponding to each core, thereby ensuring the prediction efficiency of each core.

[0138] In some optional embodiments, the method further comprises: in the case of failure of the main branch predictor, continuing to perform the step of identifying the conditional branch instruction; or obtaining the prediction result of the main branch predictor obtained in parallel with the prediction jump result, and obtaining the target jump result based on the prediction result of the main branch predictor and the prediction jump result.

[0139] In this embodiment, the jump predictor is deployed in parallel with the main branch predictor, and can also be used as a backup predictor when the main branch predictor fails. When deployed in parallel, the target jump result is obtained by combining the prediction result of the main branch predictor and the prediction jump result, to improve the prediction accuracy.

[0140] For ease of understanding, in combination with Figure 3 , as shown in Figure 3 , is a flowchart of instruction jump prediction in another embodiment. First, the instruction decoding unit decodes to obtain a to-be-processed instruction, then detects whether the to-be-processed instruction is a conditional branch instruction, if yes, outputs the program counter to the feature extractor and the perceptron selection module, the feature extractor determines the structural behavior information corresponding to the conditional branch instruction based on the program counter, then extracts the structural behavior information to obtain the structural feature, the perceptron selection module determines the perceptron corresponding to the conditional branch instruction based on the program counter, inputs the structural feature into the determined perceptron to obtain the jump prediction result, and obtains the actual jump result, if the two are inconsistent, updates the perceptron.

[0141] It should be understood that although each step in the flowchart involved in the above embodiments is shown in sequence according to the arrow, the steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0142] Based on the same inventive concept, the embodiments of the present application also provide an instruction jump prediction device for implementing the above-mentioned instruction jump prediction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more instruction jump prediction device embodiments provided below can refer to the limitations of the instruction jump prediction method in the above text, which will not be repeated here.

[0143] In an exemplary embodiment, as shown in Figure 4 An instruction jump prediction device is provided, comprising: an instruction acquisition module 401, an identification module 402, a feature extraction module 403, a sensor acquisition module 404, and a prediction module 405, wherein:

[0144] The instruction acquisition module 401 is configured to acquire a to-be-processed instruction obtained by decoding;

[0145] The identification module 402 is configured to, in a case where the to-be-processed instruction is identified as a conditional jump instruction, acquire structural behavior information corresponding to the conditional jump instruction;

[0146] The feature extraction module 403 is configured to perform feature extraction based on the structural behavior information to obtain a structural feature;

[0147] The sensor acquisition module 404 is configured to acquire a sensor corresponding to the conditional jump instruction;

[0148] The prediction module 405 is configured to input the structural feature into the sensor to perform jump prediction to obtain a predicted jump result.

[0149] In some optional embodiments, the structural behavior information includes at least one of a loop counter record of a number of loops, a proportion of zero values in a current tensor corresponding to the instruction, a proportion of target instructions in a historical target number instruction, and a storage location of the instruction; and the structural feature includes at least one of a critical jump point feature obtained based on the number of loops, a sparsity feature obtained based on a comparison between the proportion of zero values in the current tensor corresponding to the instruction and a jump threshold, an instruction type feature obtained based on a comparison between the proportion of target instructions in the historical target number instruction and a distribution threshold, and a hash feature obtained by hashing the storage location of the instruction.

[0150] In some optional embodiments, the structural feature includes a multi-level encoded feature for representing a jump probability; and the prediction module 405 is specifically configured to input the structural feature into the perceptron to obtain the jump probability, and take the jump probability as the predicted jump result.

[0151] In some optional embodiments, the prediction module 405 is specifically configured to obtain a weight vector and a bias vector of the perceptron; calculate a dot product result based on the structural feature, the weight vector, and the bias vector; and obtain the predicted jump result based on the dot product result.

[0152] In some optional embodiments, the apparatus further includes a training module configured to, in a case where the predicted jump result is different from the actual jump result, correct the weight vector based on the dot product result and the structural feature, so that the dot product result of the structural feature is close to a dot product corresponding to the actual jump result, and correct the bias vector based on the dot product result to adjust a classification boundary.

[0153] In some optional embodiments, the apparatus further includes a correction module configured to correct the perceptron in a case where the perceptron satisfies a correction condition; and the correction condition includes at least one of the following conditions: a number of times that the predicted jump result of the perceptron is different from the actual jump result is greater than a number threshold; or a conditional jump instruction corresponding to the perceptron is not identified within a preset time length; or the perceptron is an invalid perceptron; or there is an overflow in updating the weight vector and the bias vector of the perceptron.

[0154] In some optional embodiments, the perceptron obtaining module 404 is specifically configured to calculate a hash value corresponding to a storage address of the conditional jump instruction; and query the perceptron corresponding to the conditional jump instruction based on the hash value.

[0155] In some optional embodiments, the perceptron obtaining module 404 is specifically configured to, in a case where the processor is a multi-core processor, calculate a hash value corresponding to a storage address of the conditional jump instruction; perform an exclusive OR operation on the hash value and an identifier of a core corresponding to the conditional jump instruction to obtain a perceptron index; and query the perceptron corresponding to the conditional jump instruction based on the perceptron index.

[0156] In some optional embodiments, the apparatus further comprises a management module configured to obtain the number of valid sensors corresponding to each core; and replace the sensors of the core with the number of valid sensors greater than the valid threshold with the sensors of the core with the number of valid sensors less than the valid threshold.

[0157] In some optional embodiments, the apparatus further comprises a result determination module configured to, in the case that the main branch predictor fails, continue to perform the step of identifying the conditional branch instruction; or obtain the prediction result of the main branch predictor obtained in parallel with the predicted branch result, and obtain the target branch result based on the prediction result of the main branch predictor and the predicted branch result.

[0158] The modules in the instruction branch prediction apparatus can be implemented in whole or in part by software, hardware, or a combination thereof. The modules can be embedded in or independent of the processor in the computer device in hardware form, or in a multi-core processing, a separate auxiliary core executes the program corresponding to the instruction branch prediction method of the target core.

[0159] In an exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 5 The computer device comprises a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store structure behavior information and sensors, etc. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement an instruction branch prediction method.

[0160] Those skilled in the art can understand that Figure 5 The structure shown in the above

[0161] In an embodiment, a computer device is also provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0162] In an embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps in the above method embodiments when executed by a processor.

[0163] In an embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the steps in the above method embodiments when executed by a processor.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0165] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0166] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0167] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for predicting instruction jumps, characterized in that, This method is applied to predict the structural transitions of various neural network layers in an AI program. The structure of each neural network layer in the AI ​​program includes structural transition points, which include fixed iterations performed cyclically in convolutional layers, and activation functions that include a large number of CMP operations. In quantized or sparse models, some layers have a high ratio of zero values, and the program execution addresses exhibit staged clustering characteristics. The method includes: Obtain the decoded instructions to be processed; If the instruction to be processed is identified as a conditional jump instruction, obtain the structural behavior information corresponding to the conditional jump instruction; Based on the structural behavior information, feature extraction is performed to obtain structural features; Obtaining the perceptron corresponding to the conditional jump instruction includes: calculating the hash value corresponding to the storage address of the conditional jump instruction; querying the single-layer perceptron corresponding to the conditional jump instruction based on the hash value, and the perceptron can perform cross-layer generalization through the hash value, that is, if the structural behaviors corresponding to different addresses are similar, the perceptron will obtain a consistent response through weight training; The structural features are input into the perceptron to perform jump prediction and obtain the predicted jump result, including: obtaining the weight vector w and bias vector b of the perceptron; calculating the dot product result y based on the structural features x, the weight vector w, and the bias vector b; and obtaining the predicted jump result based on the dot product result y. If the predicted jump result is different from the actual jump result, the weight vector is corrected to w+y·x based on the dot product result and the structural features, so that the dot product result of the structural features is closer to the dot product corresponding to the actual jump result. The bias vector is corrected to b+y based on the dot product result to adjust the classification boundary.

2. The method according to claim 1, characterized in that, The structural behavior information includes at least one of the following: the number of loops recorded by the loop counter, the proportion of zero values ​​in the current tensor corresponding to the instruction, the proportion of target instructions in the historical target number instructions, and the instruction storage location; the structural features include at least one of the following: critical jump point features obtained based on the number of loops, sparsity features obtained based on the proportion of zero values ​​in the current tensor corresponding to the instruction and the jump threshold, instruction type features obtained based on the proportion of target instructions in the historical target number instructions and the distribution threshold, and hash features obtained by hashing the instruction storage location, wherein the hash feature is the hash value.

3. The method according to claim 1 or 2, characterized in that, The structural features include multi-level encoded features used to characterize jump probabilities; The step of inputting the structural features into the perceptron to perform jump prediction and obtain the predicted jump result includes: The structural features are input into the perceptron to obtain the jump probability; The jump probability is used as the predicted jump result.

4. The method according to claim 1, characterized in that, The method further includes: If the sensor meets the correction condition, the sensor is corrected. The correction condition includes at least one of the following conditions: The number of times the sensor's predicted jump result differs from the actual jump result exceeds a threshold. Alternatively, if no conditional jump command corresponding to the sensor is identified within a preset time period; Alternatively, the sensor may be an invalid sensor; Alternatively, overflow may occur when updating the weight vector and bias vector of the perceptron.

5. The method according to claim 1 or 2, characterized in that, The step of acquiring the perceptron corresponding to the conditional jump instruction includes: When the processor is a multi-core processor, calculate the hash value corresponding to the storage address of the conditional jump instruction; The perceptron index is obtained by performing an XOR operation between the hash value and the identifier of the core corresponding to the conditional jump instruction; Based on the perceptron index, query the perceptron corresponding to the conditional jump instruction.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the number of valid perceptrons corresponding to each core; A core with a number of valid sensors less than the valid threshold is used to replace a core with a number of valid sensors greater than the valid threshold.

7. The method according to claim 1 or 2, characterized in that, The method further includes: In the event of a main branch predictor failure, continue executing the steps of the conditional jump instruction; or Obtain the prediction result of the main branch predictor obtained in parallel with the predicted jump result, and obtain the target jump result based on the prediction result of the main branch predictor and the predicted jump result.

8. An instruction branch prediction device, characterized in that, This method is applied to predict the structural transitions of various neural network layers in an AI program. The structure of each neural network layer in the AI ​​program includes structural transition points, which include fixed iterations performed cyclically in convolutional layers, and activation functions that include a large number of CMP operations. In quantized or sparse models, some layers have a high ratio of zero values, and the program execution addresses exhibit staged clustering characteristics. The device includes: The instruction acquisition module is used to acquire the decoded instructions to be processed. The identification module is used to obtain the structural behavior information corresponding to the conditional jump instruction when the instruction to be processed is identified as a conditional jump instruction. The feature extraction module is used to extract features based on the structural behavior information to obtain structural features; The perceptron acquisition module is used to acquire the perceptron corresponding to the conditional jump instruction, including: calculating the hash value corresponding to the storage address of the conditional jump instruction; querying the single-layer perceptron corresponding to the conditional jump instruction based on the hash value, and the perceptron can perform cross-layer generalization through the hash value, that is, if the structural behaviors corresponding to different addresses are similar, the perceptron will obtain a consistent response through weight training; The prediction module is used to input the structural features into the perceptron and perform jump prediction to obtain the predicted jump result, including: obtaining the weight vector w and bias vector b of the perceptron; calculating the dot product result y based on the structural features x, the weight vector w, and the bias vector b; and obtaining the predicted jump result based on the dot product result y. The training module is used to correct the weight vector to w+y·x based on the dot product result and the structural features when the predicted jump result is different from the actual jump result, so that the dot product result of the structural features is closer to the dot product corresponding to the actual jump result, and to correct the bias vector to b+y based on the dot product result to adjust the classification boundary.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images