Code reasoning acceleration method and related equipment

Abstract

The invention provides a code reasoning acceleration method and related equipment. The method comprises the following steps: performing exception evaluation on a code corresponding to an obtained first lexical element sequence; in response to the exception of the code, classifying and repairing the second lexical element sequence corresponding to the exceptional code to obtain a third lexical element sequence; combining the third lexical element sequence with the first lexical element sequence except the second lexical element sequence to obtain a fourth lexical element sequence; and calculating the skipping probability of each lexical element in the fourth lexical element sequence, and performing layer skipping reasoning on the fourth lexical element sequence according to the skipping probability. According to the embodiment of the invention, the code reasoning efficiency and quality are remarkably improved through anomaly detection and repair, memory optimization and skip reasoning. Static memory allocation and state multiplexing reduce resource consumption, skip reasoning dynamically optimizes a calculation path, a loss monitoring and rollback mechanism is introduced, performance and precision are considered, and the method is suitable for various code generation scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of code reasoning acceleration, in particular to a code reasoning acceleration method and related equipment. BACKGROUND

[0002] With the deepening of the digital transformation of the power industry, enterprise information systems are increasingly complex, covering power grid dispatching, operation and maintenance, marketing, materials, safety supervision and other business fields, involving the development and operation of a large number of customized software systems. In order to improve system development efficiency and reduce labor costs, enterprises and their affiliated units are gradually introducing large language models (i.e., artificial intelligence models with strong natural language understanding and generation capabilities) for functions such as code completion, intelligent question answering, and automated script generation to assist programmers in efficiently writing, debugging, and maintaining code.

[0003] However, these large language model-based tools generally have large reasoning computation, long response time, and high resource consumption, which limits their widespread deployment and real-time invocation in enterprise internal development platforms, especially in enterprise private deployment environments, low-code platforms, and edge terminals, where model reasoning efficiency is particularly problematic. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a code reasoning acceleration method and related equipment.

[0005] To achieve the above purpose, the present application provides a code reasoning acceleration method, comprising: abnormality evaluation on the code corresponding to the obtained first token sequence; in response to the existence of the abnormality in the code, classifying and repairing the second token sequence corresponding to the code with the abnormality to obtain a third token sequence; merging the third token sequence and the first token sequence except the second token sequence to obtain a fourth token sequence; calculating the skip probability of each token in the fourth token sequence, and performing layer skipping reasoning on the fourth token sequence according to the skip probability.

[0006] In one possible implementation, before the abnormality evaluation on the code corresponding to the obtained first token sequence, the method further comprises: obtaining the maximum output dimension, batch size, and context length upper limit of each layer in the code reasoning process; allocating a corresponding memory buffer for each layer according to the maximum output dimension, batch size, and context length upper limit; creating an intermediate state pool; the intermediate state pool is used to store the intermediate representation of each token at the corresponding layer.

[0007] In a possible implementation, the exception evaluation on the obtained first token sequence corresponding code comprises: The exception evaluation value is calculated based on the structural complexity, static rule matching degree, local attention distribution entropy and embedding sparsity of the obtained code; The exception evaluation on the code is performed by using the exception evaluation value.

[0008] In a possible implementation, the second token sequence corresponding to the code with the exception is classified and repaired to obtain a third token sequence in response to the code having the exception, comprising: In response to the exception evaluation value being less than a first threshold value, it is determined that the code has the exception; The second token sequence corresponding to the code with the exception is classified to obtain an exception type label; Based on the exception type label, the second token sequence is repaired by combining the second token sequence and the corresponding context information to obtain the third token sequence.

[0009] In a possible implementation, the third token sequence is merged with the first token sequence except the second token sequence to obtain a fourth token sequence, comprising: The third token sequence is merged with the first token sequence except the second token sequence, and all necessary rotation position encodings are completed to obtain the fourth token sequence.

[0010] In a possible implementation, the skip probability of each token in the fourth token sequence is calculated, and the fourth token sequence is inferred by layer skipping according to the skip probability, comprising: The skip probability is calculated according to the exception evaluation value of each token in the fourth token sequence, the intermediate state of the current layer and the feature data output by the previous layer; In response to the skip probability of the token in the fourth token sequence being greater than a second threshold value, the intermediate state of the previous layer is reused, and the inference of the current layer is skipped; The loss increment after the layer skipping is monitored, and in response to the loss increment being greater than a third threshold value, a rollback operation is performed to re-perform the inference of the intermediate layer that is skipped.

[0011] Based on the same inventive concept, the embodiments of the present application further provide a code inference acceleration device, comprising: Based on the same inventive concept, the embodiments of the present application further provide an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the code inference acceleration method according to any one of the above embodiments when executing the program.

[0012] Based on the same inventive concept, the embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the code inference acceleration method described above.

[0013] Based on the same inventive concept, the embodiments of the present application also provide a computer program product comprising computer program instructions for causing the computer program product to execute the code inference acceleration method described above.

[0014] As can be seen from the above, the code inference acceleration method and related device provided by the present application, by performing abnormal evaluation on the obtained first token sequence corresponding code; in response to the existence of abnormal code, the second token sequence corresponding to the code with abnormality is classified and repaired to obtain a third token sequence; the third token sequence and the first token sequence except the second token sequence are merged to obtain a fourth token sequence; the skip probability of each token in the fourth token sequence is calculated, and the fourth token sequence is inferred according to the skip probability. The embodiments of the present application improve the calculation efficiency of the code inference process and the quality of the generated results through the overall process design of abnormal evaluation, classification and repair, token merging and skip layer inference of the code. Before inference, by allocating a fixed memory buffer for each layer and creating an intermediate state pool, the memory utilization is optimized, the problem of runtime memory fragmentation is effectively avoided, and state reuse is supported to reduce redundant calculation. The abnormal detection adopts a method of fusing multiple features, which can accurately identify abnormal code by analyzing structural complexity, static rule matching degree, local attention distribution entropy and embedding sparsity, and improve the accuracy of detection. In view of the abnormality, combined with the evaluation value threshold judgment and the abnormal type label, the targeted repair strategy is executed, such as completing the missing structure or adjusting the naming rule, to ensure the quality and stability of the generated code. The repaired token sequence and the normal token sequence are merged and position encoded after completion, which can be seamlessly connected to the Transformer inference engine based on the attention mechanism, avoiding semantic confusion. In addition, by calculating the skip probability of each token, it is dynamically determined whether to skip the inference of the current layer, which significantly reduces unnecessary computational overhead, and a loss monitoring and rollback mechanism is designed after skipping the layer to ensure that the skip layer does not affect the accuracy and consistency of the inference result. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0016] Figure 1 Flowchart of the code reasoning acceleration method of the embodiments of the application; Figure 2 Flowchart of the code reasoning acceleration method of the embodiments of the application; Figure 3 Schematic diagram of the code reasoning acceleration device of the embodiments of the application; Figure 4 Schematic diagram of the electronic device structure of the embodiments of the application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the embodiments and the drawings.

[0018] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the application should be understood as the usual meaning understood by those skilled in the art in the field to which the application belongs. The terms "first", "second" and the like used in the embodiments of the application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and the like do not mean physical or mechanical connection, but can include electrical connection, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationship, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] It can be understood that before using the technical solutions of various embodiments in the present disclosure, the user will be informed of the type, use range, use scenario and the like of the personal information involved in a proper manner, and the authorization of the user will be obtained.

[0020] For example, in response to receiving an active request of a user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide personal information to the software or hardware, such as an electronic device, an application program, a server, or a storage medium, performing the operation of the technical solution of the present disclosure according to the prompt information.

[0021] As an optional but non-limiting implementation, in response to receiving an active request of a user, the manner of sending prompt information to the user may, for example, be a pop-up window manner, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry selection controls for the user to select “agree” or “disagree” to provide personal information to the electronic device.

[0022] It can be understood that the above notification and obtaining of user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0023] As described in the background section, with the deepening of the digital transformation of the power industry, enterprise information systems have become increasingly complex, covering power grid dispatching, operation inspection, marketing, materials, safety monitoring, and other fields, involving a large amount of development and operation of customized software. To improve development efficiency and reduce labor costs, enterprises gradually introduce large language models (artificial intelligence models with strong natural language understanding and generation capabilities) for code completion, intelligent question answering, automatic script generation, and other functions to assist programmers in efficiently writing, debugging, and maintaining code.

[0024] However, these tools generally have problems such as large reasoning computation, long response time, and high resource consumption, which limit their widespread application in enterprise private deployment environments, low-code platforms, and edge terminals, especially in terms of model inference efficiency.

[0025] In view of the above, the embodiment of the present application proposes a code reasoning acceleration method, which performs abnormality evaluation on the code corresponding to the obtained first token sequence; in response to the existence of abnormality in the code, classifies and repairs the second token sequence corresponding to the code with abnormality to obtain a third token sequence; merges the third token sequence with the first token sequence except the second token sequence to obtain a fourth token sequence; calculates the skip probability of each token in the fourth token sequence, and performs skip layer reasoning on the fourth token sequence according to the skip probability. Through the design of abnormality detection, repair, token merging and skip layer reasoning, the embodiment of the present application significantly improves the code reasoning efficiency and generation quality. Fixed memory buffer is allocated before reasoning and intermediate state pool is created, which optimizes memory utilization and reduces redundant calculation. Abnormality detection fuses multi-feature analysis to accurately identify problems, and performs targeted repair combined with abnormal type label to ensure code quality. The repaired token is merged and position coding is completed, which seamlessly connects with normal tokens and avoids semantic disorder. Skip layer reasoning dynamically reduces calculation overhead, and loss monitoring and rollback mechanism are used to ensure accuracy and consistency, which is suitable for various code generation scenarios.

[0026] Hereinafter, the technical solutions of the embodiments of the present application will be described in detail through specific embodiments.

[0027] Reference Figure 1 The code reasoning acceleration method of the embodiment of the present application includes the following steps: Step S101, performing abnormality evaluation on the code corresponding to the obtained first token sequence; Step S102, in response to the existence of abnormality in the code, classifying and repairing the second token sequence corresponding to the code with abnormality to obtain a third token sequence; Step S103, merging the third token sequence with the first token sequence except the second token sequence to obtain a fourth token sequence; Step S104, calculating the skip probability of each token in the fourth token sequence, and performing skip layer reasoning on the fourth token sequence according to the skip probability.

[0028] Reference Figure 2 The code reasoning acceleration method of the embodiment of the present application is shown in detail in the flowchart.

[0029] First, the input code is tokenized into a sequence, and each token is evaluated for quality by the "anomaly identifier" module to determine if it has potential anomalies. The anomaly identifier calculates a comprehensive score for each token based on static rules (such as naming conventions, magic number detection) and dynamic features (such as attention distribution entropy, embedding sparsity). If the token is determined to be normal, it directly enters the "reorganization module" and does not require further processing; if the token is determined to be abnormal, it enters the "anomaly classifier" module.

[0030] In the anomaly classifier, the system performs context expansion on the abnormal token, extracts relevant semantic fragments, and classifies the anomaly through a pre-trained model to identify its specific type and output classification labels and processing suggestions. Subsequently, the abnormal token enters the "processing layer" and performs targeted repair operations according to the anomaly type, such as deleting duplicate code, rewriting nested structures, replacing magic numbers, and completing syntax structures. The repaired token is then combined with other normal tokens and enters the "reorganization module."

[0031] In the reorganization module, the system integrates the repaired abnormal code fragments and normal code fragments into a unified code sequence in the original order, and completes the Rotary Position Embedding (RoPE) position encoding to generate a complete sequence that meets the input format of the Transformer, preparing for the inference stage.

[0032] Next, the integrated code sequence enters the hierarchical inference calculation part. The calculation of each layer is dynamically decided by the "layer skip controller" module. Based on the features of the input token (such as L2 norm, anomaly score, attention distribution entropy, etc.), the layer skip controller calculates the skip probability through a lightweight model or rule engine. If a token is skipped at the current layer, the intermediate state of the previous layer is directly reused to avoid repeated calculation; if a token needs to be calculated, the calculation unit is called to normally execute the calculation of the current layer, and the calculation result is stored in the "state storage" module.

[0033] In the state storage module, the intermediate state of each layer is stored according to the token and layer index, supporting the state reuse of subsequent layers. If the layer skipping mechanism causes a significant increase in the loss of downstream layers, the layer skip controller can trigger "dynamic rollback" to recompute the intermediate state of the skipped layer and update the state storage to ensure output quality.

[0034] The entire process progresses layer by layer after the completion of the processing of each layer until the termination condition is met (such as reaching the maximum number of layers or the output stabilizing). Finally, the system decodes the final output code based on the valid state of the last layer. Through this process, the system realizes efficient code anomaly detection and repair, dynamic inference optimization, and final code generation tasks.

[0035] The following describes each step in detail.

[0036] For step S101, before step S101, in some embodiments, before performing the exception evaluation on the obtained code corresponding to the first token sequence, the method further includes: obtaining a maximum output dimension, a batch size, and a context length upper limit of each layer in a code inference process; allocating a corresponding memory buffer for each layer according to the maximum output dimension, the batch size, and the context length upper limit; creating an intermediate state pool; and the intermediate state pool is used to store an intermediate representation of each token at a corresponding layer.

[0037] In this embodiment, the above steps can be completed by a static memory manager, which mainly manages efficient memory allocation and access during inference, avoids runtime redundant memory operations, optimizes memory utilization, and improves inference efficiency. Through pre-allocating memory regions, aligning blocks, priority management, and state reuse, the module achieves efficient memory operation and data storage management.

[0038] The main functions of the static memory manager include: Memory allocation for all layers before inference: Before the inference starts, a fixed-size memory buffer is allocated for each layer of the model, avoiding the overhead of dynamic memory allocation and release at runtime. A memory alignment strategy is adopted to optimize the access efficiency of hardware such as Graphics Processing Unit (GPU) and Tensor Processing Unit (TPU).

[0039] Setting the storage priority of each token: Based on the importance of tokens (such as exception tokens and key context tokens), the storage priority of tokens is dynamically adjusted. Important tokens are preferentially allocated to high-speed storage channels (such as cache or memory regions closer to the computing unit).

[0040] Creating an intermediate state pool: Memory slots are allocated for the intermediate states of each layer calculation, supporting state reuse across layers. Layers that skip calculation can directly reference the intermediate state, avoiding repeated calculation.

[0041] First, all required memory allocations are completed before the inference starts, ensuring that no dynamic memory allocation / release is required during the entire inference process. Specifically, each layer of the model is traversed to count the following key parameters: the maximum output dimension D, which is the output vector dimension of each layer. The batch size B, which is the number of data samples sent into the model for inference each time. The context length upper limit T, which is the maximum number of tokens supported each time. Then, the memory size required by each layer is calculated .

[0042] Further, a memory buffer of a fixed size is allocated for each layer, which is statically allocated and used to store the intermediate representation of the layer. The memory buffers of all layers are allocated during the initialization phase, avoiding the additional overhead of runtime dynamic allocation. Among them, the memory allocation adopts a memory alignment block strategy to support efficient access of GPU / TPU memory.

[0043] Further, a state storage slot StatePool(i, l) is allocated for each token and layer, where i represents the position of the token in the sequence. l represents the layer index of the model. The state pool is used to store the intermediate representation of each token at different layers, supporting fast retrieval and state reuse.

[0044] In some embodiments, a storage priority can also be calculated for each token, and different memory channels are allocated according to its importance. For example, abnormal tokens or other important tokens are allocated high-speed channels to ensure their calculation and access efficiency. Normal tokens without abnormalities are allocated normal channels. It should be noted that before the initialization is completed, the memory buffer pointers of all layers must be registered to ensure that the memory mapping of each layer is unique during the inference process.

[0045] Further, for step S101, the code corresponding to the obtained first token sequence is evaluated for abnormality. In some embodiments, the code corresponding to the obtained first token sequence is evaluated for abnormality, including: calculating an abnormality evaluation value based on the structural complexity, static rule matching degree, local attention distribution entropy and embedding sparsity of the obtained code; and evaluating the code for abnormality using the abnormality evaluation value.

[0046] In the present embodiment, the above steps can be implemented using a code abnormality identifier. Specifically, its main function is to evaluate the quality of the input code segment (represented in the form of a token sequence), identify potential abnormal code segments, and provide basic support for subsequent abnormal classification and processing. Through the combination of static rules and dynamic features, the code abnormality identifier can efficiently and accurately judge the abnormality of the code and output the classification basis.

[0047] It analyzes the tokenized code segment, combines static rules and dynamic features, calculates the comprehensive score, and judges whether the code is abnormal. Abnormality usually refers to structural problems, syntax problems or potential logic problems in the code. Further, according to a preset threshold, normal code and abnormal code are marked. Then the abnormal code is sent to the subsequent step for classification and repair, while the normal code directly enters the code reorganization step.

[0048] In this embodiment, multiple features (such as code complexity, static rule matching, attention distribution entropy, and embedding sparsity) are used for comprehensive scoring to achieve fine-grained code evaluation.

[0049] For each lexical unit, extract the following four types of features: One is code complexity ( This is used to quantify the structural complexity of the semantic unit (statement / block / function) where a word resides. For example: Cyclomatic complexity: the complexity of control flow in the code (such as nested if-else statements, nested loops, etc.). Nesting level: the depth of nested structures in the code (such as the number of levels of nested if statements within a for loop).

[0050] Specifically, an Abstract Syntax Tree (AST) and a Control Flow Graph (CFG) are first extracted from the input code. The AST is then used to calculate the maximum nesting depth of if / for / while / try statements, etc. The cyclomatic complexity is calculated from the control flow graph. ,Right now

[0051] in, Indicate cyclomatic complexity, This indicates the number of edges in the control flow graph. This indicates the number of nodes in the control flow graph. This represents the number of connected components (usually 1).

[0052] Furthermore, regarding nesting depth Cyclic complexity Normalization, i.e.

[0053]

[0054] in, Indicates the normalized nesting depth. Indicates the first Nesting depth of semantic units This represents the normalized cyclomatic complexity. Indicates the first Cyclomatic complexity of semantic units.

[0055] Finally, calculate the word-level complexity, i.e.

[0056] in, Indicates the first Lexical complexity of 1 lexical unit This indicates the adjustment of hyperparameters.

[0057] In practical applications, static analysis tools can be used to parse the code structure and count the number of control flow paths or the nesting depth.

[0058] The second is the static rule matching degree ( Static rule matching indicates the hit rate of "code smells" such as non-standard naming, magic numbers, verbose lines, and questionable application programming interfaces (APIs). It is used to detect whether code violates common programming conventions or potential error patterns, such as: Non-standard naming: Variable names and function names do not conform to naming conventions (e.g., the variable name is a single character 'x').

[0059] Magic numbers: numerical values ​​hard-coded in the code (such as 42 in int x = 42).

[0060] Missing parentheses: For example, curly braces are missing in `if (x>0) printf("hello");`.

[0061] It should be noted that the above examples are for reference only, and the exceptions in this application are not limited to the examples above.

[0062] Specifically, the first step is to construct a rule base. This includes non-standard naming (uncommon style / containing special characters), magic numbers (not 0 / 1 / 2 / −1 and not constant quantization), etc., and then marking whether each lexical element matches in the rule base, i.e.

[0063] Calculate the hit rate, i.e.

[0064] in, Indicates the first The static rule matching degree of each word element. This indicates the total number of rules contained in the rule base. Indicates the first The weight of each rule, weight and , Indicates the first Does the word element match the first rule in the rule base? Rules.

[0065] Thirdly, the local attention distribution entropy ( Local attention distribution entropy is used to quantify information redundancy or attention anomalies in lexical units. It describes the uniformity of attention distribution in the Transformer model; low-information lexical units are usually accompanied by a decrease in attention entropy. Specifically, for each lexical unit, its distribution entropy in the attention heads is calculated.

[0066] Specifically, first, the attention score is calculated. ( The attention scores for each attentional head are:

[0067] in, Indicates the first In the attention head, the first The word pair in the context of the first word Attention score for each word element Indicates word elements The query vector, Indicates word elements The key vector, superscript This represents the transpose of a column vector. Represents the vector dimension.

[0068] Furthermore, calculate the normalized attention entropy:

[0069] in, Indicates the first The word element in the first Normalized attention entropy in each attention head Indicates the context length. Indicates the first In the attention head, the first The word pair in the context of the first word Attention score for each word.

[0070] Combine multi-head attention entropy and calculate the distribution mean and standard deviation:

[0071]

[0072] in, Indicates the first The average attention entropy of each word across all attention heads. This indicates the total number of heads of attention. Indicates the first The word element in the first Normalized attention entropy in each attention head Indicates the first The standard deviation of the entropy values of the word pieces in all attention heads.

[0073] The final attention distribution entropy of the word piece is:

[0074] wherein, represents the final attention distribution entropy of the word piece, represents the average attention entropy of the word piece in all attention heads, represents the hyperparameter, represents the average attention entropy of the word piece in all attention heads, represents the hyperparameter, represents the average attention entropy of the word piece in all attention heads, represents the standard deviation of the entropy values of the word piece in all attention heads.

[0075] The greater the value, the more focused the attention, and the more likely to be a low information or abnormal redundant word piece.

[0076] The fourth is the embedding sparsity degree (S). It is used to describe the sparsity of the embedding vector of the word piece. The sparse embedding usually may represent an anomaly. The embedding sparsity degree judges whether the word piece is abnormal according to the sparsity (number of non-zero dimensions) of the word piece embedding.

[0077] Specifically, the embodiment uses Hoyer sparsity to calculate the embedding sparsity, that is,

[0078] wherein, represents the embedding sparsity of the word piece, represents the embedding vector of the word piece, represents the embedding dimension, represents the norm of the vector, represents the norm of the vector. Further, the abnormality evaluation value is calculated based on the above four elements:

[0079]

[0080] wherein, represents the abnormality evaluation value, represents the word piece level complexity of the word piece, represents the static rule matching degree of the word piece, represents the final attention distribution entropy of the word piece, represents the final attention distribution entropy of the word piece, represents the final attention distribution entropy of the word piece, represents the final attention distribution entropy of the word piece, represents the final attention distribution entropy of the word piece, ​​​The embedding sparsity of each word element 、 、 、 Both indicate the adjustment of hyperparameters, and .

[0081] After calculating the anomaly evaluation value, the code is evaluated for anomalies using the anomaly evaluation value.

[0082] Furthermore, in step S102, in response to the existence of an anomaly in the code, the second word sequence corresponding to the anomaly is classified and repaired to obtain the third word sequence.

[0083] In some embodiments, the step of classifying and repairing the second lexical sequence corresponding to the abnormal code to obtain a third lexical sequence in response to the existence of an anomaly in the code includes: determining that the code is abnormal in response to the anomaly evaluation value being less than a first threshold; classifying the second lexical sequence corresponding to the abnormal code to obtain an anomaly type label; and repairing the second lexical sequence based on the anomaly type label, combined with the second lexical sequence and corresponding context information, to obtain the third lexical sequence.

[0084] In this embodiment, for each input token, it is considered abnormal if it meets one of the following conditions: Static rule matching rate ; Standard deviation of attention distribution ; Comprehensive abnormal score ; in , , Both represent hyperparameters.

[0085] Furthermore, in this embodiment, an anomaly classifier is used to classify the second lexical sequences corresponding to the abnormal code to obtain anomaly type labels. Its main task is to further refine the analysis of the code lexical sequences already marked as abnormal, identify the specific type of anomaly, and provide classification results and processing suggestions. By combining a deep learning model and a rule engine, the anomaly classifier can efficiently classify various anomaly types (such as duplicate code, non-standard naming, excessive nesting, etc.) and provide targeted strategies for subsequent anomaly handling layers.

[0086] Specifically, the abnormal term is expanded in context and combined with the surrounding code snippets to determine the specific type of the abnormality.

[0087] The supported exception types include but are not limited to: Duplicate code: Redundant code segments that repeat other parts of the context.

[0088] Long function: Functions that are too long or complex in content, which may need to be split.

[0089] Overly nested: Code control flow is nested too many layers, affecting readability.

[0090] Naming irregularities: Variable and function names do not conform to naming conventions.

[0091] Magic numbers: Hard-coded values are not defined as constants.

[0092] Lack of structure: Syntax structures such as brackets, semicolons, etc. are missing.

[0093] Further, using a pre-trained model combined with a multi-layer perceptron classifier, the abnormal word units are subjected to deep semantic analysis, and then the abnormal type label and its corresponding confidence are output.

[0094] The exception type label can be magic numbers or naming irregularities, etc.

[0095] In some embodiments, based on the classification results, exception handling suggestions can also be generated.

[0096] Further, after obtaining the exception type label, based on the exception type label, the second word unit sequence is repaired and processed in combination with the corresponding context information.

[0097] The main functions of the exception handling layer include: According to the exception type identified by the exception classifier (such as duplicate code, long function, naming irregularities, etc.), select the appropriate exception handling strategy.

[0098] Support for processing needs of multiple exception types, such as deletion, rewriting, replacement, and block splitting.

[0099] Semantic repair and embedding rewriting of abnormal word units to ensure that the processed word units are consistent with the original logic in semantics.

[0100] Support for soft masks (non-all-zero dilution attention) and embedding replacement to avoid semantic loss.

[0101] During processing, ensure that the information retention degree of the repaired code segment is greater than the corresponding threshold to avoid introducing new semantic conflicts due to repair.

[0102] Pass the repaired word units and their context to the subsequent code restructuring step to ensure the integrity of the code sequence structure.

[0103] Specifically, in the present embodiment, the abnormality processing layer is adopted to repair the second token sequence, which is responsible for formulating and executing a targeted processing strategy according to the abnormality type label and context information output by the abnormality classifier. By means of trimming, masking, structure rewriting or other repair operations on the abnormal token, the abnormality processing layer can optimize the code structure under the premise of ensuring the integrity of the code semantics, and provide high-quality input for the subsequent code restructuring and reasoning stage.

[0104] For example, for repeated code, i.e. code fragments are repeated with other parts in the context, it can be redundant code. The repeated fragments can be deleted, the unique code parts are retained, or the repeated fragments are merged to avoid redundancy.

[0105] For long functions, i.e. functions are too long or the logic is too complex, affecting readability. The long function can be split into multiple sub-functions. The logic consistency of the original function call is maintained.

[0106] For excessive nesting, i.e. multi-layer nested control flow logic is complex, affecting code readability. Early return or condition merging can be used to simplify the nested structure.

[0107] For non-standard naming, i.e. variable names or function names are too simple or do not conform to naming rules. It can be changed to a more semantic name.

[0108] For magic numbers, i.e. hard-coded numerical values (magic numbers) are not defined as constants, affecting code maintainability. Magic numbers can be replaced by constants.

[0109] It should be noted that in the present embodiment, the repaired code must satisfy the information retention degree greater than 0.8, and the repaired code must pass the semantic consistency verification. The aforementioned 0.8 can be set by the user as needed, and is not fixedly constrained here.

[0110] Further, for step S103, the third token sequence is merged with the first token sequence excluding the second token sequence to obtain a fourth token sequence.

[0111] In some embodiments, the merging of the third token sequence with the first token sequence excluding the second token sequence to obtain a fourth token sequence comprises: merging the third token sequence with the first token sequence excluding the second token sequence, and completing all necessary rotation position encodings to obtain the fourth token sequence.

[0112] Specifically, after completing the repair of the second token sequence to obtain a third token sequence, the third token sequence is merged with the first token sequence excluding the second token sequence to obtain a fourth token sequence.

[0113] In this embodiment, the code reorganization module is used to merge the third token sequence and the first token sequence except the second token sequence. It is responsible for integrating the abnormal code fragments repaired by the exception handling layer with the normal code fragments to generate a unified code sequence with complete semantics and correct structure. This module ensures that the repaired code fragments and normal fragments can seamlessly connect, while providing high-quality input for subsequent Transformer layer reasoning.

[0114] The main functions of the code reorganization module include: Integrate normal code and repaired abnormal code into a unified token sequence to ensure semantic and structural integrity. Ensure that the repaired code fragments can accurately connect the context. Perform RoPE position encoding completion on the integrated code sequence to ensure that the position information is consistent with the model reasoning requirements. Output a reorganized code token sequence as input for the Transformer layer. And during the sequence reorganization process, ensure that the code semantics do not have discontinuity or logical errors due to repair or reorganization operations.

[0115] First, receive the token sequence of the normal code fragment and the token sequence of the repaired abnormal code fragment. Then, according to the original order of the input code, merge the normal code fragment with the repaired abnormal code fragment. During the merging process, it is necessary to check whether the connecting symbols (such as semicolons, parentheses) are missing between the code fragments. It is necessary to ensure the logical consistency of the repaired fragment and the normal fragment. For example: whether the repaired abnormal fragment correctly replaces the original fragment and whether the merged sequence preserves the original code intent.

[0116] Further, recompute all necessary rotation position encodings for the integrated code sequence to ensure that the order information of each token is correct. It should be noted that the completed position encoding must be compatible with the input format of the Transformer model. The dimension of the position encoding should be consistent with the embedding dimension (such as 768 dimensions). Then output a complete code token sequence after reorganization as input for the subsequent Transformer layer.

[0117] Further, for step S104, calculate the skip probability of each token in the fourth token sequence, and perform layer skipping reasoning on the fourth token sequence according to the skip probability.

[0118] In some embodiments, the calculating the skip probability of each word piece in the fourth word piece sequence, and performing the skip-layer reasoning on the fourth word piece sequence according to the skip probability comprises: calculating the skip probability according to the anomaly evaluation value of each word piece in the fourth word piece sequence, the intermediate state of the current layer, and the feature data output by the previous layer; in response to the skip probability of a word piece in the fourth word piece sequence being greater than a second threshold value, reusing the intermediate state of the previous layer and skipping the reasoning of the current layer; and monitoring the loss increment after the skip-layer, and in response to the loss increment being greater than a third threshold value, performing a rollback operation and re-performing the reasoning of the skipped intermediate layer.

[0119] In the present embodiment, the importance and necessity of the calculation of each layer are evaluated by the layer skip controller, and the calculation of certain layers is selectively skipped to reduce the invalid calculation overhead and improve the model reasoning efficiency, while ensuring that the quality of the final output will not be significantly affected. The layer skip controller makes a layer-by-layer decision based on a lightweight decision model, in combination with the dynamic features and context information of the word pieces.

[0120] The main functions of the layer skip controller include: dynamically evaluating each word piece of each layer to determine whether to skip the calculation of the current layer. High-confidence normal word pieces are preferentially skipped, while abnormal word pieces are generally not skipped unless the reused state is reliable. A lightweight neural network model, such as a multi-layer perceptron (MLP) or a rule engine, is used to make decisions based on the features of the input word pieces and the features output by the previous layer. The impact of skip-layer is evaluated by considering indicators such as loss, attention entropy, L2 norm, etc. If skip-layer results in a significant increase in downstream calculation loss, the skipped layer can be dynamically rolled back for re-computation. And the number of floating-point operations is reduced through the skip-layer mechanism to optimize the reasoning speed and resource utilization.

[0121] Specifically, the intermediate state of the current layer (the output of the previous layer) is obtained. The feature data (such as L2 norm, attention distribution entropy) of the output of the previous layer, and the anomaly score of the current word piece, i.e., the aforementioned anomaly evaluation value, are obtained. Then, the skip probability is calculated using the aforementioned parameters. When the skip probability of a word piece in the fourth word piece sequence is greater than a second threshold value, the calculation of the current layer is skipped. For the skipped word piece, the intermediate state of the previous layer is directly reused.

[0122] At the same time, the loss increment after skip-layer needs to be monitored. The loss increment is calculated by calculating the difference between the loss obtained by re-computing the layer and the loss obtained by skipping the layer to determine whether to trigger the rollback mechanism. If the loss increment is greater than a third threshold value, a rollback operation is performed to re-perform the reasoning of the skipped intermediate layer. It should be noted that high-confidence normal word pieces can be directly skipped, while abnormal word pieces are generally not skipped (unless the reused state is reliable) Specifically, at the beginning of each layer, the controller invokes a lightweight multilayer perceptron, based on the input word scores and the output of the previous layer. Norm and local cross-entropy are used to output the skip probability of the term. The calculation method is as follows:

[0123] in, Indicates the current layer The Middle The probability of skipping each word element Indicates the bias term. Indicates the current layer The Middle Dynamic scores for each word element. This represents the temperature parameter.

[0124]

[0125] in, , , Both indicate the adjustment of hyperparameters. Indicates the upper level Norm:

[0126] in, Indicates the first The hidden state of each word element in the previous layer. express Norm.

[0127] Represents the local cross-entropy of the previous layer:

[0128]

[0129] in, Represents the vocabulary embedding set, Indicates the first The hidden state of each word element in the previous layer. This represents an embedded word in the vocabulary embedding set. This represents the weight matrix of the previous layer.

[0130] Furthermore, based on the skip probability calculated above, it is determined whether to skip the current layer calculation of the word element.

[0131] Next, calculate the distributional cross-entropy loss for that lexical term:

[0132] in, Indicates the first The distribution cross-entropy loss of each word element, Indicates the skipped layer. This represents the set of all layers that were not skipped. The superscript indicates an embedded word in the vocabulary embedding set. This represents the transpose of a column vector.

[0133] Supports runtime rerouting; if a downstream term is skipped in the current layer, the loss is reduced. , If there is a sudden increase, the skipped layers will be rolled back and recalculated, that is, the calculation of the previously skipped layers will be executed normally after a callback.

[0134] It should be noted that if the probability is skipped If the current layer is skipped, the loss of the current word after skipping the layer is calculated. and downstream word loss , ( (for hyperparameters), for ,calculate ,if Then a rollback is triggered for high-confidence normal terms ( () can be skipped directly, but abnormal words are not skipped. , , All of these are hyperparameters.

[0135] Furthermore, this embodiment also includes a reuse control module. The aforementioned intermediate state pool and reuse control module primarily aim to efficiently manage the storage and reuse of intermediate states across layers, optimizing the resource utilization efficiency of layer computations. By storing and reusing the intermediate states output by each layer's computation, this module can significantly reduce redundant computations, especially when supporting a "layer skip" mechanism, effectively improving inference efficiency.

[0136] The main functions of the intermediate state pool and reuse control module include: allocating fixed memory slots for the computation output of each layer to support the storage of intermediate states; quickly locating the intermediate state of each term in a specific layer using terms and levels as indexes; directly referencing the intermediate state of the previous layer's computation result in the skip layer mechanism to avoid redundant calculations; promptly releasing the corresponding state slot if a term is deleted to save memory; managing the lifecycle of each intermediate state using reference counting to ensure the reasonable allocation and release of state resources; and verifying the integrity of state data before reusing a state to ensure that the reused state meets expectations.

[0137] During each token inference, the output is written to an intermediate state pool. If the next layer is skipped, the controller directly references the data in the previous layer's intermediate state pool as input. In addition, if a token is deleted, its state is also removed from the pool to save memory.

[0138] As can be seen from the above embodiments, the code inference acceleration method described in the embodiments of the present application performs abnormal evaluation on the code corresponding to the obtained first token sequence; in response to the code having an abnormality, classifies and repairs a second token sequence corresponding to the code having the abnormality to obtain a third token sequence; merges the third token sequence with the first token sequence except the second token sequence to obtain a fourth token sequence; calculates a skip probability of each token in the fourth token sequence, and performs layer skipping inference on the fourth token sequence according to the skip probability. The embodiments of the present application significantly improve the efficiency and the quality of the generated code in the code inference process. Through abnormal detection, the syntax, structure, or naming problems in the code can be identified in a timely manner, and the model processing is ensured not to be disturbed by the abnormality; the classification and repair of the abnormal code further improve the correctness and readability of the code, thereby enhancing the output quality of the model. The merging operation of the tokens ensures that the code after abnormal repair can seamlessly connect with the normal code, avoiding semantic confusion or context fault problems; the layer skipping inference reduces invalid calculation by dynamically adjusting the calculation path, improves the inference efficiency, and at the same time, on the basis of layer skipping, the strict control of the result quality is retained, ensuring that the accuracy of the final output will not be affected by skipping calculation.

[0139] Further, a memory optimization configuration step before inference is added, which can avoid the fragmentation problem caused by dynamic memory allocation at runtime during the inference process, and at the same time, improve the utilization rate of GPU or TPU. In addition, the introduction of the intermediate state pool provides convenience for state reuse, which can store and quickly retrieve the intermediate representation in multi-layer calculation, avoid redundant calculation, and further reduce the memory bandwidth pressure, thereby improving the inference speed and resource utilization efficiency of the system.

[0140] Further, the specific implementation of abnormal detection is refined, and an abnormal evaluation method based on multi-dimensional features (such as structural complexity, static rule matching degree, local attention distribution entropy, and embedding sparsity) is proposed. This multi-dimensional feature fusion detection mechanism can comprehensively evaluate the quality of the code and identify potential risks or abnormal situations. The feature calculation method combines the advantages of static rules and dynamic features, for example, static rule matching can quickly capture common problems (such as non-standard naming, magic numbers, etc.), while dynamic features (such as attention distribution entropy) can more deeply analyze the structure and semantic quality of the code. This multi-angle abnormal evaluation method improves the accuracy and robustness of detection, ensuring that the model can more efficiently process the input code.

[0141] Further, the process of exception repair is refined by threshold judgment to determine whether the code has exceptions, and combined with exception type labels and context information to perform repair operations. For example, according to the exception type, the code with non-standard naming can be renamed, the code with deep nesting can be optimized in structure, or the missing brackets can be completed. This targeted repair strategy can preserve the semantic information of the original code as much as possible, while improving the readability and maintainability of the code. The introduction of context information makes the repair process more intelligent and accurate, avoiding isolated handling of exceptions that can cause new problems, thereby ensuring the stability of the quality of the repaired code.

[0142] In addition, by merging the repaired word sequence with the normal word sequence and completing the rotation position encoding, it is ensured that the merged word sequence is completely matched with the input format of the Transformer inference engine. This merging method can effectively avoid semantic drift problems caused by incomplete or disordered position encoding. In addition, the merging operation can maintain the integrity of the code structure while considering the coherence of the repaired code semantics and the original context, thereby improving the accuracy and consistency of the inference results.

[0143] In addition, by calculating the skip probability of each word, it is dynamically determined whether to skip the inference of the current layer, further optimizing the efficiency of inference calculation. The skip layer mechanism can effectively reduce the repeated calculation of low-value words, significantly reducing the computational amount and time overhead of inference. At the same time, the claim designs a loss monitoring and rollback mechanism after skipping layers, which can roll back and recalculate the skipped layers when the loss increment caused by skipping layers exceeds the tolerance threshold, thereby ensuring the accuracy and consistency of the inference results. This dynamic adjustment mechanism balances performance optimization and result reliability, and is particularly suitable for complex scenarios that require efficient processing of diverse inputs.

[0144] It should be noted that the method of the embodiments of the present application can be executed by a single device, such as a computer or a server, etc. The method of the embodiments of the present application can also be applied to a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present application, and the multiple devices can interact with each other to complete the method.

[0145] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order described above and still achieve the desired results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0146] Based on the same inventive concept, the present application also provides a code inference acceleration device corresponding to any of the above-mentioned embodiment methods.

[0147] Reference Figure 3 The code inference acceleration device comprises: A code exception identifier 31 configured to perform exception evaluation on the acquired first token sequence corresponding code; An exception classifier and exception processing layer 32 configured to, in response to the existence of an exception in the code, classify and repair the second token sequence corresponding to the code with the exception to obtain a third token sequence; A code recombination module 33 configured to combine the third token sequence with the first token sequence excluding the second token sequence to obtain a fourth token sequence; A layer skipping controller 34 configured to calculate the skipping probability of each token in the fourth token sequence, and perform layer skipping inference on the fourth token sequence according to the skipping probability.

[0148] For the convenience of description, the above device is described in various modules according to functions. Of course, in the implementation of the present application, the functions of each module can be implemented in one or more software and / or hardware.

[0149] The device of the above-mentioned embodiment is used to implement the corresponding code inference acceleration method of any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0150] Based on the same inventive concept, the present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the code inference acceleration method of any of the above-mentioned embodiments when executing the program.

[0151] Figure 4 A more specific hardware structure of an electronic device is shown in the embodiment, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0152] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided by the embodiments of the present specification.

[0153] The memory 1020 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the relevant program codes are saved in the memory 1020 and called and executed by the processor 1010.

[0154] The input / output interface 1030 is configured to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input devices can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices can include a display, a speaker, a vibrator, an indicator light, etc.

[0155] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to implement the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0156] The bus 1050 includes a channel for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0157] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only include the components necessary to implement the solutions of the embodiments of the present specification, and does not have to include all the components shown in the figure.

[0158] The electronic device of the above-mentioned embodiments is configured to implement the corresponding code inference acceleration method of any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0159] Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the code inference acceleration method of any of the preceding embodiments.

[0160] The computer-readable medium of the present embodiments includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0161] The computer instructions stored in the storage medium of the above-mentioned embodiments are configured to cause the computer to perform the code inference acceleration method of any of the preceding embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0162] Based on the same inventive concept, the present disclosure also provides a computer program product including computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the code inference acceleration method. Corresponding to the execution subject of each step in each embodiment of the code inference acceleration method, the processor performing the corresponding step can belong to the corresponding execution subject.

[0163] The computer program product of the above-mentioned embodiments is configured to cause the computer and / or the processor to perform the code inference acceleration method of any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0164] Those of ordinary skill in the art will realize that the foregoing discussion of any of the embodiments has been presented for the purpose of illustration and description and is not intended to be exhaustive or to limit the application to the precise forms described, and that various adaptations and modifications are possible within the scope and spirit of the application. For example, although the embodiments discussed above are described in the context of a memory device, the embodiments discussed above can be used in other contexts as well. Furthermore, the above embodiments may

[0165] In addition, to simplify the description and discussion, and so as not to make the embodiments of the application difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. Furthermore, devices can be shown in block diagram form in order to avoid making the embodiments of the application difficult to understand, and this also takes into account the fact that the details regarding the implementation of these block diagram devices are highly dependent on the platform in which the embodiments of the application are to be implemented (i.e., these details should be well within the understanding of one of ordinary skill in the art). Where specific details (e.g., circuitry) are set forth in order to describe an illustrative embodiment of the application, it should be apparent to one of ordinary skill in the art that the embodiments of the application can be practiced without or with variations of these specific details. Thus, the description should not be considered to be limiting in nature.

[0166] While the application has been described in connection with specific embodiments thereof, it will be understood that many modifications, variations and alternatives will be apparent to those skilled in the art as a result of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0167] The embodiments of the application are intended to cover all such modifications, variations and alternatives as come within the scope of the appended claims. Thus, any and all such modifications, variations and alternatives are intended to be included within the scope of the application.

Claims

1. A code inference acceleration method, characterized by, The method comprises the following steps: abnormality evaluation is performed on the code corresponding to the obtained first token sequence; in response to the existence of an abnormality in the code, a second token sequence corresponding to the code with the abnormality is classified and repaired to obtain a third token sequence; the third token sequence is merged with the first token sequence excluding the second token sequence to obtain a fourth token sequence; the skip probability of each token in the fourth token sequence is calculated, and the fourth token sequence is inferred based on the skip probability.

2. The method of claim 1, wherein, Before the abnormality evaluation is performed on the code corresponding to the obtained first token sequence, the method further comprises the following steps: obtaining the maximum output dimension, batch size and context length upper limit of each layer in the code inference process; allocating a corresponding memory buffer for each layer according to the maximum output dimension, batch size and context length upper limit; creating an intermediate state pool; the intermediate state pool is used to store the intermediate representation of each token at the corresponding layer.

3. The method of claim 1, wherein, The abnormality evaluation on the code corresponding to the obtained first token sequence comprises the following steps: calculating an abnormality evaluation value based on the structural complexity, static rule matching degree, local attention distribution entropy and embedding sparsity of the obtained code; performing abnormality evaluation on the code by using the abnormality evaluation value.

4. The method of claim 3, wherein, In response to the existence of an abnormality in the code, the second token sequence corresponding to the code with the abnormality is classified and repaired to obtain the third token sequence, which comprises the following steps: in response to the abnormality evaluation value being less than a first threshold value, it is determined that the code has an abnormality; classifying the second token sequence corresponding to the code with the abnormality to obtain an abnormality type label; based on the abnormality type label, the second token sequence and the corresponding context information are combined to perform repair processing on the second token sequence to obtain the third token sequence.

5. The method of claim 1, wherein, The third token sequence is merged with the first token sequence excluding the second token sequence to obtain the fourth token sequence, which comprises the following steps: the third token sequence is merged with the first token sequence excluding the second token sequence, and all necessary rotary position encodings are completed to obtain the fourth token sequence.

6. The method of claim 3, wherein, The skip probability of each token in the fourth token sequence is calculated, and the fourth token sequence is inferred based on the skip probability, which comprises the following steps: the skip probability is calculated based on the abnormality evaluation value of each token in the fourth token sequence, the intermediate state of the current layer and the feature data output by the previous layer; in response to the skip probability of the token in the fourth token sequence being greater than a second threshold value, the intermediate state of the previous layer is reused, and the inference of the current layer is skipped; the loss increment after the layer skipping is monitored, and in response to the loss increment being greater than a third threshold value, a rollback operation is performed to re-perform the inference of the skipped intermediate layer.

7. A code speculation acceleration device, comprising: The method comprises the following steps: a code abnormality identifier is configured to perform abnormality evaluation on the code corresponding to the obtained first token sequence; an abnormality classifier and an abnormality processing layer are configured to classify and repair the second token sequence corresponding to the code with the abnormality to obtain the third token sequence in response to the existence of an abnormality in the code; a code recombination module configured to combine the third word sequence with the first word sequence other than the second word sequence to obtain a fourth word sequence; a layer skipping controller configured to calculate a skipping probability of each word in the fourth word sequence, and perform layer skipping reasoning on the fourth word sequence according to the skipping probability.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method in any one of claims 1 to 6 when executing the computer program.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to make the computer execute the method in any one of claims 1 to 6.

10. A computer program product comprising computer program instructions which, when executed on a computer, cause the computer to perform the method in any one of claims 1 to 6.

Images