Speculative method, apparatus, storage medium, electronic device and program of code statements

Abstract

The present disclosure provides a code sentence speculation method, device, storage medium, electronic equipment and program. A specific embodiment of the method comprises: in response to obtaining a current prior token sequence, performing speculation decoding on the current prior token sequence to generate a plurality of candidate token sequences for connecting the current prior token sequence; verifying whether each candidate token sequence is valid; in response to determining that any candidate token sequence is invalid, correcting the invalid candidate token sequence to a valid candidate token sequence; determining a current target token sequence matching the current prior token sequence from each valid candidate token sequence; and in response to identifying an end identifier from the current target token sequence, combining the current prior token sequence and the current target token sequence to form a code sentence. The embodiment improves the accuracy of the code sentence.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of code generation, and in particular, to a code statement speculation method and device, a storage medium, an electronic device, and a program. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, when generating code, an artificial intelligence model can be used to generate code statements of a program language.

[0003] Generally, an LLM model (Large Language Model) can be used to generate code statements, but for program languages with complex syntax rules, such as DSL (Domain Specific Languages), the LLM often has difficulty in strictly complying with the syntax rules, resulting in a high error rate of the generated code statements, and the process of checking the generated code statements significantly reduces efficiency.

[0004] On the other hand, the general inference acceleration mode of the LLM model is often to use speculative decoding to directly generate complete code statements, which are then checked by the LLM model. However, since speculative decoding is more difficult to apply to complex syntax rules, the complete code statements generated by it have a higher error rate, which not only fails to achieve acceleration, but also further reduces the efficiency of the LLM model due to the high error rate. SUMMARY

[0005] Therefore, embodiments of the present disclosure provide a code statement speculation method, device, storage medium, electronic device, and program.

[0006] In a first aspect, embodiments of the present disclosure provide a code statement speculation method, which includes: in response to obtaining a current prior token sequence, performing speculative decoding on the current prior token sequence to generate a plurality of candidate token sequences for connecting the current prior token sequence;

[0007] verifying whether each candidate token sequence is valid;

[0008] in response to determining that any candidate token sequence is invalid, modifying the invalid candidate token sequence to a valid candidate token sequence;

[0009] determining a current target token sequence matching the current prior token sequence from each valid candidate token sequence;

[0010] in response to identifying an end identifier from the current target token sequence, combining the current prior token sequence and the current target token sequence to form a code statement.

[0011] In some optional embodiments, verifying whether each candidate token sequence is valid comprises:

[0012] verifying whether each candidate token sequence has a syntax error according to a preset syntax rule;

[0013] determining the candidate token sequence having at least one syntax error as an invalid candidate token sequence, and determining the candidate token sequence having no syntax error as a valid candidate token sequence.

[0014] In some optional embodiments, each candidate token sequence comprises at least one token connected in a predetermined order.

[0015] Accordingly, correcting the invalid candidate token sequence to a valid candidate token sequence comprises:

[0016] determining an invalid token corresponding to each syntax error in each invalid candidate token sequence;

[0017] deleting each invalid token and all tokens connected after each invalid token to obtain a corrected token sequence;

[0018] performing sentence prediction on each corrected token sequence to generate a valid candidate token sequence.

[0019] In some optional embodiments, performing sentence prediction on each corrected token sequence to generate a valid candidate token sequence comprises:

[0020] performing sentence prediction on each corrected token sequence to determine a token probability distribution corresponding to each corrected token sequence, the token probability distribution representing a probability of selecting each token after each corrected token sequence;

[0021] adjusting the token probability distribution corresponding to each corrected token sequence;

[0022] determining a token connected to each corrected token sequence according to the adjusted token probability distribution to obtain a valid candidate token sequence.

[0023] In some optional embodiments, adjusting the token probability distribution corresponding to each corrected token sequence comprises:

[0024] determining a probability of selecting a corresponding invalid token from the token probability distribution corresponding to each corrected token sequence;

[0025] adjusting the probability of selecting the corresponding invalid token to obtain an adjusted token probability distribution.

[0026] In some optional embodiments, after determining a current target token sequence matching the current prior token sequence from each valid candidate token sequence, the following steps can be further performed:

[0027] In response to not recognizing the end identifier from the current target token sequence, the current target token sequence and the current prior token sequence are combined to form a next prior token sequence;

[0028] A speculative decoding is performed on the next prior token sequence to generate a plurality of candidate token sequences corresponding to the next prior token sequence;

[0029] A next target token sequence that matches the next prior token sequence is determined from the plurality of candidate token sequences corresponding to the next prior token sequence;

[0030] In response to recognizing the end identifier from the next target token sequence, the next prior token sequence and the next target token sequence are combined to form a code statement.

[0031] In some optional embodiments, the code statement is used to respond to a received operation instruction;

[0032] Correspondingly, before responding to the obtained current prior token sequence, the following steps can also be performed in advance:

[0033] In response to receiving the operation instruction, a corresponding initial prior token sequence is generated, wherein the initial prior token sequence includes at least one token in the code statement;

[0034] The initial prior token sequence is determined as the current prior token sequence.

[0035] In a second aspect, embodiments of the present disclosure provide a speculative device for a code statement, which comprises:

[0036] A speculative decoding model response module is configured to, in response to obtaining a current prior token sequence, perform a speculative decoding on the current prior token sequence to generate a plurality of candidate token sequences for connecting the current prior token sequence;

[0037] An analysis verifier response module is configured to verify whether each candidate token sequence is valid;

[0038] A correction module is configured to, in response to determining that any candidate token sequence is invalid, correct the invalid candidate token sequence to a valid candidate token sequence;

[0039] A large language model response module is configured to determine, from each valid candidate token sequence, a current target token sequence that matches the current prior token sequence;

[0040] The speculative decoding model response module is further configured to, in response to recognizing an end identifier from the current target token sequence, combine the current prior token sequence and the current target token sequence to form a code statement.

[0041] In some optional embodiments, the parsing verifier response module is further configured to:

[0042] verify whether each candidate token sequence has a syntax error according to the preset syntax rule;

[0043] determine the candidate token sequence having at least one syntax error as an invalid candidate token sequence, and determine the candidate token sequence not having a syntax error as a valid candidate token sequence.

[0044] In some optional embodiments, each candidate token sequence comprises at least one token connected in a predetermined order;

[0045] Correspondingly, the correction module is further configured to:

[0046] determine the invalid token corresponding to each syntax error from each invalid candidate token sequence;

[0047] delete each invalid token and all tokens connected after each invalid token to obtain a corrected token sequence;

[0048] perform sentence prediction on each corrected token sequence to generate a valid candidate token sequence.

[0049] In some optional embodiments, performing sentence prediction on each corrected token sequence to generate a valid candidate token sequence comprises:

[0050] performing sentence prediction on each corrected token sequence to determine a token probability distribution corresponding to each corrected token sequence, the token probability distribution representing a probability of selecting each token after each corrected token sequence;

[0051] adjusting the token probability distribution corresponding to each corrected token sequence;

[0052] determining a token connected to each corrected token sequence according to the adjusted token probability distribution to obtain a valid candidate token sequence.

[0053] In some optional embodiments, adjusting the token probability distribution corresponding to each corrected token sequence comprises:

[0054] determining a probability of selecting the corresponding invalid token from the token probability distribution corresponding to each corrected token sequence;

[0055] adjusting the probability of selecting the corresponding invalid token to obtain an adjusted token probability distribution.

[0056] In some optional embodiments, the code statement speculation device further comprises:

[0057] The iteration module is configured to perform the following steps after determining the current target token sequence matching the current prior token sequence from the respective valid candidate token sequences:

[0058] In response to not recognizing the end identifier from the current target token sequence, composing the next prior token sequence from the current target token sequence and the current prior token sequence;

[0059] Performing speculative decoding on the next prior token sequence to generate a plurality of candidate token sequences corresponding to the next prior token sequence;

[0060] Determining the next target token sequence matching the next prior token sequence from the plurality of candidate token sequences corresponding to the next prior token sequence;

[0061] In response to recognizing the end identifier from the next target token sequence, composing the code statement from the next prior token sequence and the next target token sequence.

[0062] In some optional embodiments, the speculative device of the code statement further comprises:

[0063] The initial module is configured to perform the following steps before responding to obtaining the current prior token sequence:

[0064] In response to receiving the operation instruction, generating a corresponding initial prior token sequence, wherein the initial prior token sequence comprises at least one token in the code statement;

[0065] Determining the initial prior token sequence as the current prior token sequence.

[0066] In a third aspect, embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementation manners of the first aspect.

[0067] In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by one or more processors to implement the method described in any of the implementation manners of the first aspect.

[0068] In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising computer programs / instructions, wherein the computer programs / instructions are executed by a processor to implement the method described in any of the implementation manners of the first aspect.

[0069] To solve the problem of low efficiency and high error rate of LLM model generating code statements, the embodiment of the disclosure provides a code statement speculation method, device, storage medium, electronic equipment and program. The method comprises the following steps: performing speculation decoding on the obtained current prior token sequence to generate a plurality of candidate token sequences spliced with the current prior token sequence, instead of directly generating a complete code statement, thereby avoiding the problem of reduced efficiency caused by directly generating a complete code statement. Then, each candidate token sequence is verified to quickly determine the invalid candidate token sequence. Compared with verifying the entire complete code statement, the verification rate is significantly improved. After verifying the invalid candidate token sequence, the invalid candidate token sequence is corrected, and a plurality of candidate token sequences can still be ensured, so that the current target token sequence can be selected from a sufficient and extensive backup option, i.e. selected from a large number of candidate token sequences. Compared with directly generating the current target token sequence or selecting the current target token sequence from a small number of candidate token sequences, a more accurate current target token sequence can be obtained. After splicing and combining the current prior token sequence and the current target token sequence, a correct partial code statement or a complete code statement can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0070] Other features, objects, and advantages of the disclosure will become more apparent from the following detailed description of non-limiting embodiments made with reference to the accompanying drawings. The drawings are for the purpose of illustrating specific embodiments and are not to be considered limiting of the application. In the drawings:

[0071] Figure 1 An exemplary system architecture diagram to which an embodiment of the disclosure can be applied;

[0072] Figure 2A A flowchart of an embodiment of the code statement speculation method according to the disclosure;

[0073] Figure 2B A flowchart of another embodiment of the code statement speculation method according to the disclosure;

[0074] Figure 3A A decomposition flowchart of an embodiment of the code statement speculation method step 213 according to the disclosure;

[0075] Figure 3B A decomposition flowchart of an embodiment of the code statement speculation method step 303 according to the disclosure;

[0076] Figure 4 A structural schematic diagram of an embodiment of the code statement speculation device according to the disclosure;

[0077] Figure 5Structure diagram of a computer system for an electronic device suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION

[0078] The present disclosure will be further described below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that, for the sake of description, only the parts related to the application are shown in the drawings.

[0079] It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the drawings and embodiments.

[0080] Figure 1 An exemplary system architecture 100 of embodiments of the speculative method, device, storage medium, electronic device and program to which the code statements of the present disclosure can be applied is shown.

[0081] As shown in Figure 1 The system architecture 100 can include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0082] A user can use the first terminal device 101, the second terminal device 102, the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, the third terminal device 103, such as natural language processing applications, speech recognition applications, short video social applications, audio and video conference applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0083] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be hardware or software. When the first terminal device 101, the second terminal device 102, and the third terminal device 103 are hardware, they can be various electronic devices having a sound collection device (for example, a microphone), a video collection device (for example, a camera), and a display screen, including but not limited to a smart phone, a tablet computer, an electronic book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer, a desktop computer, and the like. When the first terminal device 101, the second terminal device 102, and the third terminal device 103 are software, they can be installed in the terminal devices listed above. They can be implemented as multiple software or software modules (for example, to provide a speculation service for code sentences) or as a single software or software module. No specific limitation is made herein.

[0084] In some cases, the code sentence speculation method provided by the present disclosure can be executed by the first terminal device 101, the second terminal device 102, and the third terminal device 103, and accordingly, the code sentence speculation apparatus can be arranged in the first terminal device 101, the second terminal device 102, and the third terminal device 103. At this time, the system architecture 100 can also not include the server 105.

[0085] In some cases, the code sentence speculation method provided by the present disclosure can be executed by the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105 together, for example, the step of "performing speculation decoding on the current prior word sequence to generate a plurality of candidate word sequences for connecting the current prior word sequence in response to obtaining the current prior word sequence" can be executed by the first terminal device 101, the second terminal device 102, and the third terminal device 103, and the step of "verifying whether each candidate word sequence is valid" can be executed by the server 105. The present disclosure does not make a limitation in this regard. Accordingly, the code sentence speculation apparatus can also be arranged in the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105, respectively.

[0086] In some cases, the code sentence speculation method provided by the present disclosure can be executed by the server 105, and accordingly, the code sentence speculation apparatus can also be arranged in the server 105, at this time, the system architecture 100 can also not include the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0087] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.

[0088] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks, and servers.

[0089] It should be noted that the code statement speculation method of the present disclosure is implemented by deploying a speculation constraint decoding system in the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105, which includes a pre-trained LLM model, a pre-trained speculation decoding model, a pre-trained parsing verifier, and a pre-trained fallback controller, etc. Figure 1 The speculation constraint decoding system in the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105 is implemented by deploying a pre-trained LLM model, a pre-trained speculation decoding model, a pre-trained parsing verifier, and a pre-trained fallback controller, etc.

[0090] In the training process of the LLM model, the speculation decoding model, and the parsing verifier, the syntax rules of the code language processed by the speculation constraint decoding system are trained, for example, the syntax rules of the DSL are trained.

[0091] That is, the LLM model, the speculation decoding model, and the parsing verifier in the speculation constraint decoding system can operate and process each of the following steps with the syntax rules of the DSL.

[0092] In addition, the LLM model in the speculation constraint decoding system can receive an operation instruction, so that the speculation constraint decoding system generates a code statement for responding to the operation instruction based on the operation instruction.

[0093] The operation instruction is expressed in a manner that the LLM model can analyze and process, which can be a complete or incomplete sentence expressed in natural language, or one or more prompt words expressed in natural language, so that the LLM model can recognize and process the natural language when receiving the operation instruction; the operation instruction can also be expressed in other code languages; no specific limitation is made here to the operation instruction.

[0094] Continuing to refer to Figure 2A which shows the flow 210 of one embodiment of the code statement speculation method of the present disclosure, which includes the following steps:

[0095] At step 211, in response to obtaining the current prior token sequence, performing speculative decoding on the current prior token sequence to generate a plurality of candidate token sequences for connecting to the current prior token sequence.

[0096] In the present embodiment, a token sequence represents a segment composed of one or more tokens in a predetermined order, and a plurality of token sequences compose a complete code statement in a predetermined order.

[0097] In the present embodiment, a prior token sequence represents a token sequence with no or negligible syntax error, which is generated by the LLM model, and the current prior token sequence is a prior token sequence obtained or generated at the current time.

[0098] In some optional embodiments, when the LLM generates a prior token sequence for the first time, the first generated prior token sequence, i.e., the initial prior token sequence, can be taken as the current prior token sequence. In this case, the current prior token sequence can be one or more tokens generated by the LLM model based on the received operation instruction. Since the one or more tokens at this time are the starting one or more tokens in a large number of tokens in a complete code statement, the probability of syntax error of the one or more tokens is very low and can be ignored. Therefore, it can be directly taken as the initial prior token sequence without verifying whether it is valid or not.

[0099] In another optional embodiment, when the LLM generates a prior token sequence for the first time, the current prior token sequence can be composed of the target token sequence selected from the plurality of syntax-verified valid token sequences and the previous prior token sequence.

[0100] In the present step, based on the current prior token sequence generated by the LLM model, it can be input to the speculative decoding module and subjected to speculative decoding by the speculative decoding module, so as to simultaneously generate a plurality of possible candidate token sequences, wherein each candidate token sequence is composed of one or more tokens, for connecting to the current prior token sequence and composing a code statement or a part of a code statement together with the current prior token sequence.

[0101] It should be noted that in some cases, each word unit can contain both characters with actual semantic meaning and characters without actual semantic meaning, for example, the candidate word unit sequence is "META_TYPE IS LOCATION WITH", which contains word units: "META_TYPE", "IS", "LOCATION" and "WITH", it can be seen that "IS" is a word unit "IS" with 2 characters composed of character "I" and character "S", and in other cases, the word unit at this position can also include spaces, character "I" and character "S", forming a word unit " IS" with spaces and 2 characters.

[0102] Further, after generating the candidate word unit sequence, for the generated candidate word unit sequence, a corresponding decoding log can be created for it, in which the word unit position of each word unit in each candidate word unit sequence can be recorded, so that the speculative constraint decoding system has the function of dynamically tracking each word unit.

[0103] Among them, the word unit position in the decoding log can be specifically represented by the word unit index value of the word unit, and through the word unit index value, specific information such as the line number, column number and / or specific character corresponding to the word unit can be traced back.

[0104] That is, by recording the word unit index value, the speculative constraint decoding system can accurately locate each word unit.

[0105] In some optional embodiments, the speculative decoding module can be, for example, an EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) framework.

[0106] It should be noted that the speculative decoding based on the current prior word unit sequence is a widely researched and applied technology at present, which is not specifically limited here. For example, a tree reasoning structure can be constructed based on the hidden state of the operation instruction, and according to the logical relationship between each node in the tree reasoning structure, a plurality of candidate word unit sequences as leaf nodes can be inferred. Among them, the hidden state can be, for example, a tensor in the form of a multidimensional matrix, used to represent the semantic abstract information contained in the operation instruction that the LLM model, specifically including, for example, semantic context information, grammatical structure and / or domain knowledge of DSL, etc., is obtained by the LLM model in the process of identifying the operation instruction and generating the current prior word unit sequence.

[0107] Based on the above, in the manner of generating multiple candidate token sequences in parallel based on the current prior token sequence for speculative decoding, there can be more sufficient and extensive choices when determining the current target token sequence, thereby improving the accuracy of the current target token sequence.

[0108] In step 212, it is verified whether each candidate token sequence is valid.

[0109] Based on the multiple candidate token sequences determined in the foregoing step 211, since each candidate token sequence is a possible option when connecting token sequences after the current prior token sequence, if there is an invalid candidate token sequence, in the process of selecting the target token sequence from the multiple possible candidate token sequences, the invalid candidate token sequence may be selected as the target token sequence, which forms a code statement with the prior token sequence, thereby wasting the running resources of the electronic device.

[0110] Based on the above, before determining the target token sequence therefrom, it can be verified whether each candidate token sequence is valid, so as to avoid the invalid candidate token sequence interfering in the process of selecting the target token sequence, thereby improving the accuracy of the target token sequence, and further improving the accuracy of the code statement formed by the target token sequence and the prior token sequence.

[0111] In some optional embodiments, the process of verifying whether each candidate token sequence is valid can be performed by a parsing verifier in the speculative constraint decoding system, which can be, for example, an ANTLR grammar parser (ANother Tool for Language Recognition).

[0112] In this step, since the parsing verifier is trained with the predetermined DSL grammar rules in the pre-training process, as an optional embodiment, when verifying whether each candidate token sequence is valid, it can be determined according to whether the grammar of the candidate token sequence conforms to the preset DSL grammar rules.

[0113] Specifically, the ANTLR grammar parser can parse each candidate token sequence into a tree-like parsing structure, where each tree-like parsing structure has multiple leaf nodes, and each leaf node is a token in the corresponding candidate token sequence.

[0114] Further, an error listener can be set in the ANTLR grammar parser, which can be used to detect syntax errors in the tree-like parsing structure.

[0115] In some cases, taking the candidate word-gram sequence "META_TYPE IS LOCATION WITH" as an example, after parsing it into a tree-like parsing structure, each leaf node is the word-gram "META_TYPE", "IS", "LOCATION" and "WITH".

[0116] Based on this, the error listener in the ANTLR grammar parser can verify whether each leaf node has a grammatical error according to the DSL grammar rules. For example, there is a grammatical error in the "IS" of the leaf node, which should be lowercase "is" in the correct grammar.

[0117] In some optional embodiments, based on the verification of the ANTLR grammar parser, for any candidate word-gram sequence, if it is verified that the candidate word-gram sequence has one or more grammatical errors, the candidate word-gram sequence can be determined to be invalid, and the following step 213 can be further executed; if it is verified that the candidate word-gram sequence does not have any grammatical errors, the candidate word-gram sequence can be determined to be valid, and the following step 214 can be further directly executed.

[0118] In some other optional implementations, in the actual application of DSL, since some grammatical errors with minor effects can be ignored, these negligible minor grammatical errors can also be pre-defined. When the ANTLR parser verifies that a candidate word sequence has only one or more negligible minor grammatical errors, the candidate word sequence can also be determined to be valid.

[0119] Furthermore, when the error listener detects a grammatical error, it can record the error type of the grammatical error and accurately mark the location where the grammatical error occurs. For example, after determining the leaf node with the grammatical error, it can record the word index value of the word corresponding to the leaf node and determine the word with the grammatical error as an invalid word.

[0120] Based on the determined grammatical errors and the recorded word index values ​​of the invalid words, the invalid words and their positions can be accurately located in the following subsequent steps, thereby improving the efficiency of the correction operation in the following steps.

[0121] In step 213 , in response to determining that any candidate word-gram sequence is invalid, the invalid candidate word-gram sequence is modified into a valid candidate word-gram sequence.

[0122] Based on the verification of each candidate word sequence in the aforementioned step 212, if any invalid candidate word sequence is verified, each invalid candidate word sequence can be corrected separately to ensure that the accuracy of the target word sequence is improved when selecting the target word sequence below.

[0123] In this step, the modification of the invalid candidate token sequence can be performed by the backtracking controller and the LLM model in the speculative constrained decoding system.

[0124] In some optional embodiments, based on the decoding log created in the foregoing step 211 and the token index value of the invalid token recorded in the step 212, the backtracking controller can accurately locate the invalid token and determine the error type of the invalid token.

[0125] Accordingly, the backtracking controller can perform a backtracking operation on the invalid candidate token sequence according to the invalid token and the error type thereof in the invalid candidate token sequence, restore the invalid candidate token sequence to the most recent valid state in the history, and further perform sentence prediction on the backtracked candidate token sequence by the LLM model to generate another candidate token sequence.

[0126] Specifically, the process of modifying the invalid candidate token sequence in the step 213, i.e., the backtracking operation and the sentence prediction, can include the following Figure 3A The steps 301 to 303 shown in the figure:

[0127] Further, for the generated another candidate token sequence, the verification of the step 212 can be performed again to verify whether the generated another candidate token sequence again has a syntax error.

[0128] If the generated another candidate token sequence again has a syntax error, i.e., the generated another candidate token sequence again is an invalid candidate token sequence, the backtracking controller and the LLM model can repeatedly perform the above-mentioned modification operation on the generated another candidate token sequence and perform the verification of the step 212 again.

[0129] If the generated another candidate token sequence does not have a syntax error, i.e., the generated another candidate token sequence is a valid candidate token sequence, the subsequent steps described below can be continued.

[0130] Step 214: determining a current target token sequence matching the current prior token sequence from each valid candidate token sequence.

[0131] Based on the verification of the foregoing step 212 and the modification of the foregoing step 213, when all candidate token sequences are valid token sequences, a current target token sequence can be selected from each valid candidate token sequence.

[0132] Specifically, as mentioned above, since the target token sequence is the token sequence connected after the corresponding prior token sequence, in this step, each valid candidate token sequence can be input into the LLM model, and the LLM model can determine the matching degree between each valid candidate token sequence and the current prior token sequence based on the current prior token sequence, and select the candidate token sequence with the highest matching degree as the current target token sequence matched with the current prior token sequence.

[0133] In step 215, in response to identifying the end identifier from the current target token sequence, the current prior token sequence and the current target token sequence are combined to form a code statement.

[0134] Based on the current target token sequence determined in the foregoing step 214, the current target token sequence can be spliced after the current prior token sequence, and a complete code statement or a partial code statement can be obtained according to whether the current target token sequence contains the end identifier.

[0135] In some optional embodiments, a special token can be pre-set, which has an end identifier and is used to indicate the completion of the code statement, that is, in the case where the current target token sequence contains the special token, it can be considered that the current target token sequence is the last token sequence in the complete code statement, so that after splicing the current target token sequence after the current prior token sequence, a complete code statement can be obtained.

[0136] The code statement speculation method provided by the above embodiments of the present disclosure generates a plurality of candidate token sequences spliced with the current prior token sequence by performing speculation decoding on the obtained current prior token sequence, rather than directly generating a complete code statement, thereby avoiding the problem of reduced efficiency caused by directly generating a complete code statement. Then, by verifying each candidate token sequence, the invalid candidate token sequence can be quickly determined, and the verification rate is significantly improved compared to verifying the entire complete code statement. After verifying the invalid candidate token sequence, it can still be guaranteed to have a plurality of candidate token sequences, so that when selecting the current target token sequence, it can be selected from a sufficiently extensive backup option, that is, from a large number of candidate token sequences. Compared with directly generating the current target token sequence or selecting the current target token sequence from a small number of candidate token sequences, a more accurate current target token sequence can be obtained, and then after splicing the current prior token sequence and the current target token sequence, a correct partial code statement or a complete code statement can be obtained.

[0137] On the other hand, before obtaining the current prior token sequence, the LLM model only generates the current prior token sequence containing one or more tokens, rather than directly generating a complete code statement, whether generating one or more tokens according to the operation instruction or splicing the current prior token sequence from the previous target token sequence and the previous prior token sequence, both of which greatly improve the correctness of the prior token sequence.

[0138] With reference to the foregoing Figure 2B which shows a flow 220 of yet another embodiment of the code statement speculation method according to the present disclosure. The code statement speculation method comprises the following steps:

[0139] Step 221, in response to not identifying the end identifier from the current target token sequence, the current target token sequence and the current prior token sequence form a next prior token sequence.

[0140] Based on the current target token sequence determined in the foregoing step 214, after identifying whether it holds the end identifier, if the special token containing the end identifier cannot be identified from the current target token sequence, it can be considered that the current target token sequence is not the last token sequence in the complete code statement, and further generation of the next target token sequence is still required.

[0141] In this case, the current target token sequence can be spliced after the current prior token sequence, and the spliced token sequence can be used as the next prior token sequence, and as the current prior token sequence in the next round of code statement generation process.

[0142] Meanwhile, after generating the next round of current prior token sequence, the hidden state of the current prior token sequence is further generated.

[0143] Wherein, the hidden state of the next round is used to represent the information of semantic abstraction contained in the current prior token sequence, which specifically includes, for example, semantic context information, grammatical structure and / or domain knowledge of DSL, and can still be represented in the form of a tensor.

[0144] Step 222, performing speculation decoding on the next prior token sequence to generate a plurality of candidate token sequences corresponding to the next prior token sequence.

[0145] Based on the next round of current prior token sequence determined in the foregoing step 221, when performing the next round of code statement generation process, the speculation decoding can be performed again in the manner of the foregoing step 211, and the plurality of candidate token sequences generated in the next round.

[0146] Step 223, determining the next target token sequence matching the next prior token sequence from the plurality of candidate token sequences corresponding to the next prior token sequence.

[0147] Based on the plurality of candidate token sequences determined in the foregoing step 223, it can be verified whether each candidate token sequence is valid.

[0148] Further, after verifying that each candidate token sequence is valid, the candidate token sequence with the highest matching degree with the current prior token sequence of the next round can be selected from each valid candidate token sequence as the target token sequence of the next round, i.e., the next target token sequence.

[0149] If any invalid candidate token sequence is verified, it can be corrected to a valid candidate token sequence in the manner of the foregoing step 213.

[0150] Step 224, in response to identifying the end identifier from the next target token sequence, composing the next prior token sequence and the next target token sequence into a code statement.

[0151] Based on the target token sequence of the next round determined in the foregoing step 223, it can be identified whether the special token containing the end identifier is included, and if the special token containing the end identifier is identified, the target token sequence of the next round is spliced after the prior token sequence of the next round to compose a complete code statement.

[0152] From Figure 2B It can be seen that, compared with Figure 2A the code statement speculation method in the embodiment is based on the case that the current target token sequence and the current prior token sequence cannot be spliced to compose a complete code statement, and the process of the target token sequence of the next round is performed again, that is, each token sequence in the complete code statement is generated step by step in the embodiment, and the validity of each token sequence is verified in the process of generating each token sequence. Compared with directly generating a complete code statement, the following technical effects can be achieved, including but not limited to:

[0153] The process of verifying validity is only for a single token sequence, i.e., each candidate token sequence, and the verification speed is obviously improved compared with verifying a complete code statement, so that the speed of the speculation constraint decoding system generating a complete code statement is also correspondingly greatly improved.

[0154] Since only a single token sequence in the complete code statement is generated in each round, the accuracy of generating a single token sequence can be obviously improved compared with directly generating a complete code statement, and since each token sequence is verified for validity, the error rate of the complete code statement spliced from the token sequences is obviously reduced.

[0155] With reference to the foregoing description of the method 200, the method 200 can be implemented by a computer system 1000 as shown in FIG. 10. The computer system 1000 includes a processor 1001, a memory 1002, and a storage 1003. The processor 1001 is configured to execute instructions stored in the memory 1002 and / or the storage 1003. The memory 1002 and the storage 1003 store data and / or instructions for execution by the processor 1001. The memory 1002 and the storage 1003 can be implemented by one or more computer-readable media. The computer system 1000 further includes a display 1004, an input device 1005, and a communication interface 1006. The display 1004 is configured to display information to a user. The input device 1005 is configured to receive input from a user. The communication interface 1006 is configured to communicate with other devices via a network. Figure 3A FIG. 3 shows an exploded flow 300 of one embodiment of the step 213 of the present disclosure. The exploded flow 300 includes the following steps:

[0156] Step 301: determining, from each invalid candidate token sequence, the invalid token corresponding to each item of syntax error, respectively.

[0157] In this step, based on the decoding log created in the foregoing step 211 and the token index value of the invalid token recorded in the step 212, the rollback controller can accurately dynamically track, i.e., accurately locate, the invalid token and determine the error type of the invalid token.

[0158] Step 302: deleting all tokens connected after each invalid token to obtain a revised token sequence.

[0159] Based on the invalid token determined in the foregoing step 301 and the error type thereof, the rollback controller can delete the invalid token and simultaneously delete other tokens connected after the invalid token, so that the invalid candidate token sequence is rolled back to the most recent valid state in history to complete the rollback operation and obtain a revised token sequence.

[0160] In some specific embodiments, taking the candidate token sequence of “META_TYPE IS LOCATION WITH” as an example, if the token “LOCATION ” therein has a syntax error, when performing the rollback operation, a plurality of continuously spliced tokens “LOCATION WITH” can be deleted, only 2 tokens “META_TYPE IS” are retained as the most recent valid state in history of the candidate token sequence, and “META_TYPE IS” is taken as the revised token sequence.

[0161] Step 303: performing sentence prediction on each revised token sequence to generate a valid candidate token sequence.

[0162] Based on the revised token sequence determined in the foregoing step 302, it can be input into the LLM model to perform sentence prediction, thereby generating another valid candidate token sequence.

[0163] Specifically, the process of modifying the invalid candidate token sequence in the step 303, i.e., the rollback operation and the sentence prediction, can include the following steps 3031 to 3033 as shown. Figure 3B

[0164] ​Based on this, the decomposition process 300 of step 213 in this embodiment can, by performing a backtracking operation on the invalid word units in the invalid candidate word unit sequence, remove the syntax errors caused by the invalid word units while still retaining the correct word units, compared with deleting the entire candidate word unit sequence, it avoids excessive waste of computing resources.

[0165] With continued reference to Figure 3B which shows a decomposition process 3030 of one embodiment of step 303 of the present disclosure. The decomposition process 3030 includes the following steps:

[0166] Step 3031, performing sentence prediction on each modified word unit sequence to determine a corresponding word unit probability distribution, the word unit probability distribution representing the probability of selecting each word unit after the modified word unit sequence.

[0167] Based on the modified word unit sequence determined in the preceding step 302, after inputting it into the LLM model, the LLM model can first generate a word unit probability distribution corresponding to the modified word unit sequence during the process of performing sentence prediction on the modified word unit sequence, so that another corresponding candidate word unit sequence can be determined according to the word unit probability distribution in the following steps.

[0168] Among them, the word unit probability distribution specifically represents the probability that the LLM model considers when concatenating other word units after the modified word unit sequence, for each word unit available for selection, the probability of selecting each word unit, which represents the probability that the word unit is the correct option, or represents the matching rate between the word unit and the modified word unit sequence.

[0169] Step 3032, adjusting the word unit probability distribution corresponding to each modified word unit sequence.

[0170] Based on the word unit probability distribution determined in the preceding step 3031, and the word unit index value of the invalid word unit recorded in the preceding step 212, the probability corresponding to the invalid word unit in the word unit probability distribution in the invalid candidate word unit sequence can be determined.

[0171] Based on this, the probability corresponding to the invalid word unit can be adjusted.

[0172] In some optional embodiments, the probability corresponding to the invalid word unit can be adjusted to 0, so that the LLM model no longer selects the invalid word unit as the next concatenated word unit when performing sentence prediction on the modified word unit sequence, thereby avoiding the same error.

[0173] In some optional embodiments, the probability corresponding to the invalid token can also be reduced to a preset probability value, so that the possibility of the LLM model selecting the invalid token is greatly reduced when performing sentence prediction on the modified token sequence, and since the probability corresponding to the invalid token is not 0, the global integrity of the LLM model when performing sentence prediction can be ensured while reducing the probability of selecting the invalid token.

[0174] In step 3033, the valid candidate token sequence is obtained by determining the token connected to the corresponding modified token sequence using the adjusted token probability distribution.

[0175] Based on the adjustment of the token probability distribution in the foregoing step 3032, the possibility of obtaining another valid candidate token sequence is greatly improved when the LLM model performs sentence prediction on the modified token sequence using the adjusted token probability distribution.

[0176] In some optional embodiments, since the predicted another candidate token sequence at this time is obtained based on the adjusted token probability distribution, it can be directly used as a valid candidate token sequence to improve the overall operation efficiency of the speculative constraint decoding system.

[0177] In some optional embodiments, for the predicted another candidate token sequence, it can also be input into the parsing verifier again to verify the another candidate token sequence to ensure that all candidate token sequences are valid.

[0178] In this embodiment, if the another candidate token sequence is still invalid after verification, it can be modified again, that is, the backtracking operation and the sentence prediction are performed again, and after the modification is completed, it is verified whether the candidate token sequence obtained again is valid.

[0179] In this repeated modification and verification process, the recursion depth can be set in advance to avoid the speculative constraint decoding system falling into too many repeated modification and verification processes, thereby wasting computing resources, or to avoid it falling into an unlimited repeated modification and verification situation.

[0180] Specifically, the recursion depth can be set to, for example, for an invalid candidate token sequence that has not undergone the backtracking operation, the number of times of performing the backtracking operation after the backtracking operation is not more than, for example, 20 times.

[0181] Accordingly, after the invalid candidate token sequence is subjected to 20 times of backtracking to generate 20 different candidate token sequences, and still cannot obtain a valid candidate token sequence, the invalid candidate token sequence can be deleted, thereby ensuring the operation efficiency of the speculative constraint decoding system.

[0182] Further reference Figure 4As an implementation of the method shown in the above figures, the disclosure provides an embodiment of a keyword extraction device, which corresponds to the method embodiment shown in FIG. 2, and the device can be specifically applied to various electronic devices.

[0183] As shown in Figure 4 , the keyword extraction device 400 of the embodiment includes a speculation decoding model response module 401, an analysis verifier response module 402, a correction module 403, and a large language model response module 404. Among them, the speculation decoding model response module 401 is configured to, in response to obtaining a current prior token sequence, perform speculation decoding on the current prior token sequence to generate a plurality of candidate token sequences for connecting the current prior token sequence; the analysis verifier response module 402 is configured to verify whether each candidate token sequence is valid; the correction module 403 is configured to, in response to determining that any candidate token sequence is invalid, correct the invalid candidate token sequence to a valid candidate token sequence; and the large language model response module 404 is configured to determine a current target token sequence matching the current prior token sequence from each valid candidate token sequence; the speculation decoding model response module 401 is further configured to, in response to identifying an end identifier from the current target token sequence, combine the current prior token sequence and the current target token sequence into a code statement.

[0184] In the embodiment, the specific processing of the speculation decoding model response module 401, the analysis verifier response module 402, the correction module 403, and the large language model response module 404 and the technical effects brought by them can be respectively referred to Figure 2A The related descriptions of steps 211, 212, 213, 214, and 215 in the corresponding embodiment will not be repeated here.

[0185] In some optional implementations, the analysis verifier response module 402 is further configured to:

[0186] verify whether each candidate token sequence has a syntax error according to a preset syntax rule;

[0187] determine a candidate token sequence having at least one syntax error as an invalid candidate token sequence, and determine a candidate token sequence having no syntax error as a valid candidate token sequence.

[0188] In some optional implementations, each candidate token sequence includes at least one token connected in a predetermined order;

[0189] Correspondingly, the correction module 403 is further configured to:

[0190] determine an invalid token corresponding to each syntax error from each invalid candidate token sequence;

[0191] deleting each invalid token and all tokens connected after each invalid token, to obtain a modified token sequence;

[0192] performing sentence prediction on each modified token sequence to generate a valid candidate token sequence.

[0193] In some optional embodiments, performing sentence prediction on each modified token sequence to generate a valid candidate token sequence comprises:

[0194] performing sentence prediction on each modified token sequence to determine a corresponding token probability distribution, the token probability distribution representing probabilities of selecting each token after each modified token sequence;

[0195] adjusting the token probability distribution corresponding to each modified token sequence;

[0196] determining, by using the adjusted token probability distribution, a token connected to each modified token sequence to obtain a valid candidate token sequence.

[0197] In some optional embodiments, adjusting the token probability distribution corresponding to each modified token sequence comprises:

[0198] determining, from the token probability distribution corresponding to each modified token sequence, a probability of selecting a corresponding invalid token;

[0199] adjusting the probability of selecting the corresponding invalid token to obtain an adjusted token probability distribution.

[0200] In some optional embodiments, the code statement prediction apparatus further comprises:

[0201] The iteration module 405 is configured to, after determining, from each valid candidate token sequence, a current target token sequence matching the current prior token sequence, perform the following steps:

[0202] in response to not recognizing an end identifier from the current target token sequence, combining the current target token sequence and the current prior token sequence to form a next prior token sequence;

[0203] performing prediction decoding on the next prior token sequence to generate a plurality of candidate token sequences corresponding to the next prior token sequence;

[0204] determining, from the plurality of candidate token sequences corresponding to the next prior token sequence, a next target token sequence matching the next prior token sequence;

[0205] in response to recognizing an end identifier from the next target token sequence, combining the next prior token sequence and the next target token sequence to form the code statement.

[0206] In some alternative embodiments, the speculative means of code statement further comprises:

[0207] The initial module 406 is configured to perform the following steps in response to obtaining the current prior token sequence:

[0208] In response to receiving the operation instruction, an initial prior token sequence corresponding to the code statement is generated, wherein the initial prior token sequence comprises at least one token in the code statement.

[0209] The initial prior token sequence is determined as the current prior token sequence. It should be noted that the implementation details and technical effects of each unit in the keyword extraction device provided by the embodiments of the present disclosure can refer to the description of other embodiments of the present disclosure, and will not be described here.

[0210] Reference will be made to Figure 5 which shows a structural schematic diagram of a computer system 500 suitable for implementing an electronic device of the present disclosure. Figure 5 The computer system 500 shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0211] As shown in Figure 5 , the computer system 500 can include a processing device (such as a central processor, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the computer system 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0212] Generally, the following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication devices 509 can allow the computer system 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The computer system 500 with various devices is shown, but it should be understood that all the devices shown are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0213] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of embodiments of the present disclosure are executed.

[0214] Note that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, device, or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take on many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can transmit, propagate, or transport program for use by or in connection with an instruction execution system, device, or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF (radio frequency), or the like, or any suitable combination of the foregoing.

[0215] The computer readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and not be assembled into the electronic device.

[0216] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to implement the speculation method of the code statements shown in the embodiment and optional implementation manner thereof as shown in FIG. 2, and / or the speculation method of the code statements shown in the embodiment and optional implementation manner thereof as shown in FIG. 3.

[0217] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages or combinations of languages including object or visual programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0218] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0219] The units or modules involved in the embodiments of the present disclosure can be implemented by software or by hardware. In some cases, the name of the unit or module does not constitute a limitation on the unit itself, for example, the speculation decoding model response module can also be described as "a module that, in response to obtaining a current prior word sequence, performs speculation decoding on the current prior word sequence to generate a plurality of candidate word sequences for connecting the current prior word sequence".

[0220] The above description is merely that of the preferred embodiments of the present disclosure and a description of the technical principles of the present disclosure. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions with the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the concept of the above disclosure. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed in the present disclosure (but not limited to) without departing from the concept of the above disclosure.

Claims

1. A method for inferring a code statement, comprising: In response to obtaining the current prior word-gram sequence, performing speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence; Verify that each candidate word sequence is valid; In response to determining that any candidate word-gram sequence is invalid, revising the invalid candidate word-gram sequence to a valid candidate word-gram sequence; Determine a current target word sequence that matches the current prior word sequence from each valid candidate word sequence; In response to identifying an end marker from the current target word-gram sequence, the current prior word-gram sequence and the current target word-gram sequence are combined into a code statement.

2. The method according to claim 1, wherein Verifying whether each candidate word-meta sequence is valid includes: Use the preset grammatical rules to verify whether each candidate word sequence has grammatical errors; A candidate word-gram sequence having at least one grammatical error is determined as an invalid candidate word-gram sequence, and a candidate word-gram sequence having no grammatical error is determined as a valid candidate word-gram sequence.

3. The method according to claim 2, wherein: Each candidate word-gram sequence includes at least one word-gram connected in a predetermined order; The step of correcting an invalid candidate word-gram sequence into a valid candidate word-gram sequence includes: Determining the invalid tokens corresponding to the respective grammatical errors from each invalid candidate token sequence; Delete each invalid word and all words connected to each invalid word to obtain a corrected word sequence; Sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence.

4. The method according to claim 3, wherein: The step of performing sentence prediction on each modified word-gram sequence to generate a valid candidate word-gram sequence includes: performing sentence prediction on each modified word-gram sequence to determine a corresponding word-gram probability distribution, wherein the word-gram probability distribution represents the probability of selecting each word-gram after each modified word-gram sequence; Adjusting the word unit probability distribution corresponding to each modified word unit sequence; The word units connected to the corresponding modified word unit sequence are determined using the adjusted probability distribution of each word unit to obtain a valid candidate word unit sequence.

5. The method according to claim 4, wherein The adjusting of the word unit probability distribution corresponding to each modified word unit sequence includes: Determining the probability of selecting the corresponding invalid word from the word probability distribution corresponding to each modified word sequence; The probability of selecting the corresponding invalid word is adjusted to obtain an adjusted word probability distribution.

6. The method according to claim 1, wherein After determining a current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence, the method further includes: In response to not recognizing an end marker from the current target word-gram sequence, combining the current target word-gram sequence and the current prior word-gram sequence into a next prior word-gram sequence; Performing speculative decoding on the next prior word-gram sequence to generate multiple candidate word-gram sequences corresponding to the next prior word-gram sequence; Determining a next target word-gram sequence that matches the next prior word-gram sequence from a plurality of candidate word-gram sequences corresponding to the next prior word-gram sequence; In response to identifying an end marker from the next target word-gram sequence, the next prior word-gram sequence and the next target word-gram sequence are combined into a code statement.

7. The method according to claim 1, wherein The code statement is used to respond to the received operation instruction; Before responding to obtaining the current prior word-gram sequence, the method further includes: In response to receiving the operation instruction, generating a corresponding initial a priori word-gram sequence, wherein the initial a priori word-gram sequence includes at least one word-gram in the code statement; The initial a priori word-gram sequence is determined as a current a priori word-gram sequence.

8. A device for inferring code statements, comprising: a speculative decoding model response module configured to, in response to obtaining the current prior word-gram sequence, perform speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence; a parsing validator response module configured to verify whether each candidate word-meta sequence is valid; a correction module configured to, in response to determining that any candidate word-gram sequence is invalid, correct the invalid candidate word-gram sequence into a valid candidate word-gram sequence; The large language model response module is configured to determine a current target word sequence that matches the current prior word sequence from each valid candidate word sequence; The inferred decoding model response module is further configured to, in response to identifying an end marker from the current target word sequence, combine the current prior word sequence and the current target word sequence into a code statement.

9. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the code statement speculation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by one or more processors, the computer program implements the method for inferring code statements according to any one of claims 1 to 7.

11. A computer program product comprising computer program instructions, wherein: When the computer program instructions are executed on a computer, the computer is caused to execute the method for inferring the code statements according to any one of claims 1 to 7.

Images