Code statement speculation method and device, storage medium, electronic equipment and program

Abstract

The invention provides a code statement speculation method and device, a storage medium, electronic equipment and a program. A specific embodiment of the method comprises the following steps: in response to an obtained current priori lexical element sequence, executing speculative decoding on the current priori lexical element sequence, and generating a plurality of candidate lexical element sequences for connecting the current priori lexical element sequence; verifying whether each candidate lexical element sequence is valid or not; in response to determining that any candidate lexical element sequence is invalid, correcting the invalid candidate lexical element sequence into an effective candidate lexical element sequence; determining a current target lexical element sequence matched with the current priori lexical element sequence from each effective candidate lexical element sequence; and forming a code statement by the current priori lexical element sequence and the current target lexical element sequence in response to an end identifier recognized from the current target lexical element sequence. According to the embodiment, the accuracy of the code statements is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of code generation technology, and in particular to a method, device, storage medium, electronic device, and program for inferring code statements. Background Art

[0002] With the rapid development of artificial intelligence technology, artificial intelligence models can be used to generate code statements in programming languages ​​when generating code.

[0003] Typically, code statements can be generated using the LLM (Large Language Model) model. However, for programming languages ​​with complex grammatical rules, such as DSLs (Domain Specific Languages), LLM often has difficulty strictly adhering to their grammatical rules, resulting in a high error rate in the generated code statements. Furthermore, the verification process for the generated code statements significantly reduces efficiency.

[0004] On the other hand, the general reasoning acceleration method of the LLM model is often to use speculative decoding to directly generate complete code statements, which are then verified by the LLM model. However, because speculative decoding is more difficult to apply to complex grammatical rules, the error rate of the complete code statements it generates is higher. Not only does this fail to achieve an acceleration effect, but the higher error rate will further reduce the efficiency of the LLM model. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide a method, apparatus, storage medium, electronic device, and program for inferring code statements.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for inferring a code statement, the method comprising: in response to obtaining a current prior word-gram sequence, performing speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence; Verify that each candidate word sequence is valid; In response to determining that any candidate word-gram sequence is invalid, revising the invalid candidate word-gram sequence to a valid candidate word-gram sequence; Determine a current target word sequence that matches the current prior word sequence from each valid candidate word sequence; In response to identifying an end marker from the current target word-gram sequence, the current prior word-gram sequence and the current target word-gram sequence are combined into a code statement.

[0007] In some optional implementations, verifying whether each candidate word-gram sequence is valid includes: Use the preset grammatical rules to verify whether each candidate word sequence has grammatical errors; A candidate word-gram sequence having at least one grammatical error is determined as an invalid candidate word-gram sequence, and a candidate word-gram sequence having no grammatical error is determined as a valid candidate word-gram sequence.

[0008] In some optional embodiments, each candidate word-gram sequence includes at least one word-gram connected in a predetermined order; Accordingly, the invalid candidate word-gram sequence is corrected into a valid candidate word-gram sequence, including: Determining the invalid tokens corresponding to the respective grammatical errors from each invalid candidate token sequence; Delete each invalid word and all words connected to each invalid word to obtain a corrected word sequence; Sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence.

[0009] In some optional implementations, sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence, including: performing sentence prediction on each modified word-gram sequence to determine a corresponding word-gram probability distribution, wherein the word-gram probability distribution represents the probability of selecting each word-gram after each modified word-gram sequence; Adjusting the word unit probability distribution corresponding to each modified word unit sequence; The word units connected to the corresponding modified word unit sequence are determined using the adjusted probability distribution of each word unit to obtain a valid candidate word unit sequence.

[0010] In some optional implementations, adjusting the word-gram probability distribution corresponding to each modified word-gram sequence includes: Determining the probability of selecting the corresponding invalid word from the word probability distribution corresponding to each modified word sequence; The probability of selecting the corresponding invalid word is adjusted to obtain an adjusted word probability distribution.

[0011] In some optional implementations, after determining the current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence, the following steps may be further performed: In response to not recognizing an end marker from the current target word-gram sequence, combining the current target word-gram sequence and the current prior word-gram sequence into a next prior word-gram sequence; Performing speculative decoding on the next prior word-gram sequence to generate multiple candidate word-gram sequences corresponding to the next prior word-gram sequence; Determining a next target word-gram sequence that matches the next prior word-gram sequence from a plurality of candidate word-gram sequences corresponding to the next prior word-gram sequence; In response to identifying an end marker from the next target word-gram sequence, the next prior word-gram sequence and the next target word-gram sequence are combined into a code statement.

[0012] In some optional implementations, the code statement is used to respond to a received operation instruction; Accordingly, before responding to obtaining the current prior word sequence, the following steps may be performed in advance: In response to receiving the operation instruction, generating a corresponding initial a priori word-gram sequence, wherein the initial a priori word-gram sequence includes at least one word-gram in the code statement; The initial a priori word-gram sequence is determined as a current a priori word-gram sequence.

[0013] In a second aspect, an embodiment of the present disclosure provides a device for inferring code statements, the device comprising: a speculative decoding model response module configured to, in response to obtaining the current prior word-gram sequence, perform speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence; a parsing validator response module configured to verify whether each candidate word-meta sequence is valid; a correction module configured to, in response to determining that any candidate word-gram sequence is invalid, correct the invalid candidate word-gram sequence into a valid candidate word-gram sequence; The large language model response module is configured to determine a current target word sequence that matches the current prior word sequence from each valid candidate word sequence; The inferred decoding model response module is further configured to, in response to identifying an end marker from the current target word sequence, combine the current prior word sequence and the current target word sequence into a code statement.

[0014] In some optional embodiments, the parsing validator response module is further configured to: Use the preset grammatical rules to verify whether each candidate word sequence has grammatical errors; A candidate word-gram sequence having at least one grammatical error is determined as an invalid candidate word-gram sequence, and a candidate word-gram sequence having no grammatical error is determined as a valid candidate word-gram sequence.

[0015] In some optional embodiments, each candidate word-gram sequence includes at least one word-gram connected in a predetermined order; Accordingly, the correction module is further configured to: Determine the invalid word units corresponding to the respective grammatical errors from each invalid candidate word unit sequence; Delete each invalid word and all words connected to each invalid word to obtain a corrected word sequence; Sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence.

[0016] In some optional implementations, sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence, including: performing sentence prediction on each modified word-gram sequence to determine a corresponding word-gram probability distribution, wherein the word-gram probability distribution represents the probability of selecting each word-gram after each modified word-gram sequence; Adjusting the word unit probability distribution corresponding to each modified word unit sequence; The word units connected to the corresponding modified word unit sequence are determined using the adjusted probability distribution of each word unit to obtain a valid candidate word unit sequence.

[0017] In some optional implementations, adjusting the word-gram probability distribution corresponding to each modified word-gram sequence includes: Determining the probability of selecting the corresponding invalid word from the word probability distribution corresponding to each modified word sequence; The probability of selecting the corresponding invalid word is adjusted to obtain an adjusted word probability distribution.

[0018] In some optional implementations, the code statement inference device further includes: The iteration module is configured to, after determining a current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence, perform the following steps: In response to not recognizing an end marker from the current target word-gram sequence, combining the current target word-gram sequence and the current prior word-gram sequence into a next prior word-gram sequence; Performing speculative decoding on the next prior word-gram sequence to generate multiple candidate word-gram sequences corresponding to the next prior word-gram sequence; Determining a next target word-gram sequence that matches the next prior word-gram sequence from a plurality of candidate word-gram sequences corresponding to the next prior word-gram sequence; In response to identifying an end marker from the next target word-gram sequence, the next prior word-gram sequence and the next target word-gram sequence are combined into a code statement.

[0019] In some optional implementations, the code statement inference device further includes: The initialization module is configured to perform the following steps before responding to obtaining the current prior word sequence: In response to receiving the operation instruction, generating a corresponding initial a priori word-gram sequence, wherein the initial a priori word-gram sequence includes at least one word-gram in the code statement; The initial a priori word-gram sequence is determined as a current a priori word-gram sequence.

[0020] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0021] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by one or more processors, implements the method described in any implementation manner in the first aspect.

[0022] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in any implementation manner in the first aspect when the computer program / instruction is executed by a processor.

[0023] To address the low efficiency and high error rate of code statement generation using the LLM model, embodiments of the present disclosure provide a code statement inference method, apparatus, storage medium, electronic device, and program. These methods perform inference decoding on an acquired current prior word-gram sequence to generate multiple candidate word-gram sequences to be concatenated with the prior word-gram sequence, rather than directly generating a complete code statement. This avoids the efficiency reduction caused by directly generating a complete code statement. Subsequently, each candidate word-gram sequence is verified to quickly identify invalid candidate word-gram sequences. Compared to verifying the entire code statement, the verification rate is significantly improved. Furthermore, after validating an invalid candidate word-gram sequence, corrections can be made to ensure that multiple candidate word-gram sequences are available. This allows selection from a wide range of backup options when selecting a current target word-gram sequence, i.e., selection from a large number of candidate word-gram sequences. This allows a more accurate current target word-gram sequence to be obtained compared to directly generating the current target word-gram sequence or selecting the current target word-gram sequence from a small number of candidate word-gram sequences. Furthermore, after concatenating the current prior word-gram sequence with the current target word-gram sequence, a correct partial code statement or a complete code statement can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects, and advantages of the present disclosure will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are for illustration purposes only and are not to be considered as limiting the present invention. In the drawings: Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied; Figure 2A is a flowchart of an embodiment of a code statement inference method according to the present disclosure; Figure 2Bis a flowchart of another embodiment of the code statement inference method according to the present disclosure; Figure 3A A decomposition flow chart of an embodiment of the code statement inference method step 213 according to the present disclosure; Figure 3B is a decomposed flow chart of an embodiment of the code statement inference method step 303 according to the present disclosure; Figure 4 1 is a schematic structural diagram of an embodiment of a code statement inference device according to the present disclosure; Figure 5 A schematic diagram of the structure of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0026] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, storage medium, electronic device, and program for speculating code statements of the present disclosure may be applied.

[0028] like Figure 1 As shown, system architecture 100 may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. Network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0029] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as natural language processing applications, voice recognition applications, short video social applications, audio and video conferencing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0030] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be hardware or software. When the first terminal device 101, the second terminal device 102, and the third terminal device 103 are hardware, they can be various electronic devices with a sound capture device (e.g., a microphone), a video capture device (e.g., a camera), and a display screen, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, and the like. When the first terminal device 101, the second terminal device 102, and the third terminal device 103 are software, they can be installed in the terminal devices listed above. They can be implemented as multiple software programs or software modules (e.g., to provide a code statement inference service) or as a single software program or software module. This is not specifically limited here.

[0031] In some cases, the code statement inference method provided by the present disclosure can be executed by the first terminal device 101, the second terminal device 102, and the third terminal device 103. Accordingly, the code statement inference apparatus can be set in the first terminal device 101, the second terminal device 102, and the third terminal device 103. In this case, the system architecture 100 may also not include the server 105.

[0032] In some cases, the code statement inference method provided by the present disclosure can be jointly executed by the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. For example, the step of "in response to obtaining the current prior word-gram sequence, performing inference decoding on the current prior word-gram sequence, and generating multiple candidate word-gram sequences for connecting the current prior word-gram sequence" can be executed by the first terminal device 101, the second terminal device 102, and the third terminal device 103, and the steps of "verifying whether each candidate word-gram sequence is valid" can be executed by the server 105. The present disclosure does not limit this. Accordingly, the code statement inference device can also be respectively set in the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105.

[0033] In some cases, the code statement inference method provided in the present disclosure can be executed by the server 105. Accordingly, the code statement inference device can also be set in the server 105. In this case, the system architecture 100 may also not include the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0034] It should be noted that server 105 can be either hardware or software. When server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or software module. This is not specifically limited here.

[0035] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0036] It should be noted that the code statement inference method disclosed in the present invention is deployed in Figure 1 It is implemented by the speculation constraint decoding system in the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105, and the speculation constraint decoding system includes a pre-trained LLM model, a pre-trained speculation decoding model, a pre-trained parsing verifier and a pre-trained fallback controller, etc.

[0037] During the training process, the LLM model, the inference decoding model, and the parsing verifier are trained based on the grammatical rules of the code language processed by the inference constraint decoding system, for example, based on the grammatical rules of the DSL.

[0038] That is, the LLM model, the speculative decoding model, and the parsing verifier in the inference constraint decoding system can perform the following operations according to the grammatical rules of the DSL.

[0039] In addition, the LLM model in the speculation constraint decoding system can receive an operation instruction, so that the speculation constraint decoding system generates a code statement for responding to the operation instruction based on the operation instruction.

[0040] The operation instruction is expressed in a manner that can be analyzed and processed by the LLM model. It can be a complete sentence or an incomplete sentence expressed in natural language, or one or more prompt words expressed in natural language, so that when the LLM model receives the operation instruction, it can recognize and process the natural language; the operation instruction can also be expressed in other code languages; there is no specific limitation on the operation instruction here.

[0041] Continue to refer Figure 2A , which shows a process 210 of an embodiment of the code statement inference method disclosed herein, the code statement inference method includes the following steps: Step 211 : In response to obtaining the current prior word-gram sequence, perform speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence.

[0042] In this embodiment, a word-gram sequence represents a segment composed of one or more words in a predetermined order, and multiple word-gram sequences in a predetermined order constitute a complete code statement.

[0043] Among them, the prior word-gram sequence represents a word-gram sequence without grammatical errors or with negligible grammatical errors. The prior word-gram sequence is generated by the LLM model, and the current prior word-gram sequence is the prior word-gram sequence obtained or generated at the current moment.

[0044] In some optional embodiments, when the LLM generates a priori word-gram sequence for the first time, the first generated priori word-gram sequence, that is, the initial priori word-gram sequence, can be used as the current priori word-gram sequence. In this case, the current priori word-gram sequence can be one or more words generated by the LLM model based on the received operation instructions. Since each word-gram at this time is one or more starting words among many words in a complete code statement, the probability of one or more words at this time having grammatical errors is very low and can be ignored. Therefore, it can be directly used as the initial priori word-gram sequence without verifying whether its grammar is valid.

[0045] In some other optional implementations, when the LLM does not generate a priori word-gram sequence for the first time, the current priori word-gram sequence may be composed of a target word-gram sequence selected by the LLM model from a plurality of grammatically validated word-gram sequences and a previous priori word-gram sequence.

[0046] In this step, the current prior word-gram sequence generated based on the LLM model can be input into the inference decoding module, and the inference decoding module performs inference decoding, thereby simultaneously generating multiple possible candidate word-gram sequences, wherein each candidate word-gram sequence is composed of one or more words, which are used to connect to the current prior word-gram sequence and together with the current prior word-gram sequence form a code statement or a part of a code statement.

[0047] It should be noted that, in some cases, each word may contain both characters with actual semantic meaning and characters without actual semantic meaning. For example, the candidate word sequence is "META_TYPE IS LOCATION WITH", which contains the words: "META_TYPE", "IS", "LOCATION" and "WITH". It can be seen that "IS" is a two-character word "IS" consisting of the characters "I" and "S". In other cases, the word here may also include a space, the character "I" and the character "S", forming a word "IS" with a space and two characters.

[0048] Furthermore, after generating the candidate word-gram sequence, a corresponding decoding log can be created for the generated candidate word-gram sequence, and the decoding log can record the word-gram position of each word-gram in each candidate word-gram sequence, so that the inference constraint decoding system has the dynamic tracking function of each word-gram.

[0049] The word unit position can be specifically represented by the word unit index value in the decoding log. The word unit index value can be used to trace back specific information such as the row number, column number and / or specific character of the corresponding word unit.

[0050] In other words, by recording the word-unit index values, the inference constraint decoding system can accurately locate each word-unit.

[0051] In some optional implementations, the speculative decoding module may specifically be, for example, an EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) framework.

[0052] It should be noted that speculative decoding based on the current prior word sequence is a technology that is currently widely studied and applied, and is not specifically limited here. For example, a tree-shaped reasoning structure can be constructed based on the hidden state of the operation instruction, and according to the logical relationship between the various nodes in the tree-shaped reasoning structure, multiple candidate word sequences as leaf nodes can be inferred. Among them, the hidden state can be, for example, a tensor in the form of a multi-dimensional matrix, which is used to represent the semantic abstract information contained in the operation instruction by the LLM model, specifically including, for example, semantic context information, grammatical structure and / or DSL domain knowledge, etc. The hidden state is obtained by the LLM model in the process of identifying the operation instruction and generating the current prior word sequence.

[0053] Based on this, the method of generating multiple candidate word sequences in parallel by inferring decoding based on the current prior word sequence can provide more sufficient and extensive choices when subsequently determining the current target word sequence, thereby improving the accuracy of the current target word sequence.

[0054] Step 212: Verify whether each candidate word-gram sequence is valid.

[0055] Based on the multiple candidate word-gram sequences determined in the aforementioned step 211, since each candidate word-gram sequence is a multiple possible option when connecting a word-gram sequence after the current prior word-gram sequence, if there is an invalid candidate word-gram sequence, then in the process of selecting a target word-gram sequence from multiple possible candidate word-gram sequences, the invalid candidate word-gram sequence will be used as the target word-gram sequence and form a code statement with the prior word-gram sequence, thereby wasting the operating resources of the electronic device.

[0056] Based on this, before determining the target word sequence, it is possible to first verify whether each candidate word sequence is valid, so as to avoid invalid candidate word sequences from interfering in the process of selecting the target word sequence below, thereby improving the accuracy of the target word sequence, and then improving the accuracy of the code statement composed of the target word sequence and the prior word sequence.

[0057] In some optional embodiments, the process of verifying whether each candidate word-gram sequence is valid may be performed by a parsing verifier in the inference constraint decoding system, which may be, for example, an ANTLR parser (ANother Tool for Language Recognition).

[0058] In this step, since the parsing verifier is trained with predetermined DSL grammar rules during the pre-training process, as an optional implementation, when verifying whether each candidate word-unit sequence is valid, it can be judged based on whether the grammar of the candidate word-unit sequence conforms to the preset DSL grammar rules.

[0059] Specifically, the ANTLR grammar parser can parse each candidate word-unit sequence into a tree-like parsing structure, wherein each tree-like parsing structure has a plurality of leaf nodes, and each leaf node is a word-unit in the corresponding candidate word-unit sequence.

[0060] Furthermore, an error listener may be set in the ANTLR grammar parser, and the error listener may be used to detect grammatical errors in the tree-like parsing structure.

[0061] In some cases, taking the candidate word-gram sequence "META_TYPE IS LOCATION WITH" as an example, after parsing it into a tree-like parsing structure, each leaf node is the word-gram "META_TYPE", "IS", "LOCATION" and "WITH".

[0062] Based on this, the error listener in the ANTLR grammar parser can verify whether each leaf node has a grammatical error according to the DSL grammar rules. For example, there is a grammatical error in the "IS" of the leaf node, which should be lowercase "is" in the correct grammar.

[0063] In some optional embodiments, based on the verification of the ANTLR grammar parser, for any candidate word-gram sequence, if it is verified that the candidate word-gram sequence has one or more grammatical errors, the candidate word-gram sequence can be determined to be invalid, and the following step 213 can be further executed; if it is verified that the candidate word-gram sequence does not have any grammatical errors, the candidate word-gram sequence can be determined to be valid, and the following step 214 can be further directly executed.

[0064] In some other optional implementations, in the actual application of DSL, since some grammatical errors with minor effects can be ignored, these negligible minor grammatical errors can also be pre-defined. When the ANTLR parser verifies that a candidate word sequence has only one or more negligible minor grammatical errors, the candidate word sequence can also be determined to be valid.

[0065] Furthermore, when the error listener detects a grammatical error, it can record the error type of the grammatical error and accurately mark the location where the grammatical error occurs. For example, after determining the leaf node with the grammatical error, it can record the word index value of the word corresponding to the leaf node and determine the word with the grammatical error as an invalid word.

[0066] Based on the determined grammatical errors and the recorded word index values ​​of the invalid words, the invalid words and their positions can be accurately located in the following subsequent steps, thereby improving the efficiency of the correction operation in the following steps.

[0067] In step 213 , in response to determining that any candidate word-gram sequence is invalid, the invalid candidate word-gram sequence is modified into a valid candidate word-gram sequence.

[0068] Based on the verification of each candidate word sequence in the aforementioned step 212, if any invalid candidate word sequence is verified, each invalid candidate word sequence can be corrected separately to ensure that the accuracy of the target word sequence is improved when selecting the target word sequence below.

[0069] In this step, the correction of invalid candidate word-gram sequences can be performed by the backoff controller and LLM model in the inferred constraint decoding system.

[0070] In some optional embodiments, based on the decoding log created in the aforementioned step 211 and the word index value of the invalid word recorded in step 212, the fallback controller can accurately locate the invalid word and determine the error type of the invalid word.

[0071] Based on this, the fallback controller can perform a fallback operation based on the invalid words and their error types in the invalid candidate word sequence, and restore this invalid candidate word sequence to the most recent valid state in history; and further use the LLM model to perform sentence prediction on the candidate word sequence after fallback to generate another candidate word sequence.

[0072] Specifically, the process of correcting the invalid candidate word sequence in step 213, i.e., the fallback operation and sentence prediction, may include the following steps: Figure 3A Steps 301 to 303 shown: Furthermore, for another generated candidate word-gram sequence, the verification in step 212 may be performed again to verify whether the generated candidate word-gram sequence has any grammatical errors.

[0073] If the regenerated candidate word-gram sequence still has grammatical errors, that is, the regenerated candidate word-gram sequence is still an invalid candidate word-gram sequence, the fallback controller and the LLM model can repeat the above correction operation on the regenerated candidate word-gram sequence and perform the verification of step 212 again.

[0074] If the regenerated candidate word-gram sequence does not have any grammatical errors, that is, the regenerated candidate word-gram sequence is a valid candidate word-gram sequence, then the following subsequent steps may be continued.

[0075] Step 214 : Determine a current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence.

[0076] Based on the verification in step 212 and the correction in step 213 , when all candidate word-gram sequences are valid word-gram sequences, the current target word-gram sequence can be selected from each valid candidate word-gram sequence.

[0077] Specifically, as mentioned above, since the target word sequence is the word sequence connected after the corresponding prior word sequence, in this step, each valid candidate word sequence can be input into the LLM model. The LLM model can determine the degree of matching between each valid candidate word sequence and the current prior word sequence based on the current prior word sequence, and select the candidate word sequence with the highest matching degree as the current target word sequence that matches the current prior word sequence.

[0078] Step 215 , in response to identifying an end marker from the current target word-gram sequence, composing the current prior word-gram sequence and the current target word-gram sequence into a code statement.

[0079] Based on the current target word sequence determined in the aforementioned step 214, the current target word sequence can be concatenated after the current prior word sequence, and a complete code statement or a partial code statement can be obtained depending on whether the current target word sequence contains an end marker.

[0080] In some optional embodiments, a special word can be pre-set. The special word has an end mark and is used to indicate the completion of the code statement. That is, when the current target word sequence contains a special word, the current target word sequence can be considered to be the last word sequence in the complete code statement, so that after splicing the current target word sequence to the current prior word sequence, a complete code statement can be obtained.

[0081] The code statement inference method provided by the above-mentioned embodiments of the present disclosure generates multiple candidate word-meta sequences for concatenation by performing inference decoding on the acquired current prior word-meta sequence, rather than directly generating a complete code statement. This avoids the problem of reduced efficiency caused by directly generating a complete code statement. Subsequently, by verifying each candidate word-meta sequence, invalid candidate word-meta sequences can be quickly determined. Compared with verifying the entire complete code statement, the verification rate is significantly improved. Moreover, after verifying an invalid candidate word-meta sequence, by correcting it, multiple candidate word-meta sequences can still be guaranteed. Therefore, when selecting the current target word-meta sequence, it is possible to select from a sufficiently wide range of backup options, that is, to select from a large number of candidate word-meta sequences. Compared with directly generating the current target word-meta sequence or selecting the current target word-meta sequence from a small number of candidate word-meta sequences, a more accurate current target word-meta sequence can be obtained. Furthermore, after concatenating the current prior word-meta sequence with the current target word sequence, a correct partial code statement or a complete code statement can be obtained.

[0082] On the other hand, before obtaining the current prior word-gram sequence, the LLM model only generates the current prior word-gram sequence containing one or more words, rather than directly generating a complete code statement. Whether it is generating one or more words according to the operation instructions, or using the previous target word-gram sequence and the previous prior word-gram sequence to splice the current prior word-gram sequence, the correctness of the prior word-gram sequence is greatly improved.

[0083] Continue to refer Figure 2B , which shows a process 220 of another embodiment of the method for inferring code statements according to the present disclosure. The method for inferring code statements includes the following steps: Step 221 : In response to not recognizing an end marker from the current target word-gram sequence, the current target word-gram sequence and the current prior word-gram sequence are combined into a next prior word-gram sequence.

[0084] Based on the current target word sequence determined in the aforementioned step 214, after identifying whether it contains an end marker, if a special word containing an end marker cannot be identified from the current target word sequence, it can be considered that the current target word sequence is not the last word sequence in the complete code statement, and the next target word sequence still needs to be further generated.

[0085] In this case, the current target word sequence can be spliced ​​after the current prior word sequence, and the spliced ​​word sequence can be used as the next prior word sequence and the current prior word sequence in the next round of code statement generation.

[0086] At the same time, after generating the current prior word-unit sequence of the next round, the hidden state of the current prior word-unit sequence is further generated.

[0087] Among them, the hidden state of the next round is used to represent the LLM model's semantic abstract information contained in the current prior word sequence, including, for example, semantic context information, grammatical structure and / or DSL domain knowledge, etc., and can still be represented by a tensor in the form of a multi-dimensional matrix.

[0088] Step 222 : Perform speculative decoding on the next prior word-gram sequence to generate a plurality of candidate word-gram sequences corresponding to the next prior word-gram sequence.

[0089] Based on the current prior word sequence of the next round determined in the aforementioned step 221, when performing the next round of code statement generation process, speculative decoding can be performed again in the manner of the aforementioned step 211 to generate multiple candidate word sequences for the next round.

[0090] Step 223 : Determine a next target word-gram sequence that matches the next prior word-gram sequence from the multiple candidate word-gram sequences corresponding to the next prior word-gram sequence.

[0091] Based on the multiple candidate word-gram sequences determined in the aforementioned step 223 , it may be verified whether each candidate word-gram sequence is valid.

[0092] Furthermore, after verifying that all candidate word-gram sequences are valid, the one with the highest degree of match with the current prior word-gram sequence of the next round can be selected from the valid candidate word-gram sequences as the target word-gram sequence of the next round, that is, the next target word-gram sequence.

[0093] If any invalid candidate word-gram sequence is verified, it can be corrected to a valid candidate word-gram sequence according to the aforementioned step 213 .

[0094] Step 224 , in response to identifying an end marker from the next target word-gram sequence, composing the next prior word-gram sequence and the next target word-gram sequence into a code statement.

[0095] After determining the target word sequence for the next round in the aforementioned step 223, it is possible to identify whether a special word containing an end marker is included therein. If a special word containing an end marker is identified, the target word sequence for the next round is concatenated after the prior word sequence for the next round to form a complete code statement.

[0096] from Figure 2B It can be seen that with Figure 2A In contrast, the code statement inference method of this embodiment is based on the process of re-generating the next round of target word sequence when the current target word sequence and the current prior word sequence cannot be spliced ​​together to form a complete code statement. In other words, this embodiment gradually generates each word sequence in the complete code statement and verifies its validity during the generation process of each word sequence. Compared with directly generating a complete code statement, this embodiment can achieve, including but not limited to, the following technical effects: The process of verifying validity is only for a single word sequence, that is, each candidate word sequence is verified. The verification speed is significantly improved compared with verifying the complete code statement, which greatly improves the speed of the inferred constraint decoding system to generate complete code statements.

[0097] Since each round only generates a single word sequence in the complete code statement, the accuracy of generating a single word sequence can be significantly improved compared to directly generating the complete code statement. Moreover, since each word sequence has been verified for validity, the error rate of the complete code statement formed by splicing them together is significantly reduced.

[0098] Continue to refer Figure 3A , which shows a decomposition process 300 of an embodiment of step 213 of the present disclosure. The decomposition process 300 includes the following steps: Step 301 : Determine the invalid word-grams corresponding to the respective grammatical errors from each invalid candidate word-gram sequence.

[0099] In this step, based on the decoding log created in the aforementioned step 211 and the word index value of the invalid word recorded in step 212, the fallback controller can perform precise dynamic tracking, that is, accurately locate the invalid word and determine the error type of the invalid word.

[0100] Step 302: Delete each invalid word and all words connected after each invalid word to obtain a modified word sequence.

[0101] Based on the invalid word determined in the aforementioned step 301 and its corresponding error type, the fallback controller can delete the invalid word and simultaneously delete other words connected to the invalid word, so that the invalid candidate word sequence falls back to the most recent valid state in history, thereby completing the fallback operation and obtaining a corrected word sequence.

[0102] In some specific implementations, taking the candidate word-gram sequence "META_TYPE IS LOCATION WITH" as an example, if the word "LOCATION" therein has a grammatical error, when performing a rollback operation, multiple consecutively spliced ​​word-grams "LOCATION WITH" can be deleted, and only two word-grams "META_TYPE IS" are retained as the most recent valid state in the history of the candidate word-gram sequence, and "META_TYPE IS" is used as the corrected word-gram sequence.

[0103] Step 303: perform sentence prediction on each modified word-gram sequence to generate a valid candidate word-gram sequence.

[0104] Based on the modified word-gram sequence determined in the aforementioned step 302 , it can be input into the LLM model for sentence prediction, thereby generating another valid candidate word-gram sequence.

[0105] Specifically, the process of correcting the invalid candidate word sequence in step 303, i.e., the fallback operation and sentence prediction, may include the following steps: Figure 3B Steps 3031 to 3033 are shown.

[0106] Based on this, the decomposition process 300 of step 213 in this embodiment can remove grammatical errors caused by invalid words while retaining correct words by performing a rollback operation on invalid words in the invalid candidate word sequence. Compared with deleting the entire candidate word sequence, it avoids excessive waste of computing resources.

[0107] Continue to refer Figure 3B, which shows a decomposition process 3030 of an embodiment of step 303 of the present disclosure. The decomposition process 3030 includes the following steps: Step 3031 , perform sentence prediction on each modified word-gram sequence to determine a corresponding word-gram probability distribution, where the word-gram probability distribution represents the probability of selecting each word-gram after each modified word-gram sequence.

[0108] Based on the modified word sequence determined in the aforementioned step 302, after it is input into the LLM model, the LLM model can first generate a word probability distribution corresponding to the modified word sequence during the sentence prediction process of the modified word sequence, so that another corresponding candidate word sequence can be determined according to the word probability distribution in the following steps.

[0109] Among them, the word-gram probability distribution specifically represents the probability of selecting each word-gram for each available word-gram when other words are spliced ​​after the modified word-gram sequence. This probability represents the probability that the word-gram is the correct option, or represents the matching rate between the word-gram and the modified word-gram sequence.

[0110] Step 3032: Adjust the word-gram probability distribution corresponding to each modified word-gram sequence.

[0111] Based on the word-gram probability distribution determined in the aforementioned step 3031 and the word-gram index value of the invalid word-gram recorded in the aforementioned step 212, the probability of the invalid word-gram in the word-gram probability distribution in the invalid candidate word-gram sequence can be determined.

[0112] Based on this, the probability corresponding to the invalid word can be adjusted.

[0113] In some optional implementations, the probability corresponding to the invalid word can be adjusted to 0, so that the LLM model no longer selects the invalid word as the next spliced ​​word when predicting a sentence for the modified word sequence, thereby avoiding the same error.

[0114] In other optional implementations, the probability corresponding to the invalid word can also be reduced to a preset probability value, so that the possibility of the LLM model selecting the invalid word when predicting a sentence for the modified word sequence is greatly reduced. Since the probability corresponding to the invalid word is not 0, the global integrity of the LLM model when predicting a sentence can be guaranteed while reducing the probability of selecting the invalid word.

[0115] Step 3033 , using the adjusted probability distribution of each word-gram, determine the word-grams connected to the corresponding modified word-gram sequence to obtain a valid candidate word-gram sequence.

[0116] Based on the adjustment of the word-gram probability distribution in the aforementioned step 3032 , the LLM model has a significantly higher probability of obtaining another valid candidate word-gram sequence when using the adjusted word-gram probability distribution to perform sentence prediction on the modified word-gram sequence.

[0117] In some optional implementations, since the other candidate word-gram sequence predicted at this time is obtained based on the adjusted word-gram probability distribution, it can be directly used as a valid candidate word-gram sequence to improve the overall operating efficiency of the inference constraint decoding system.

[0118] In some other optional implementations, another predicted candidate word-gram sequence may be input into the parsing verifier again to verify the other candidate word-gram sequence to ensure that all candidate word-gram sequences are valid.

[0119] In this embodiment, if another candidate word-gram sequence is still invalid after verification, it can be revised again, that is, a fallback operation and sentence prediction are performed on it, and after the revision is completed, it is verified whether the candidate word-gram sequence obtained again is valid.

[0120] During this iterative correction and verification process, the recursion depth can be pre-set to prevent the inferred constraint decoding system from falling into too many iterative correction and verification processes, thereby wasting computing resources, or to prevent it from falling into unlimited iterative correction and verification.

[0121] Specifically, the recursion depth may be set to, for example, for an invalid candidate word-unit sequence that has not been subjected to a backoff operation, the number of times the backoff operation is performed does not exceed, for example, 20 times.

[0122] Accordingly, if a valid selected word-gram sequence cannot be obtained after the invalid candidate word-gram sequence has been backed off 20 times and 20 different candidate word-gram sequences have been generated, the invalid candidate word-gram sequence can be deleted, thereby ensuring the operating efficiency of the inference constraint decoding system.

[0123] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a keyword extraction device. The device embodiment corresponds to the method embodiment shown in Figure 2, and the device can be specifically applied to various electronic devices.

[0124] like Figure 4As shown, the keyword extraction device 400 of this embodiment includes: a speculative decoding model response module 401, a parsing and validating module 402, a correction module 403, and a large language model response module 404. The speculative decoding model response module 401 is configured to, in response to obtaining a current prior word-gram sequence, perform speculative decoding on the current prior word-gram sequence to generate multiple candidate word-gram sequences for connecting the current prior word-gram sequence; the parsing and validating module 402 is configured to verify whether each candidate word-gram sequence is valid; the correction module 403 is configured to, in response to determining that any candidate word-gram sequence is invalid, correct the invalid candidate word-gram sequence to a valid candidate word-gram sequence; the large language model response module 404 is configured to determine a current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence; the speculative decoding model response module 401 is further configured to, in response to identifying an end marker from the current target word-gram sequence, combine the current prior word-gram sequence and the current target word-gram sequence into a code statement.

[0125] In this embodiment, the specific processing of the inferred decoding model response module 401, the parsing verifier response module 402, the correction module 403 and the large language model response module 404 and the technical effects thereof can be referred to respectively. Figure 2A The relevant descriptions of step 211, step 212, step 213, step 214 and step 215 in the corresponding embodiment are not repeated here.

[0126] In some optional implementations, the parsing validator response module 402 is further configured to: Use the preset grammatical rules to verify whether each candidate word sequence has grammatical errors; A candidate word-gram sequence having at least one grammatical error is determined as an invalid candidate word-gram sequence, and a candidate word-gram sequence having no grammatical error is determined as a valid candidate word-gram sequence.

[0127] In some optional embodiments, each candidate word-gram sequence includes at least one word-gram connected in a predetermined order; Accordingly, the correction module 403 is further configured to: Determine the invalid word units corresponding to the respective grammatical errors from each invalid candidate word unit sequence; Delete each invalid word and all words connected to each invalid word to obtain a corrected word sequence; Sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence.

[0128] In some optional implementations, sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence, including: performing sentence prediction on each modified word-gram sequence to determine a corresponding word-gram probability distribution, wherein the word-gram probability distribution represents the probability of selecting each word-gram after each modified word-gram sequence; Adjusting the word unit probability distribution corresponding to each modified word unit sequence; The word units connected to the corresponding modified word unit sequence are determined using the adjusted probability distribution of each word unit to obtain a valid candidate word unit sequence.

[0129] In some optional implementations, adjusting the word-gram probability distribution corresponding to each modified word-gram sequence includes: Determining the probability of selecting the corresponding invalid word from the word probability distribution corresponding to each modified word sequence; The probability of selecting the corresponding invalid word is adjusted to obtain an adjusted word probability distribution.

[0130] In some optional implementations, the code statement inference device further includes: The iteration module 405 is configured to, after determining a current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence, perform the following steps: In response to not recognizing an end marker from the current target word-gram sequence, combining the current target word-gram sequence and the current prior word-gram sequence into a next prior word-gram sequence; Performing speculative decoding on the next prior word-gram sequence to generate multiple candidate word-gram sequences corresponding to the next prior word-gram sequence; Determining a next target word-gram sequence that matches the next prior word-gram sequence from a plurality of candidate word-gram sequences corresponding to the next prior word-gram sequence; In response to identifying an end marker from the next target word-gram sequence, the next prior word-gram sequence and the next target word-gram sequence are combined into a code statement.

[0131] In some optional implementations, the code statement inference device further includes: The initialization module 406 is configured to perform the following steps before responding to obtaining the current prior word-gram sequence: In response to receiving the operation instruction, generating a corresponding initial a priori word-gram sequence, wherein the initial a priori word-gram sequence includes at least one word-gram in the code statement; The initial prior word-gram sequence is determined as the current prior word-gram sequence. It should be noted that the implementation details and technical effects of each unit in the keyword extraction device provided by the embodiment of the present disclosure can be referred to the description of other embodiments in the present disclosure, and will not be repeated here.

[0132] Reference below Figure 5 , which shows a schematic structural diagram of a computer system 500 suitable for implementing the electronic device of the present disclosure. Figure 5 The computer system 500 shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present disclosure.

[0133] like Figure 5 As shown, computer system 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of computer system 500. Processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0134] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the computer system 500 to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The computer system 500 of the electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0135] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0136] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0137] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0138] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device implements the code statement inference method shown in the embodiment and its optional implementation manner as shown in Figure 2, and / or the code statement inference method shown in the embodiment and its optional implementation manner as shown in Figure 3.

[0139] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit or module does not, in some cases, limit the unit itself. For example, the speculative decoding model response module may also be described as "a module that, in response to obtaining a current prior word sequence, performs speculative decoding on the current prior word sequence and generates multiple candidate word sequences for connecting the current prior word sequence."

[0142] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A method for inferring code statements, comprising: In response to obtaining the current prior word-gram sequence, performing speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence; Verify that each candidate word sequence is valid; In response to determining that any candidate word-gram sequence is invalid, revising the invalid candidate word-gram sequence to a valid candidate word-gram sequence; Determine a current target word sequence that matches the current prior word sequence from each valid candidate word sequence; In response to identifying an end marker from the current target word-gram sequence, the current prior word-gram sequence and the current target word-gram sequence are combined into a code statement.

2. The method according to claim 1, wherein Verifying whether each candidate word-meta sequence is valid includes: Use the preset grammatical rules to verify whether each candidate word sequence has grammatical errors; A candidate word-gram sequence having at least one grammatical error is determined as an invalid candidate word-gram sequence, and a candidate word-gram sequence having no grammatical error is determined as a valid candidate word-gram sequence.

3. The method according to claim 2, wherein: Each candidate word-gram sequence includes at least one word-gram connected in a predetermined order; The step of correcting an invalid candidate word-gram sequence into a valid candidate word-gram sequence includes: Determine the invalid word units corresponding to the respective grammatical errors from each invalid candidate word unit sequence; Delete each invalid word and all words connected to each invalid word to obtain a corrected word sequence; Sentence prediction is performed on each modified word-gram sequence to generate a valid candidate word-gram sequence.

4. The method according to claim 3, wherein: The step of performing sentence prediction on each modified word-gram sequence to generate a valid candidate word-gram sequence includes: performing sentence prediction on each modified word-gram sequence to determine a corresponding word-gram probability distribution, wherein the word-gram probability distribution represents the probability of selecting each word-gram after each modified word-gram sequence; Adjusting the word unit probability distribution corresponding to each modified word unit sequence; The word units connected to the corresponding modified word unit sequence are determined using the adjusted probability distribution of each word unit to obtain a valid candidate word unit sequence.

5. The method according to claim 4, wherein The adjusting of the word unit probability distribution corresponding to each modified word unit sequence includes: Determining the probability of selecting the corresponding invalid word from the word probability distribution corresponding to each modified word sequence; The probability of selecting the corresponding invalid word is adjusted to obtain an adjusted word probability distribution.

6. The method according to claim 1, wherein After determining a current target word-gram sequence that matches the current prior word-gram sequence from each valid candidate word-gram sequence, the method further includes: In response to not recognizing an end marker from the current target word-gram sequence, combining the current target word-gram sequence and the current prior word-gram sequence into a next prior word-gram sequence; Performing speculative decoding on the next prior word-gram sequence to generate multiple candidate word-gram sequences corresponding to the next prior word-gram sequence; Determining a next target word-gram sequence that matches the next prior word-gram sequence from a plurality of candidate word-gram sequences corresponding to the next prior word-gram sequence; In response to identifying an end marker from the next target word-gram sequence, the next prior word-gram sequence and the next target word-gram sequence are combined into a code statement.

7. The method according to claim 1, wherein The code statement is used to respond to the received operation instruction; Before responding to obtaining the current prior word-gram sequence, the method further includes: In response to receiving the operation instruction, generating a corresponding initial a priori word-gram sequence, wherein the initial a priori word-gram sequence includes at least one word-gram in the code statement; The initial a priori word-gram sequence is determined as a current a priori word-gram sequence.

8. A device for inferring code statements, comprising: a speculative decoding model response module configured to, in response to obtaining the current prior word-gram sequence, perform speculative decoding on the current prior word-gram sequence to generate a plurality of candidate word-gram sequences for connecting the current prior word-gram sequence; a parsing validator response module configured to verify whether each candidate word-meta sequence is valid; a correction module configured to, in response to determining that any candidate word-gram sequence is invalid, correct the invalid candidate word-gram sequence into a valid candidate word-gram sequence; The large language model response module is configured to determine a current target word sequence that matches the current prior word sequence from each valid candidate word sequence; The inferred decoding model response module is further configured to, in response to identifying an end marker from the current target word sequence, combine the current prior word sequence and the current target word sequence into a code statement.

9. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the code statement speculation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by one or more processors, the computer program implements the method for inferring code statements according to any one of claims 1 to 7.

11. A computer program product comprising computer program instructions, wherein: When the computer program instructions are executed on a computer, the computer is caused to execute the method for inferring the code statements according to any one of claims 1 to 7.

Images