Code generation method and device based on token occurrence probability adjustment and computer equipment

Abstract

The invention provides a token occurrence probability adjustment-based code generation method and apparatus, and a computer device. The method comprises the steps of screening out a plurality of candidate tokens meeting constraints from a preset token table; calculating the adjusted occurrence probability of the second target token according to a plurality of original occurrence probabilities corresponding to the plurality of candidate tokens one by one and the original occurrence probability of the second target token; adjusting the original occurrence probability of the second target token into the adjusted occurrence probability; performing normalization adjustment on the original occurrence probabilities of the plurality of candidate tokens to obtain the adjusted occurrence probabilities of the plurality of candidate tokens; calculating the occurrence probability of a plurality of second candidate codes according to the adjusted occurrence probability of the plurality of candidate tokens and the adjusted occurrence probability of the second target token; and generating a target code according to the occurrence probability of the plurality of second candidate codes. Probability distribution of the language model cannot be distorted, and code generation accuracy of the language model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a code generation method, apparatus, and computer equipment based on token occurrence probability adjustment. Background Art

[0002] The code generated by the language model is usually composed of multiple tokens, each of which refers to a basic unit in the code, such as a string. The code generated by the language model may not meet the preset constraints (for example, it does not meet the grammatical requirements). For this reason, the prior art provides a constraint decoding method to verify whether the tokens in the code to be generated meet the preset constraints: when each token in the code is about to be generated, the tokens in the token table that may cause the preset constraints to be not met are removed, and the probability of occurrence of the remaining tokens is adjusted so that the sum of the probability of occurrence of the remaining tokens is 1.

[0003] The above-mentioned probability adjustment method only refers to the remaining tokens at the same level as the token that does not meet the preset constraints, without considering the probability of occurrence of the relevant tokens above, thereby distorting the probability distribution of the language model and causing low code generation accuracy of the language model. Summary of the Invention

[0004] In view of this, the present application proposes a code generation method, apparatus, and computer device based on token occurrence probability adjustment to solve the problem in related technologies of low accuracy of code generation of language models due to distorted probability distribution of language models.

[0005] The first embodiment of the present application proposes a code generation method based on token occurrence probability adjustment, the method comprising:

[0006] In response to an instruction to generate a first target token of a first candidate code, screening a plurality of candidate tokens that meet preset constraints from a preset token table; the first candidate code includes the plurality of generated tokens;

[0007] Calculating an adjusted probability of occurrence of the second target token based on the multiple original probability of occurrence corresponding to the multiple candidate tokens and the original probability of occurrence of the second target token; the second target token refers to a token with an original probability of occurrence less than 1 among the multiple generated tokens;

[0008] Adjusting the original occurrence probability of the second target token to the adjusted occurrence probability;

[0009] Normalizing and adjusting the multiple original occurrence probabilities corresponding to the multiple candidate tokens one by one to obtain adjusted occurrence probabilities of the multiple candidate tokens;

[0010] Combining the multiple candidate tokens with the first candidate code respectively to obtain multiple second candidate codes;

[0011] Calculating the occurrence probabilities of the plurality of second candidate codes according to the adjusted occurrence probabilities of the plurality of candidate tokens and the adjusted occurrence probability of the second target token;

[0012] A target code is generated according to the occurrence probabilities of the plurality of second candidate codes.

[0013] In the embodiment of the present application, while adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, the method further includes:

[0014] The original occurrence probabilities of all tokens at the same level as the second target token are adjusted according to the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

[0015] In an embodiment of the present application, calculating the adjusted occurrence probability of the second target token according to the multiple original occurrence probabilities corresponding one-to-one to the multiple candidate tokens and the original occurrence probability of the second target token includes:

[0016] For any candidate token among the multiple candidate tokens, calculate the overall occurrence probability corresponding to the candidate token according to the original occurrence probability of the candidate token and the original occurrence probability of the second target token;

[0017] Summing the overall occurrence probabilities of the multiple candidate tokens to obtain a probability summation result;

[0018] The probability summation result and the occurrence probabilities of all tokens at the same level as the second target token are normalized and adjusted to obtain an adjusted occurrence probability of the second target token.

[0019] In this embodiment of the present application, the first target token is the tth token in the first candidate code;

[0020] Filter out multiple candidate tokens that meet the preset constraints from the preset token table, including:

[0021] For any token among all tokens in the preset token table, determine whether the token meets the preset constraint condition based on the token and the first t-1 tokens in the first candidate code;

[0022] If the token meets the preset constraints, the token is used as a candidate token.

[0023] In an embodiment of the present application, after responding to an instruction to generate a first target token of a first candidate code, the method further includes:

[0024] A plurality of tokens that do not satisfy the preset constraint conditions are screened out from the preset token table, and the occurrence probabilities of the plurality of tokens that do not satisfy the preset constraint conditions are set to zero.

[0025] In an embodiment of the present application, the method further includes:

[0026] The generation probability of the target code is calculated according to the following steps:

[0027] When t=1, the probability of generating the first token is calculated according to the preset constraint conditions; the target code includes t tokens;

[0028] When t>1, the probability of generating the tth token is calculated based on the previous t-1 tokens and the preset constraints;

[0029] The generation probability of the target code is calculated based on the t generation probabilities corresponding to the t tokens.

[0030] In the embodiments of this application,

[0031]

[0032] Where S represents the target code containing t tokens, C represents the preset constraints, P(S|C) represents the probability of generating the target code based on t tokens and the preset constraints, t represents the sequence number of the token in the target code, S t represents the tth token, P(s t |c,s ＜t ) indicates that the probability of generating the tth token is calculated based on the first t-1 tokens and the preset constraints.

[0033] In this embodiment of the present application, the probability of generating the tth token is calculated based on the first t-1 tokens and the preset constraints, including:

[0034] Calculate the first probability of generating the tth token based on the previous t-1 tokens;

[0035] Calculate the second probability of satisfying the preset constraint condition based on the first t-1 tokens and the t-th token;

[0036] Determine the probability of generating the tth token based on the first probability and the second probability.

[0037] An embodiment of the second aspect of the present application provides a code generation device based on token occurrence probability adjustment, including:

[0038] A token screening module, configured to, in response to an instruction to generate a first target token of a first candidate code, screen out a plurality of candidate tokens that meet preset constraints from a preset token table; the first candidate code includes a plurality of generated tokens;

[0039] an occurrence probability calculation module, configured to calculate an adjusted occurrence probability of the second target token based on the multiple original occurrence probabilities corresponding one-to-one to the multiple candidate tokens and the original occurrence probability of the second target token; the second target token refers to a token with an original occurrence probability less than 1 among the multiple generated tokens;

[0040] A first probability adjustment module, configured to adjust the original occurrence probability of the second target token to the adjusted occurrence probability;

[0041] A second probability adjustment module is configured to normalize and adjust the multiple original occurrence probabilities corresponding to the multiple candidate tokens to obtain adjusted occurrence probabilities of the multiple candidate tokens;

[0042] A second candidate code generating module, configured to combine the plurality of candidate tokens with the first candidate code respectively to obtain a plurality of second candidate codes;

[0043] a code occurrence probability calculation module, configured to calculate the occurrence probabilities of the plurality of second candidate codes based on the adjusted occurrence probabilities of the plurality of candidate tokens and the adjusted occurrence probability of the second target token;

[0044] The target code generating module is configured to generate a target code according to the occurrence probabilities of the plurality of second candidate codes.

[0045] An embodiment of the third aspect of the present application provides a computer device, which includes a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the code generation method based on token occurrence probability adjustment described in the first aspect above by executing the computer instructions.

[0046] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to enable a computer to execute the code generation method based on token occurrence probability adjustment described in the first aspect above.

[0047] Additional aspects and advantages of the present application will be given in part in the description below and in part will become apparent from the description below or learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. Throughout the accompanying drawings, the same reference numerals are used to denote the same components.

[0049] In the attached figure:

[0050] Figure 1 A schematic diagram of the above code of the completed code provided in one embodiment of the present application is shown;

[0051] Figure 2 A schematic diagram showing relevant information of candidate codes provided by an embodiment of the present application;

[0052] Figure 3 A schematic diagram showing the probability of occurrence of candidate codes after token occurrence probability adjustment provided by an embodiment of the present application is shown;

[0053] Figure 4 A schematic diagram illustrating a method for adjusting the probability of occurrence in the prior art provided by an embodiment of the present application is shown;

[0054] Figure 5 A flowchart of a code generation method based on token occurrence probability adjustment provided by an embodiment of the present application is shown;

[0055] Figure 6 A flowchart of a code generation method based on token occurrence probability adjustment provided by an embodiment of the present application is shown;

[0056] Figures 7 to 10A flowchart of another code generation method based on token occurrence probability adjustment provided by an embodiment of the present application is shown;

[0057] Figure 11 A schematic diagram of the structure of a code generation device based on token occurrence probability adjustment provided by an embodiment of the present application is shown;

[0058] Figure 12 A schematic diagram of the structure of a computer device provided in one embodiment of the present application is shown;

[0059] Figure 13 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0060] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0061] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0062] pass Figure 1 、 Figure 2 、 Figure 3 and Figure 4 , the technical scenarios and technical problems involved in the embodiments of this application are explained:

[0063] Figure 1 The above code shows the completed code, which defines a function "matrix_rank" that accepts a parameter x of type torch.Tensor and calculates the rank of this tensor.

[0064] Figure 2 This shows the language model's generation of multiple candidate codes based on the preceding code, the probability of each candidate code, and whether each candidate code satisfies the preset constraints. For example, the probability of the candidate code "matrix_rank" is 57.44%, which does not meet the preset constraints; the probability of the candidate code "linalg.matrix_rank" is 19.21%, which meets the preset constraints; and the probability of the candidate code "matrix_power" is 0.20%, which meets the preset constraints.

[0065] In order to solve the problem that the code generated by the language model may not meet the preset constraints (for example, it does not meet the grammatical requirements), a constraint decoding method is proposed in the prior art to remove tokens that do not meet the preset constraints, such as Figure 2 The "rank" in the "matrix_rank" does not meet the preset constraints, and the candidate code "matrix_rank" also does not meet the requirements of the preset constraints. Therefore, the candidate code "matrix_rank" is deleted and the occurrence probability of the remaining candidate codes is adjusted. For example, Figure 3 As shown in the figure, the probability of the candidate code "matrix_power" is adjusted from 0.20% to 57.70%, while the probability of the candidate code "linalg.matrix_rank" is adjusted from 19.21% to 20.12%. Obviously, this adjustment method is not reasonable. The candidate codes with a low probability of occurrence have a higher probability after adjustment, and the candidate codes with a high probability of occurrence have a lower probability after adjustment, thus distorting the probability distribution of the language model and causing the problem of low code generation accuracy of the language model.

[0066] The reason why the above-mentioned prior art has the problem that "the candidate code with a low probability of occurrence has a higher probability of occurrence after adjustment, and the candidate code with a high probability of occurrence has a lower probability of occurrence after adjustment" is that the probability adjustment method in the prior art is only for the remaining tokens at the same level as the token that does not meet the preset constraints, and does not adjust the probability of occurrence of the above-mentioned related tokens. Figure 4 As shown:

[0067] Figure 4 The red-marked "rank" indicates that the token does not meet the preset constraints, and the other green-marked tokens indicate that the tokens meet the preset constraints.

[0068] In the existing technology, only the occurrence probabilities of the remaining tokens at the same level as the tokens that do not meet the preset constraints are adjusted. For example, the occurrence probability of "power" and "exp" marked in red at the same level as "rank" is set to zero, and the occurrence probabilities of multiple tokens at the same level except "rank" marked in red are normalized, that is, the occurrence probability of "power" is adjusted from 0.35% to 97.07%, and the occurrence probability of "exp" is adjusted from 0.01% to 2.81%.

[0069] After adjusting the probability of occurrence using the above method, the adjusted probability of the candidate code "matrix_power" is: 57.96% * 97.07% ≈ 57.70%; while the adjusted probability of the candidate code "matrix_power" is: 22.51% * 89.49% * 99.87% ≈ 20.12%. This distorts the probability distribution of the language model, resulting in low code generation accuracy.

[0070] In response to the above technical problems, an embodiment of the present application provides a code generation method based on token occurrence probability adjustment, which can adjust the occurrence probability of remaining tokens at the same level as the tokens that do not meet the preset constraints while considering the occurrence probability of the above-mentioned related tokens, thereby ensuring that the probability distribution of the language model will not be distorted, thereby improving the code generation accuracy of the language model.

[0071] According to an embodiment of the present application, an embodiment of a code generation method based on token occurrence probability adjustment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0072] In this embodiment, a code generation method based on token occurrence probability adjustment is provided. Figure 5 is a flowchart of a code generation method based on token occurrence probability adjustment according to an embodiment of the present application, such as Figure 5 As shown, the process includes the following steps:

[0073] Step S101 , in response to an instruction to generate a first target token of a first candidate code, a plurality of candidate tokens that meet preset constraints are screened out from a preset token table.

[0074] In the embodiment of the present application, the first candidate code includes multiple generated tokens, such as Figure 6 The "matrix" and "_" in the code are replaced by "_". The first target token refers to the token to be generated. For example, if t-1 tokens have been generated in the first candidate code, the first target token is the tth token to be generated.

[0075] In the embodiment of the present application, the preset token table contains all tokens. The preset constraints are not specifically limited in this application and can be set according to actual conditions. For example, if a token is generated after another token, it does not meet the grammatical requirements.

[0076] In some specific embodiments, multiple candidate tokens that meet preset constraints can be screened from the preset token table in the following manner:

[0077] When the first target token is the tth token in the first candidate code, for any token among all the tokens in the preset token table, determine whether the token meets the preset constraints based on the token and the first t-1 tokens in the first candidate code; if the token meets the preset constraints, the token is used as a candidate token.

[0078] Step S102 : calculating the adjusted occurrence probability of the second target token according to the multiple original occurrence probabilities corresponding one-to-one to the multiple candidate tokens and the original occurrence probability of the second target token.

[0079] In the embodiment of the present application, the multiple original occurrence probabilities corresponding to the multiple candidate tokens can be understood as Figure 6 The second target token refers to the token whose original occurrence probability is less than 1 among the multiple generated tokens, for example Figure 6 The matrix in .

[0080] In some specific embodiments, the above step S102 includes steps S1021 to S1023:

[0081] Step S1021 : For any candidate token among the multiple candidate tokens, calculate the overall occurrence probability corresponding to the candidate token according to the original occurrence probability of the candidate token and the original occurrence probability of the second target token.

[0082] In the embodiments of this application, Figure 6 and Figure 7 As shown:

[0083] The overall occurrence probability corresponding to the candidate token "power" is calculated by the original occurrence probability of 0.35% of the candidate token "power" and the original occurrence probability of 60.46% of the second target token, that is, 60.46% * 0.35% of "matrix_power" ≈ 0.21%; similarly, the overall occurrence probability corresponding to the candidate token "exp" is calculated by the original occurrence probability of 0.01% of the candidate token "exp" and the original occurrence probability of 60.46% of the second target token, that is, 60.46% * 0.01% of "matrix_exp" ≈ 0.01%;

[0084] In some specific embodiments, the method further includes: screening out a plurality of tokens that do not satisfy preset constraints from a preset token table, and setting the occurrence probabilities of the plurality of tokens that do not satisfy the preset constraints to zero.

[0085] In the embodiments of this application, Figure 6 、 7 As shown: the "rank" of tokens that do not meet the preset constraints is set to zero, that is, 60.21% → 0.00%.

[0086] Step S1022: summing up the overall occurrence probabilities of the plurality of candidate tokens to obtain a probability summation result.

[0087] In the embodiment of the present application, the overall occurrence probabilities of multiple candidate tokens are summed, for example Figure 7 In Chinese: 0.21% + 0.01% = 0.22%, thus obtaining the probability sum result, which is 0.22%.

[0088] Step S1023 , normalizing and adjusting the probability summation result and the occurrence probabilities of all tokens at the same level as the second target token to obtain the adjusted occurrence probability of the second target token.

[0089] In the embodiments of this application, Figure 7 As shown, the occurrence probabilities of "matrix_power," "matrix_exp," ..., and "l" at the same level are normalized, i.e., the occurrence probability of "matrix_power" is adjusted from 0.21% to 0.53%, the occurrence probability of "matrix_exp" is adjusted from 0.01% to 0.02%, and the occurrence probability of "l" is adjusted from 22.51% to 56.58%. This makes the sum of the adjusted occurrence probabilities of "matrix_power," "matrix_exp," ..., and "l" at the same level 1. The adjusted occurrence probability of the second target token "matrix" is the sum of the occurrence probabilities of "matrix_power" and "matrix_exp," i.e., 0.53% + 0.02% = 0.55%.

[0090] On this basis, the adjusted occurrence probability of the candidate code “linalg.matrix_rank” is: 56.58%*89.49%*99.87%≈50.57%.

[0091] Step S103: Adjust the original occurrence probability of the second target token to the adjusted occurrence probability.

[0092] In some specific embodiments, while adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, the method further includes:

[0093] The original occurrence probabilities of all tokens at the same level as the second target token are adjusted according to the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

[0094] In the embodiments of this application, Figure 9 As shown in the figure, the probability difference can be understood as 60.46% - 0.55% = 59.91%. The probability difference is proportionally divided among all tokens at the same level as the second target token, namely "..." and "l", so that the probability of the token "l" increases from 22.51% to 56.58%.

[0095] In the above step S103, Figure 8 As shown in the figure, whether to accept the second target token "matrix" can be determined by rejecting the sampling. When "matrix" is sampled with a sampling probability of 60.46% and can be hit, and on this basis, "matrix" can be sampled and hit again with a sampling probability of 0.55% / 60.46%=0.91%, "matrix" can be used as a candidate token in the target code.

[0096] Step S104 , normalizing and adjusting the multiple original occurrence probabilities corresponding to the multiple candidate tokens one by one to obtain adjusted occurrence probabilities of the multiple candidate tokens.

[0097] In the embodiments of this application, Figure 9 As shown in the figure, the original occurrence probabilities of multiple candidate tokens such as "power" and "exp" are normalized, that is, the occurrence probability of "power" is adjusted from 0.35% to 97.07%, and the occurrence probability of "exp" is adjusted from 0.01% to 2.81%.

[0098] Step S105: Combine the multiple candidate tokens with the first candidate code respectively to obtain multiple second candidate codes.

[0099] Step S106, calculating the occurrence probabilities of the plurality of second candidate codes according to the adjusted occurrence probabilities of the plurality of candidate tokens and the adjusted occurrence probability of the second target token;

[0100] In step S105-step S106, multiple second candidate codes are obtained by combining Figure 10 As shown, "power" and "matrix_" are combined to obtain "matrix_power", and "exp" and "matrix_" are combined to obtain "matrix_exp". Among them, the probability of occurrence of the second candidate code "matrix_power" is: 0.55%*100%*97.07%=0.53%, and the probability of occurrence of the second candidate code "matrix_exp" is: 0.55%*100%*2.81%=0.015%.

[0101] Step S107: generating a target code according to the occurrence probabilities of the plurality of second candidate codes.

[0102] Through the code generation method based on token occurrence probability adjustment provided in the embodiment of the present application, it can be seen that the probability of occurrence of the candidate code "linalg.matrix_rank" after adjustment is 50.57%, while the probability of occurrence of the candidate code "matrix_power" after adjustment is 0.53%. The comparison of the two candidate codes before and after adjustment is: 0.2% → 0.53%, and 19.21% → 50.57%. Therefore, it can be determined that the technical solution of the embodiment of the present application will not have the problem of "a candidate code with a low probability of occurrence having a higher probability of occurrence after adjustment, and a candidate code with a high probability of occurrence having a lower probability of occurrence after adjustment", thereby ensuring that the probability distribution of the language model will not be distorted, achieving the technical effect of improving the accuracy of code generation of the language model.

[0103] In some specific embodiments, the method further comprises:

[0104] The generation probability of the target code is calculated according to the following steps:

[0105] When t=1, the probability of generating the first token is calculated according to the preset constraint conditions; the target code includes t tokens;

[0106] When t>1, the probability of generating the tth token is calculated based on the previous t-1 tokens and the preset constraints;

[0107] The generation probability of the target code is calculated based on the t generation probabilities corresponding to the t tokens.

[0108] In the embodiments of this application,

[0109]

[0110] Where S represents the target code containing t tokens, C represents the preset constraints, P(S|C) represents the probability of generating the target code based on t tokens and the preset constraints, t represents the sequence number of the token in the target code, S t represents the tth token, P(s t |c,s ＜t ) indicates that the probability of generating the tth token is calculated based on the first t-1 tokens and the preset constraints.

[0111] In some specific embodiments, calculating the probability of generating the tth token based on the first t-1 tokens and the preset constraints includes:

[0112] Calculate the first probability of generating the tth token based on the previous t-1 tokens;

[0113] Calculate the second probability of satisfying the preset constraint condition based on the first t-1 tokens and the t-th token;

[0114] Determine the probability of generating the tth token based on the first probability and the second probability.

[0115] In the embodiments of this application,

[0116] P(s t ||c,S ＜t )∝P LM (S t ||S ＜t )P(c|S t , S ＜t ).

[0117] Among them, P(s t |c,s ＜t ) represents the calculation of the generation probability of the t-th token based on the first t-1 tokens and the preset constraints; P LM (s t |s ＜t ) means that the first probability of generating the t-th token is calculated based on the previous t-1 tokens, P(c|s t , s ＜t ) indicates that the second probability of satisfying the preset constraint condition is calculated based on the previous t-1 tokens and the t-th token.

[0118] In the embodiments of this application,

[0119] We use Q[x] as an estimation of P(c|x).

[0120] We also maintain N[x]sampled from P Q (·|c,x).

[0121]

[0122] Where P(c|x) represents the above P(c|s t , s ＜t ), Q[x] is used to determine the legitimacy of the above code (i.e., whether the first t-1 tokens meet the preset constraints).

[0123] Corresponding to the implementation of the above code generation method based on token occurrence probability adjustment, the embodiment of the present application also provides a code generation device based on token occurrence probability adjustment, which is used to perform the above Figures 1 to 10 The code generation method based on token occurrence probability adjustment described in any of the illustrated embodiments. Figure 11 As shown, the code generation device based on token occurrence probability adjustment includes:

[0124] A token screening module, configured to, in response to an instruction to generate a first target token of a first candidate code, screen out a plurality of candidate tokens that meet preset constraints from a preset token table; the first candidate code includes a plurality of generated tokens;

[0125] an occurrence probability calculation module, configured to calculate an adjusted occurrence probability of the second target token based on the multiple original occurrence probabilities corresponding one-to-one to the multiple candidate tokens and the original occurrence probability of the second target token; the second target token refers to a token with an original occurrence probability less than 1 among the multiple generated tokens;

[0126] A first probability adjustment module, configured to adjust the original occurrence probability of the second target token to the adjusted occurrence probability;

[0127] A second probability adjustment module is configured to normalize and adjust the multiple original occurrence probabilities corresponding to the multiple candidate tokens to obtain adjusted occurrence probabilities of the multiple candidate tokens;

[0128] A second candidate code generating module, configured to combine the plurality of candidate tokens with the first candidate code respectively to obtain a plurality of second candidate codes;

[0129] a code occurrence probability calculation module, configured to calculate the occurrence probabilities of the plurality of second candidate codes based on the adjusted occurrence probabilities of the plurality of candidate tokens and the adjusted occurrence probability of the second target token;

[0130] The target code generating module is configured to generate a target code according to the occurrence probabilities of the plurality of second candidate codes.

[0131] Optionally, the device also includes: a third probability adjustment module, which is used to: while adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, adjust the original occurrence probability of all tokens at the same level as the second target token according to the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

[0132] Optionally, the occurrence probability calculation module is also used to: for any candidate token among the multiple candidate tokens, calculate the overall occurrence probability corresponding to the candidate token based on the original occurrence probability of the candidate token and the original occurrence probability of the second target token; sum up the multiple overall occurrence probabilities of the multiple candidate tokens to obtain a probability summation result; normalize and adjust the probability summation result and the occurrence probabilities of all tokens at the same level as the second target token to obtain the adjusted occurrence probability of the second target token.

[0133] Optionally, the first target token is the tth token in the first candidate code; the token screening module is also used to determine, for any token among all tokens in a preset token table, whether the token meets the preset constraints based on the token and the first t-1 tokens in the first candidate code; if the token meets the preset constraints, the token is used as a candidate token.

[0134] Optionally, the device further includes: a probability zeroing module, configured to filter out a plurality of tokens that do not satisfy preset constraints from a preset token table, and set the occurrence probabilities of the plurality of tokens that do not satisfy the preset constraints to zero.

[0135] The code generation device based on token occurrence probability adjustment provided in the above-mentioned embodiment of the present application and the code generation method based on token occurrence probability adjustment provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0136] The present application also provides a computer device to execute the above-mentioned code generation method based on token occurrence probability adjustment. Figure 12 , which shows a schematic diagram of a computer device provided by some embodiments of the present application. Figure 12 As shown, the computer device 12 includes: a processor 1200, a memory 1201, a bus 1202 and a communication interface 1203, wherein the processor 1200, the communication interface 1203 and the memory 1201 are connected via the bus 1202; the memory 1201 stores a computer program that can be run on the processor 1200, and the processor 1200 executes the aforementioned Figures 1 to 10 A code generation method based on token occurrence probability adjustment is provided in any of the illustrated embodiments.

[0137] The memory 1201 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element and at least one other network element are connected via at least one communication interface 1203 (which may be wired or wireless), and may use the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0138] The bus 1202 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 1201 is used to store programs. The processor 1200 executes the program after receiving the execution instruction. Figures 1 to 10 The code generation method based on token occurrence probability adjustment disclosed in any of the illustrated embodiments may be applied to the processor 1200 or implemented by the processor 1200 .

[0139] The processor 1200 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the processor 1200. The above-mentioned processor 1200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 1201 , and the processor 1200 reads the information in the memory 1201 and completes the steps of the above method in combination with its hardware.

[0140] The computer device provided in the embodiment of the present application and the code generation method based on token occurrence probability adjustment provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by them.

[0141] The present application also provides a computer-readable storage medium corresponding to the code generation method based on token occurrence probability adjustment provided in the above embodiment. Please refer to Figure 13 The computer-readable storage medium shown is a CD 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by the processor, it will execute the code generation method based on token occurrence probability adjustment provided by any of the aforementioned embodiments.

[0142] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0143] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the code generation method based on token occurrence probability adjustment provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0144] It should be noted that:

[0145] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0146] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present application.

[0147] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0148] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A code generation method based on token occurrence probability adjustment, characterized in that: The method comprises: In response to an instruction to generate a first target token of a first candidate code, screening a plurality of candidate tokens that meet preset constraints from a preset token table; the first candidate code includes the plurality of generated tokens; Calculating an adjusted probability of occurrence of the second target token based on the multiple original probability of occurrence corresponding to the multiple candidate tokens and the original probability of occurrence of the second target token; the second target token refers to a token with an original probability of occurrence less than 1 among the multiple generated tokens; Adjusting the original occurrence probability of the second target token to the adjusted occurrence probability; Normalizing and adjusting the multiple original occurrence probabilities corresponding to the multiple candidate tokens one by one to obtain adjusted occurrence probabilities of the multiple candidate tokens; Combining the multiple candidate tokens with the first candidate code respectively to obtain multiple second candidate codes; Calculating the occurrence probabilities of the plurality of second candidate codes according to the adjusted occurrence probabilities of the plurality of candidate tokens and the adjusted occurrence probability of the second target token; A target code is generated according to the occurrence probabilities of the plurality of second candidate codes.

2. The method according to claim 1, characterized in that While adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, the method further includes: The original occurrence probabilities of all tokens at the same level as the second target token are adjusted according to the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

3. The method according to claim 1 or 2, characterized in that Calculating the adjusted occurrence probability of the second target token according to the multiple original occurrence probabilities corresponding one-to-one to the multiple candidate tokens and the original occurrence probability of the second target token includes: For any candidate token among the multiple candidate tokens, calculate the overall occurrence probability corresponding to the candidate token according to the original occurrence probability of the candidate token and the original occurrence probability of the second target token; Summing the overall occurrence probabilities of the multiple candidate tokens to obtain a probability summation result; The probability summation result and the occurrence probabilities of all tokens at the same level as the second target token are normalized and adjusted to obtain an adjusted occurrence probability of the second target token.

4. The method according to claim 1, wherein The first target token is the tth token in the first candidate code; Filter out multiple candidate tokens that meet the preset constraints from the preset token table, including: For any token among all tokens in the preset token table, determine whether the token meets the preset constraint condition based on the token and the first t-1 tokens in the first candidate code; If the token meets the preset constraints, the token is used as a candidate token.

5. The method according to claim 1, wherein After responding to the instruction to generate the first target token of the first candidate code, the method further includes: A plurality of tokens that do not satisfy the preset constraint conditions are screened out from the preset token table, and the occurrence probabilities of the plurality of tokens that do not satisfy the preset constraint conditions are set to zero.

6. The method according to claim 1 or 2, characterized in that The method further comprises: The generation probability of the target code is calculated according to the following steps: When t=1, the probability of generating the first token is calculated according to the preset constraint conditions; the target code includes t tokens; When t>1, the probability of generating the tth token is calculated based on the previous t-1 tokens and the preset constraints; The generation probability of the target code is calculated based on the t generation probabilities corresponding to the t tokens.

7. The method according to claim 1 or 2, characterized in that Where S represents the target code containing t tokens, C represents the preset constraints, P(S|C) represents the probability of generating the target code based on t tokens and the preset constraints, t represents the sequence number of the token in the target code, S t represents the tth token, P(s t |c,s ＜t ) indicates that the probability of generating the tth token is calculated based on the first t-1 tokens and the preset constraints.

8. The method according to claim 7, characterized in that Based on the first t-1 tokens and the preset constraints, the probability of generating the tth token is calculated, including: Calculate the first probability of generating the tth token based on the previous t-1 tokens; Calculate the second probability of satisfying the preset constraint condition based on the first t-1 tokens and the t-th token; Determine the probability of generating the tth token based on the first probability and the second probability.

9. A code generation device based on token occurrence probability adjustment, characterized in that: The device comprises: A token screening module, configured to, in response to an instruction to generate a first target token of a first candidate code, screen out a plurality of candidate tokens that meet preset constraints from a preset token table; the first candidate code includes a plurality of generated tokens; an occurrence probability calculation module, configured to calculate an adjusted occurrence probability of the second target token based on the multiple original occurrence probabilities corresponding one-to-one to the multiple candidate tokens and the original occurrence probability of the second target token; the second target token refers to a token with an original occurrence probability less than 1 among the multiple generated tokens; A first probability adjustment module, configured to adjust the original occurrence probability of the second target token to the adjusted occurrence probability; A second probability adjustment module is configured to normalize and adjust the multiple original occurrence probabilities corresponding to the multiple candidate tokens to obtain adjusted occurrence probabilities of the multiple candidate tokens; A second candidate code generating module, configured to combine the plurality of candidate tokens with the first candidate code respectively to obtain a plurality of second candidate codes; a code occurrence probability calculation module, configured to calculate the occurrence probabilities of the plurality of second candidate codes based on the adjusted occurrence probabilities of the plurality of candidate tokens and the adjusted occurrence probability of the second target token; The target code generating module is configured to generate a target code according to the occurrence probabilities of the plurality of second candidate codes.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the code generation method based on token occurrence probability adjustment according to any one of claims 1 to 8 by executing the computer instructions.

Images