Code generation method and device based on token occurrence probability adjustment, computer device

Abstract

The application provides a code generation method and device based on token appearance probability adjustment, and a computer device. The method comprises the following steps: screening a plurality of candidate tokens satisfying constraints from a preset token table; calculating an adjusted appearance probability of a second target token according to a plurality of original appearance probabilities corresponding to the plurality of candidate tokens and an original appearance probability of the second target token; adjusting the original appearance probability of the second target token to the adjusted appearance probability; performing normalization adjustment on the original appearance probabilities of the plurality of candidate tokens to obtain adjusted appearance probabilities of the plurality of candidate tokens; calculating appearance probabilities of a plurality of second candidate codes according to the adjusted appearance probabilities of the plurality of candidate tokens and the adjusted appearance probability of the second target token; and generating a target code according to the appearance probabilities of the plurality of second candidate codes. The application can ensure that the probability distribution of the language model is not distorted, and the code generation accuracy of the language model is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a code generation method, apparatus, and computer device based on adjusting the probability of token occurrence. Background Technology

[0002] The code generated by a language model typically consists of multiple tokens, each representing a basic unit of code, such as a string. The code generated by the language model may fail to meet preset constraints (e.g., it may not conform to syntactic requirements). To address this, existing technology provides a constraint decoding method that verifies whether the tokens in the code to be generated satisfy the preset constraints: when generating each token in the code, tokens in the token table that may cause non-compliance with the preset constraints are removed, and the probability of occurrence of the remaining tokens is adjusted so that the sum of the probabilities of occurrence of the remaining tokens is 1.

[0003] The above-mentioned method of adjusting the probability of occurrence only refers to the remaining tokens at the same level as the tokens that do not meet the preset constraints, without considering the probability of occurrence of the related tokens mentioned above. This distorts the probability distribution of the language model and causes the language model to generate code with low accuracy. Summary of the Invention

[0004] In view of this, this application proposes a code generation method, apparatus, and computer device based on adjusting the probability of token occurrence, in order to solve the problem of low code generation accuracy of language models caused by distorting the probability distribution of language models in related technologies.

[0005] The first aspect of this application proposes a code generation method based on adjusting the probability of token occurrence, the method comprising:

[0006] In response to the instruction to generate a first target token for a first candidate code, multiple candidate tokens that meet preset constraints are selected from a preset token table; the first candidate code includes multiple generated tokens.

[0007] The adjusted occurrence probability of the second target token is calculated based on the original occurrence probabilities corresponding to the multiple candidate tokens and the original occurrence probability of the second target token; the second target token refers to the token among the multiple generated tokens whose original occurrence probability is less than 1.

[0008] The original occurrence probability of the second target token is adjusted to the adjusted occurrence probability;

[0009] The original occurrence probabilities corresponding to the multiple candidate tokens are normalized and adjusted to obtain the adjusted occurrence probabilities of the multiple candidate tokens.

[0010] The multiple candidate tokens are combined with the first candidate code to obtain multiple second candidate codes;

[0011] The occurrence probability of the multiple second candidate codes is calculated based on the adjusted occurrence probability of the multiple candidate tokens and the adjusted occurrence probability of the second target token;

[0012] The target code is generated based on the occurrence probability of the plurality of second candidate codes.

[0013] In this embodiment of the application, while adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, the method further includes:

[0014] The original occurrence probabilities of all tokens at the same level as the second target token are adjusted based on the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

[0015] In this embodiment of the application, the adjusted occurrence probability of the second target token is calculated based on the original occurrence probabilities corresponding one-to-one with the plurality of candidate tokens and the original occurrence probability of the second target token, including:

[0016] For any candidate token among the plurality of candidate tokens, calculate the overall occurrence probability corresponding to the candidate token based on the original occurrence probability of the candidate token and the original occurrence probability of the second target token;

[0017] The probability summation result is obtained by summing the overall occurrence probabilities of the multiple candidate tokens;

[0018] The probability summation result and the occurrence probabilities of all tokens at the same level as the second target token are normalized and adjusted to obtain the adjusted occurrence probability of the second target token.

[0019] In this embodiment of the application, the first target token is the t-th token in the first candidate code;

[0020] Select multiple candidate tokens from the preset token table that meet the preset constraints, including:

[0021] For any token in the preset token table, determine whether the token satisfies the preset constraint conditions based on the token and the first t-1 tokens in the first candidate code;

[0022] If the token satisfies the preset constraints, then the token is selected as a candidate token.

[0023] In this embodiment of the application, after responding to the instruction to generate the first target token of the first candidate code, the method further includes:

[0024] Select multiple tokens from the preset token table that do not meet the preset constraints, and set the probability of occurrence of the multiple tokens that do not meet the preset constraints to zero.

[0025] In this embodiment of the application, the method further includes:

[0026] The generation probability of the target code is calculated according to the following steps:

[0027] When t=1, calculate the generation probability of the first token according to the preset constraints; the target code includes t tokens;

[0028] When t > 1, calculate the generation probability of the t-th token based on the first t-1 tokens and the preset constraints.

[0029] The generation probability of the target code is calculated based on the t generation probabilities that correspond one-to-one with the t tokens.

[0030] In the embodiments of this application,

[0031]

[0032] Where S represents the target code containing t tokens, C represents the preset constraints, P(S|C) represents the generation probability of the target code calculated based on the t tokens and the preset constraints, and t represents the index of the token in the target code. t Let P(s) represent the t-th token. t |c,s ＜t ) indicates that the generation probability of the t-th token is calculated based on the first t-1 tokens and the preset constraints.

[0033] In this embodiment of the application, the generation probability of the t-th token is calculated based on the first t-1 tokens and the preset constraints, including:

[0034] Given the previous t-1 tokens, calculate the first probability of generating the t-th token;

[0035] Using the first t-1 tokens and the t-th token as conditions, calculate the second probability of satisfying the preset constraint conditions;

[0036] The generation probability of the t-th token is determined based on the first probability and the second probability.

[0037] An embodiment of the second aspect of this application provides a code generation apparatus based on token occurrence probability adjustment, comprising:

[0038] The token filtering module is used to filter multiple candidate tokens that meet preset constraints from a preset token table in response to an instruction to generate a first target token for a first candidate code; the first candidate code includes multiple generated tokens.

[0039] The probability of occurrence calculation module is used to calculate the adjusted probability of occurrence of the second target token based on the original probability of occurrence corresponding to the multiple candidate tokens and the original probability of occurrence of the second target token; the second target token refers to the token among the multiple generated tokens whose original probability of occurrence is less than 1;

[0040] The probability adjustment module is used to adjust the original occurrence probability of the second target token to the adjusted occurrence probability.

[0041] The probability adjustment second module is used to normalize and adjust the original occurrence probabilities corresponding to the multiple candidate tokens one by one, so as to obtain the adjusted occurrence probabilities of the multiple candidate tokens.

[0042] The second candidate code generation module is used to combine the multiple candidate tokens with the first candidate code respectively to obtain multiple second candidate codes;

[0043] The code occurrence probability calculation module is used to calculate the occurrence probability of the multiple second candidate codes based on the adjusted occurrence probability of the multiple candidate tokens and the adjusted occurrence probability of the second target token;

[0044] The target code generation module is used to generate target code based on the occurrence probability of the plurality of second candidate codes.

[0045] An embodiment of the third aspect of this application provides a computer device including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the code generation method based on token occurrence probability adjustment described in the first aspect.

[0046] An embodiment of the fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the code generation method based on token occurrence probability adjustment described in the first aspect above.

[0047] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0049] In the attached diagram:

[0050] Figure 1 A schematic diagram of the above code for the completion code provided in an embodiment of this application is shown;

[0051] Figure 2 A schematic diagram illustrating relevant information about candidate codes provided in an embodiment of this application is shown;

[0052] Figure 3 This illustration shows a schematic diagram of the occurrence probability of candidate codes after adjusting the token occurrence probability according to an embodiment of this application;

[0053] Figure 4 A schematic diagram of an occurrence probability adjustment method in the prior art provided by an embodiment of this application is shown;

[0054] Figure 5 A flowchart illustrating a code generation method based on token occurrence probability adjustment according to an embodiment of this application is shown.

[0055] Figure 6 A flowchart illustrating a code generation method based on token occurrence probability adjustment according to an embodiment of this application is shown.

[0056] Figures 7 to 10A flowchart illustrating another code generation method based on token occurrence probability adjustment provided in an embodiment of this application is shown.

[0057] Figure 11 This illustration shows a schematic diagram of a code generation apparatus based on token occurrence probability adjustment according to an embodiment of this application;

[0058] Figure 12 This illustration shows a schematic diagram of the structure of a computer device according to an embodiment of this application;

[0059] Figure 13 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0060] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0061] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0062] pass Figure 1 , Figure 2 , Figure 3 and Figure 4 The technical scenarios and technical problems involved in the embodiments of this application are described below:

[0063] Figure 1 The code above, which completes the code, defines a function "matrix_rank" that takes a parameter x of type torch.Tensor and is used to calculate the rank of the tensor.

[0064] Figure 2 This demonstrates the language model generating multiple candidate codes based on the given code, the probability of each candidate code appearing, and whether each candidate code meets the preset constraints. For example, the candidate code "matrix_rank" has a probability of 57.44%, which does not meet the preset constraints; the candidate code "linalg.matrix_rank" has a probability of 19.21%, which meets the preset constraints; and the candidate code "matrix_power" has a probability of 0.20%, which meets the preset constraints.

[0065] To address the issue that code generated by language models may fail to meet preset constraints (e.g., violate grammatical requirements), existing technologies propose a constraint decoding method that removes tokens that do not meet preset constraints, for example... Figure 2 The "rank" in "matrix_rank" does not meet the preset constraints, and the candidate code "matrix_rank" also does not meet the requirements of the preset constraints. Therefore, the candidate code "matrix_rank" is deleted, and the occurrence probabilities of the remaining candidate codes are adjusted, for example... Figure 3 As shown, the probability of the candidate code "matrix_power" was adjusted from 0.20% to 57.70%, while the probability of the candidate code "linalg.matrix_rank" was adjusted from 19.21% to 20.12%. Obviously, this adjustment method is unreasonable. The candidate code with a low probability of occurrence has a higher probability of occurrence after adjustment, and the candidate code with a high probability of occurrence has a lower probability of occurrence after adjustment. This distorts the probability distribution of the language model and causes the problem of low code generation accuracy of the language model.

[0066] The reason why the aforementioned existing technology has the problem that "candidate codes with low occurrence probability become more likely to appear after adjustment, and candidate codes with high occurrence probability become less likely to appear after adjustment" is because the occurrence probability adjustment method in the existing technology only applies to the remaining tokens at the same level as the tokens that do not meet the preset constraints, and does not adjust the occurrence probability of the related tokens mentioned above. Figure 4 As shown:

[0067] Figure 4 The tokens marked in red with "rank" are tokens that do not meet the preset constraints, while the others marked in green are tokens that meet the preset constraints.

[0068] In existing technologies, the probability of occurrence of remaining tokens at the same level as the token that does not meet the preset constraints is adjusted only. For example, for "power" and "exp" at the same level as the token marked in red "rank", the probability of occurrence of the token marked in red "rank" is set to zero, and the probability of occurrence of multiple tokens at the same level other than the token marked in red "rank" is normalized and adjusted, that is, the probability of occurrence of "power" is adjusted from 0.35% to 97.07%, and the probability of occurrence of "exp" is adjusted from 0.01% to 2.81%.

[0069] After adjusting the probability of occurrence as described above, the adjusted probability of occurrence for the candidate code "matrix_power" is 57.96% * 97.07% ≈ 57.70%; while the adjusted probability of occurrence for the candidate code "matrix_power" is 22.51% * 89.49% * 99.87% ≈ 20.12%. This distorts the probability distribution of the language model, resulting in low accuracy in code generation.

[0070] To address the aforementioned technical issues, this application provides a code generation method based on adjusting the probability of token occurrence. This method can adjust the occurrence probability of remaining tokens at the same level as tokens that do not meet preset constraints while considering the occurrence probability of related tokens mentioned above, thereby ensuring that the probability distribution of the language model is not distorted and improving the accuracy of code generation from the language model.

[0071] According to an embodiment of this application, a code generation method based on token occurrence probability adjustment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0072] This embodiment provides a code generation method based on adjusting the probability of token occurrence. Figure 5 This is a flowchart of a code generation method based on token occurrence probability adjustment according to an embodiment of this application, such as... Figure 5 As shown, the process includes the following steps:

[0073] Step S101: In response to the instruction to generate the first target token of the first candidate code, select multiple candidate tokens that meet the preset constraints from the preset token table.

[0074] In this embodiment of the application, the first candidate code includes multiple generated tokens, such as Figure 6 The "matrix" and "_" in the text refer to the first target token, which is the token that will be generated soon. For example, if t-1 tokens have been generated in the first candidate code, the first target token is the t-th token that will be generated soon.

[0075] In this embodiment, the preset token table contains all tokens. The preset constraints are not specifically limited in this application and can be set according to actual conditions, such as syntax requirements; for example, generating a token after another token would not meet syntax requirements.

[0076] In some specific embodiments, multiple candidate tokens that meet preset constraints can be selected from a preset token table in the following way:

[0077] When the first target token is the t-th token in the first candidate code, for any token in the preset token table, determine whether the token satisfies the preset constraint condition based on the token and the first t-1 tokens in the first candidate code; if the token satisfies the preset constraint condition, then the token is taken as a candidate token.

[0078] Step S102: Calculate the adjusted occurrence probability of the second target token based on the original occurrence probabilities corresponding to the multiple candidate tokens and the original occurrence probability of the second target token.

[0079] In this embodiment of the application, the multiple candidate tokens corresponding to multiple original occurrence probabilities can be understood as follows: Figure 6 The percentages are 0.35% for "power" and 0.01% for "exp". The second target token refers to the token among the multiple generated tokens whose original probability of occurrence is less than 1, for example... Figure 6 The word "matrix" in the text.

[0080] In some specific embodiments, step S102 above includes steps S1021-S1023:

[0081] Step S1021: For any candidate token among the plurality of candidate tokens, calculate the overall occurrence probability corresponding to the candidate token based on the original occurrence probability of the candidate token and the original occurrence probability of the second target token.

[0082] In the embodiments of this application, such as Figure 6 and Figure 7 As shown:

[0083] The overall probability of occurrence of candidate token "power" is calculated using the original occurrence probability of 0.35% for candidate token "power" and the original occurrence probability of the second target token "60.46%" for target token "matrix_power". The result is: 60.46% * 0.35% ≈ 0.21%. Similarly, the overall probability of occurrence of candidate token "exp" is calculated using the original occurrence probability of 0.01% for candidate token "exp" and the original occurrence probability of the second target token "60.46%" for target token "matrix_exp".

[0084] In some specific embodiments, the method further includes: filtering out multiple tokens that do not meet the preset constraints from a preset token table, and setting the occurrence probability of the multiple tokens that do not meet the preset constraints to zero.

[0085] In the embodiments of this application, such as Figure 6 , 7 As shown: The "rank" of tokens that do not meet the preset constraints will be set to zero, i.e., 60.21% → 0.00%.

[0086] Step S1022: Summing the overall occurrence probabilities of the multiple candidate tokens to obtain a probability summation result.

[0087] In this embodiment of the application, the overall occurrence probabilities of multiple candidate tokens are summed, for example... Figure 7 In the middle: 0.21% + 0.01% = 0.22%, thus obtaining the probability summation result, which is 0.22%.

[0088] Step S1023: Normalize and adjust the probability summation result and the occurrence probabilities of all tokens at the same level as the second target token to obtain the adjusted occurrence probability of the second target token.

[0089] In the embodiments of this application, such as Figure 7 As shown: The occurrence probabilities of "matrix_power", "matrix_exp", ..., "l" at the same level are normalized and adjusted. Specifically, the occurrence probability of "matrix_power" is adjusted from 0.21% to 0.53%, the occurrence probability of "matrix_exp" is adjusted from 0.01% to 0.02%, and the occurrence probability of "l" is adjusted from 22.51% to 56.58%, so that the sum of the adjusted occurrence probabilities of "matrix_power", "matrix_exp", ..., "l" at the same level is 1. Therefore, the adjusted occurrence probability of the second target token "matrix" is the sum of the occurrence probabilities of "matrix_power" and "matrix_exp", which is 0.53% + 0.02% = 0.55%.

[0090] Based on this, the adjusted probability of the candidate code "linalg.matrix_rank" is: 56.58% * 89.49% * 99.87% ≈ 50.57%.

[0091] Step S103: Adjust the original occurrence probability of the second target token to the adjusted occurrence probability.

[0092] In some specific embodiments, while adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, the method further includes:

[0093] The original occurrence probabilities of all tokens at the same level as the second target token are adjusted based on the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

[0094] In the embodiments of this application, such as Figure 9 As shown: the probability difference can be understood as 60.46% - 0.55% = 59.91%; the probability difference is proportionally divided among all tokens at the same level as the second target token, namely "..." and "l", thereby increasing the probability of token "l" from 22.51% to 56.58%.

[0095] In step S103 above, as Figure 8 As shown, it can be determined whether to accept the second target token "matrix" by rejecting sampling. When sampling "matrix" with a sampling probability of 60.46% can hit the target, and on this basis, sampling "matrix" again with a sampling probability of 0.55% / 60.46% = 0.91% can hit the target token, "matrix" can be used as a candidate token in the target code.

[0096] Step S104: Normalize and adjust the original occurrence probabilities corresponding to the multiple candidate tokens to obtain the adjusted occurrence probabilities of the multiple candidate tokens.

[0097] In the embodiments of this application, such as Figure 9 As shown, the original occurrence probabilities of multiple candidate tokens such as "power" and "exp" are normalized and adjusted, that is, the occurrence probability of "power" is adjusted from 0.35% to 97.07%, and the occurrence probability of "exp" is adjusted from 0.01% to 2.81%.

[0098] Step S105: Combine the multiple candidate tokens with the first candidate code to obtain multiple second candidate codes.

[0099] Step S106: Calculate the occurrence probability of the multiple second candidate codes based on the adjusted occurrence probability of the multiple candidate tokens and the adjusted occurrence probability of the second target token;

[0100] In steps S105-S106, multiple second candidate codes are obtained by combining them, such as... Figure 10 As shown: "power" combined with "matrix_" results in "matrix_power", and "exp" combined with "matrix_" results in "matrix_exp". The probability of the second candidate code "matrix_power" appearing is 0.55% * 100% * 97.07% = 0.53%, and the probability of the second candidate code "matrix_exp" appearing is 0.55% * 100% * 2.81% = 0.015%.

[0101] Step S107: Generate target code based on the occurrence probability of the plurality of second candidate codes.

[0102] The code generation method based on token occurrence probability adjustment provided in this application embodiment shows that the occurrence probability of candidate code "linalg.matrix_rank" after adjustment is 50.57%, while the occurrence probability of candidate code "matrix_power" after adjustment is 0.53%. Comparing the two candidate codes before and after adjustment, the changes are: 0.2% → 0.53% and 19.21% → 50.57%. Therefore, it can be determined that the technical solution of this application embodiment will not cause the problem of "candidate codes with low occurrence probability becoming more likely to appear after adjustment, and candidate codes with high occurrence probability becoming less likely to appear after adjustment," thus ensuring that the probability distribution of the language model is not distorted, achieving the technical effect of improving the accuracy of code generation from the language model.

[0103] In some specific embodiments, the method further includes:

[0104] The generation probability of the target code is calculated according to the following steps:

[0105] When t=1, calculate the generation probability of the first token according to the preset constraints; the target code includes t tokens;

[0106] When t > 1, calculate the generation probability of the t-th token based on the first t-1 tokens and the preset constraints.

[0107] The generation probability of the target code is calculated based on the t generation probabilities that correspond one-to-one with the t tokens.

[0108] In the embodiments of this application,

[0109]

[0110] Where S represents the target code containing t tokens, C represents the preset constraints, P(S|C) represents the generation probability of the target code calculated based on the t tokens and the preset constraints, and t represents the index of the token in the target code. t Let P(s) represent the t-th token. t |c,s ＜t ) indicates that the generation probability of the t-th token is calculated based on the first t-1 tokens and the preset constraints.

[0111] In some specific embodiments, the generation probability of the t-th token is calculated based on the first t-1 tokens and the preset constraints, including:

[0112] Given the previous t-1 tokens, calculate the first probability of generating the t-th token;

[0113] Using the first t-1 tokens and the t-th token as conditions, calculate the second probability of satisfying the preset constraint conditions;

[0114] The generation probability of the t-th token is determined based on the first probability and the second probability.

[0115] In the embodiments of this application,

[0116] P(s t ||c,S ＜t )∝P LM (S t ||S ＜t )P(c|S t S ＜t ).

[0117] Wherein, P(s) t |c,s ＜t ) represents the probability of generating the t-th token based on the first t-1 tokens and the preset constraints; P LM (s t |s ＜t ) represents the probability of generating the t-th token given the previous t-1 tokens, P(c|s) t s ＜t ) represents the second probability of satisfying the preset constraint condition, using the previous t-1 tokens and the t-th token as conditions.

[0118] In the embodiments of this application,

[0119] We use Q[x] as an estimation of P(c|x).

[0120] We also maintain N[x]sampled from P Q (·|c,x).

[0121]

[0122] Where P(c|x) represents the above P(c|s) t s ＜t Q[x] is used to determine the validity of the code above (i.e., whether the first t-1 tokens meet the preset constraints).

[0123] Corresponding to the above implementation of the code generation method based on adjusting the probability of token occurrence, this application embodiment also provides a code generation apparatus based on adjusting the probability of token occurrence, used to execute the above... Figures 1 to 10 The code generation method based on adjusting the probability of token occurrence is illustrated in any of the suggested embodiments. Figure 11 As shown, the code generation device based on token occurrence probability adjustment includes:

[0124] The token filtering module is used to filter multiple candidate tokens that meet preset constraints from a preset token table in response to an instruction to generate a first target token for a first candidate code; the first candidate code includes multiple generated tokens.

[0125] The probability of occurrence calculation module is used to calculate the adjusted probability of occurrence of the second target token based on the original probability of occurrence corresponding to the multiple candidate tokens and the original probability of occurrence of the second target token; the second target token refers to the token among the multiple generated tokens whose original probability of occurrence is less than 1;

[0126] The probability adjustment module is used to adjust the original occurrence probability of the second target token to the adjusted occurrence probability.

[0127] The probability adjustment second module is used to normalize and adjust the original occurrence probabilities corresponding to the multiple candidate tokens one by one, so as to obtain the adjusted occurrence probabilities of the multiple candidate tokens.

[0128] The second candidate code generation module is used to combine the multiple candidate tokens with the first candidate code respectively to obtain multiple second candidate codes;

[0129] The code occurrence probability calculation module is used to calculate the occurrence probability of the multiple second candidate codes based on the adjusted occurrence probability of the multiple candidate tokens and the adjusted occurrence probability of the second target token;

[0130] The target code generation module is used to generate target code based on the occurrence probability of the plurality of second candidate codes.

[0131] Optionally, the device further includes: a probability adjustment third module, configured to: while adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, adjust the original occurrence probability of all tokens at the same level as the second target token according to the probability difference, so as to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

[0132] Optionally, the occurrence probability calculation module is further configured to: for any candidate token among the plurality of candidate tokens, calculate the overall occurrence probability corresponding to the candidate token based on the original occurrence probability of the candidate token and the original occurrence probability of the second target token; sum the multiple overall occurrence probabilities of the plurality of candidate tokens to obtain a probability summation result; and normalize and adjust the probability summation result and the occurrence probabilities of all tokens at the same level as the second target token to obtain the adjusted occurrence probability of the second target token.

[0133] Optionally, the first target token is the t-th token in the first candidate code; the token filtering module is further configured to, for any token in all tokens of the preset token table, determine whether the token satisfies the preset constraint conditions based on the token and the first t-1 tokens in the first candidate code; if the token satisfies the preset constraint conditions, then the token is selected as a candidate token.

[0134] Optionally, the device further includes a probability zeroing module, used to filter out multiple tokens that do not meet the preset constraints from the preset token table, and set the occurrence probability of the multiple tokens that do not meet the preset constraints to zero.

[0135] The code generation apparatus based on token occurrence probability adjustment provided in the above embodiments of this application and the code generation method based on token occurrence probability adjustment provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications they store.

[0136] This application also provides a computer device for executing the above-described code generation method based on token occurrence probability adjustment. Please refer to... Figure 12 This illustrates a schematic diagram of a computer device provided by some embodiments of this application. For example... Figure 12 As shown, the computer device 12 includes: a processor 1200, a memory 1201, a bus 1202, and a communication interface 1203. The processor 1200, the communication interface 1203, and the memory 1201 are connected via the bus 1202. The memory 1201 stores a computer program that can run on the processor 1200. When the processor 1200 runs the computer program, it executes the aforementioned functions described in this application. Figures 1 to 10 The illustrated implementation provides a code generation method based on adjusting the probability of token occurrence.

[0137] The memory 1201 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 1203 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0138] Bus 1202 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 1201 is used to store programs, and the processor 1200 executes the programs after receiving execution instructions. Figures 1 to 10 The code generation method based on token occurrence probability adjustment disclosed in any of the illustrated embodiments can be applied to or implemented by the processor 1200.

[0139] The processor 1200 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1200 or by instructions in software form. The processor 1200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1201. Processor 1200 reads the information in memory 1201 and completes the steps of the above method in conjunction with its hardware.

[0140] The computer device provided in this application embodiment and the code generation method based on token occurrence probability adjustment provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0141] This application also provides a computer-readable storage medium corresponding to the code generation method based on token occurrence probability adjustment provided in the foregoing embodiments. Please refer to... Figure 13 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the code generation method based on the token occurrence probability adjustment provided in any of the foregoing embodiments.

[0142] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0143] The computer-readable storage medium provided in the above embodiments of this application and the code generation method based on token occurrence probability adjustment provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0144] It should be noted that:

[0145] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0146] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0147] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0148] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A code generation method based on adjusting the probability of token occurrence, characterized in that, The method includes: In response to the instruction to generate a first target token for a first candidate code, multiple candidate tokens that meet preset constraints are selected from a preset token table; the first candidate code includes multiple generated tokens. The adjusted occurrence probability of the second target token is calculated based on the original occurrence probabilities corresponding to the multiple candidate tokens and the original occurrence probability of the second target token; the second target token refers to the token among the multiple generated tokens whose original occurrence probability is less than 1. The original occurrence probability of the second target token is adjusted to the adjusted occurrence probability; The original occurrence probabilities corresponding to the multiple candidate tokens are normalized and adjusted to obtain the adjusted occurrence probabilities of the multiple candidate tokens. The multiple candidate tokens are combined with the first candidate code to obtain multiple second candidate codes; The occurrence probability of the multiple second candidate codes is calculated based on the adjusted occurrence probability of the multiple candidate tokens and the adjusted occurrence probability of the second target token; The target code is generated based on the occurrence probability of the plurality of second candidate codes.

2. The method according to claim 1, characterized in that, While adjusting the original occurrence probability of the second target token to the adjusted occurrence probability, the method further includes: The original occurrence probabilities of all tokens at the same level as the second target token are adjusted based on the probability difference to obtain the adjusted occurrence probability of each token at the same level; the probability difference refers to the difference between the original occurrence probability of the second target token and the adjusted occurrence probability.

3. The method according to claim 1 or 2, characterized in that, The adjusted occurrence probability of the second target token is calculated based on the original occurrence probabilities corresponding to the multiple candidate tokens and the original occurrence probability of the second target token, including: For any candidate token among the plurality of candidate tokens, calculate the overall occurrence probability corresponding to the candidate token based on the original occurrence probability of the candidate token and the original occurrence probability of the second target token; The probability summation result is obtained by summing the overall occurrence probabilities of the multiple candidate tokens; The probability summation result and the occurrence probabilities of all tokens at the same level as the second target token are normalized and adjusted to obtain the adjusted occurrence probability of the second target token.

4. The method according to claim 1, characterized in that, The first target token is the t-th token in the first candidate code; Select multiple candidate tokens from the preset token table that meet the preset constraints, including: For any token in the preset token table, determine whether the token satisfies the preset constraint conditions based on the token and the first t-1 tokens in the first candidate code; If the token satisfies the preset constraints, then the token is selected as a candidate token.

5. The method according to claim 1, characterized in that, Following the instruction to generate a first target token for a first candidate code, the method further includes: Select multiple tokens from the preset token table that do not meet the preset constraints, and set the probability of occurrence of the multiple tokens that do not meet the preset constraints to zero.

6. The method according to claim 1 or 2, characterized in that, The method further includes: The generation probability of the target code is calculated according to the following steps: When t=1, the probability of generating the first token is calculated according to the preset constraints; the target code includes t tokens; When t > 1, calculate the generation probability of the t-th token based on the first t-1 tokens and the preset constraints. The generation probability of the target code is calculated based on the t generation probabilities that correspond one-to-one with the t tokens.

7. The method according to claim 1 or 2, characterized in that, Wherein, S represents target code containing t tokens, C represents preset constraint condition, P(S|C) represents generation probability of the target code according to the t tokens and the preset constraint condition, t represents serial number of the token in the target code, S t represents the tth token, P(s t |c, s ＜t t-1) represents generation probability of the tth token according to the first t-1 tokens and the preset constraint condition.

8. The method according to claim 7, characterized in that, Based on the first t-1 tokens and the preset constraints, calculate the generation probability of the t-th token, including: Given the previous t-1 tokens, calculate the first probability of generating the t-th token; Using the first t-1 tokens and the t-th token as conditions, calculate the second probability of satisfying the preset constraint conditions; The generation probability of the t-th token is determined based on the first probability and the second probability.

9. A code generation device based on adjusting the probability of token occurrence, characterized in that, The device includes: The token filtering module is used to filter multiple candidate tokens that meet preset constraints from a preset token table in response to an instruction to generate a first target token for a first candidate code; the first candidate code includes multiple generated tokens. The probability of occurrence calculation module is used to calculate the adjusted probability of occurrence of the second target token based on the original probability of occurrence corresponding to the multiple candidate tokens and the original probability of occurrence of the second target token; the second target token refers to the token among the multiple generated tokens whose original probability of occurrence is less than 1; The probability adjustment module is used to adjust the original occurrence probability of the second target token to the adjusted occurrence probability. The probability adjustment second module is used to normalize and adjust the original occurrence probabilities corresponding to the multiple candidate tokens one by one, so as to obtain the adjusted occurrence probabilities of the multiple candidate tokens. The second candidate code generation module is used to combine the multiple candidate tokens with the first candidate code respectively to obtain multiple second candidate codes; The code occurrence probability calculation module is used to calculate the occurrence probability of the multiple second candidate codes based on the adjusted occurrence probability of the multiple candidate tokens and the adjusted occurrence probability of the second target token; The target code generation module is used to generate target code based on the occurrence probability of the plurality of second candidate codes.

10. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the code generation method based on token occurrence probability adjustment as described in any one of claims 1 to 8.

Images