Data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a storage medium, and the method comprises the steps: predicting N first prediction probability lists corresponding to N lexical element positions through a sub-language model based on cue word information, and predicting N first prediction probability lists corresponding to N lexical element positions through a main language model based on the cue word information and the N first prediction probability lists, and predicting N second prediction probability lists corresponding to the N lexical element positions. And determining probability distribution characteristics corresponding to the prediction probabilities in the N first prediction probability lists, and adjusting the prediction probabilities in the N first prediction probability lists according to the probability distribution characteristics corresponding to the N first prediction probability lists to obtain N first adjustment probability lists. And similarly, obtaining N second adjustment probability lists based on the N second prediction probability lists. And according to the N first adjustment probability lists, the N second adjustment probability lists and the lexical element list, determining text content meeting the prompt word information. Through the method, the generation efficiency of the text content can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] In the realm of text content, Large Language Models (LLMs) generate text content through an autoregressive approach, whereby the LLM predicts and outputs the next lexical position sequentially. While this method offers the highest quality, its inherent sequential dependencies lead to significant computational latency. To mitigate this issue, various speculative decoding techniques have been proposed in the industry.

[0003] The core idea of ​​speculative decoding technology lies in using a less computationally intensive sub-language model (i.e., a draft model) to pre-predict candidate lexical sequences (i.e., "draft tokens") for several lexical positions. Subsequently, a more powerful but computationally expensive main language model (i.e., a target model) is used to quickly verify and correct these candidate sequences. If the candidate lexical sequence predicted by the sub-language model is accepted by the main language model, multiple predicted lexical tokens can be generated at once, significantly reducing the number of calls to the main language model and accelerating the overall inference process. However, in practical applications, due to significant differences in structure between the sub-language model and the main language model, the probability of the candidate lexical sequence predicted by the sub-language model being accepted by the main language model is relatively low, resulting in relatively low efficiency in text content generation. Summary of the Invention

[0004] This application provides a data processing method that can improve the efficiency of text content generation.

[0005] One embodiment of this application provides a data processing method, including:

[0006] Based on the prompt word information, the sub-language model predicts N first prediction probability lists corresponding to N word positions. Based on the prompt word information and the N first prediction probability lists, the main language model predicts N second prediction probability lists corresponding to the N word positions. The first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability that each candidate word in the indicator word list belongs to the predicted word at the i-th word position. N is a positive integer; i is a positive integer less than or equal to N.

[0007] Determine the probability distribution characteristics corresponding to the predicted probabilities in the above N first predicted probability lists respectively, and adjust the predicted probabilities in the above N first predicted probability lists according to the probability distribution characteristics corresponding to the above N first predicted probability lists respectively to obtain N first adjusted probability lists.

[0008] Determine the probability distribution characteristics corresponding to the predicted probabilities in the above N second prediction probability lists respectively, and adjust the predicted probabilities in the above N second prediction probability lists according to the probability distribution characteristics corresponding to the above N second prediction probability lists respectively to obtain N second adjusted probability lists.

[0009] Based on the above N first adjustment probability lists, the above N second adjustment probability lists, and the above word list, determine the text content that satisfies the above prompt word information. The above text content includes the predicted words corresponding to the above N word positions.

[0010] One embodiment of this application provides a data processing apparatus, including:

[0011] The prediction module is used to predict N first prediction probability lists corresponding to N word positions based on prompt word information using a sub-language model, and to predict N second prediction probability lists corresponding to the N word positions using a main language model based on the prompt word information and the N first prediction probability lists. The first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position; N is a positive integer; i is a positive integer less than or equal to N.

[0012] The first adjustment module is used to determine the probability distribution characteristics corresponding to the predicted probabilities in the above N first predicted probability lists respectively, and adjust the predicted probabilities in the above N first predicted probability lists according to the probability distribution characteristics corresponding to the above N first predicted probability lists respectively, so as to obtain N first adjusted probability lists.

[0013] The second adjustment module is used to determine the probability distribution characteristics corresponding to the predicted probabilities in the above N second prediction probability lists respectively, and adjust the predicted probabilities in the above N second prediction probability lists according to the probability distribution characteristics corresponding to the above N second prediction probability lists respectively, so as to obtain N second adjusted probability lists.

[0014] The processing module is used to determine the text content that satisfies the above prompt word information based on the above N first adjustment probability lists, the above N second adjustment probability lists and the above word list. The above text content includes the predicted word corresponding to the above N word positions.

[0015] One embodiment of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0016] One embodiment of this application provides a computer storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described method.

[0017] One aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0018] In this application, the prediction probabilities in N first predicted probability lists are dynamically adjusted based on the probability distribution features of the N first predicted probability lists obtained by the sub-language model, resulting in N first adjusted probability lists. Similarly, the prediction probabilities in N second predicted probability lists are dynamically adjusted based on the probability distribution features of the N second predicted probability lists obtained by the main language model, resulting in N second adjusted probability lists. This approach helps to make the probability distribution features corresponding to the adjusted probabilities in the first adjusted probability lists closer to those in the second adjusted probability lists. In other words, the difference between the probability distribution features corresponding to the adjusted probabilities in the first and second adjusted probability lists is relatively small. This increases the probability that the candidate word sequences predicted by the sub-language model (i.e., word sequences obtained based on the first adjusted probability lists) are accepted by the main language model, thereby improving the efficiency of text content generation. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application;

[0020] Figure 2 This is a diagram illustrating an intelligent question-and-answer scenario provided in an embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 ;

[0022] Figure 4a This is a data processing scenario provided in the embodiments of this application. Figure 1 ;

[0023] Figure 4b This is a data processing scenario provided in the embodiments of this application. Figure 2 ;

[0024] Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 ;

[0025] Figure 6 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 ;

[0026] Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0029] The core idea of ​​speculative decoding technology lies in using a less computationally intensive sub-language model (i.e., a draft model) to pre-specify and generate candidate lexical sequences (i.e., "draft tokens") for several lexical positions. Subsequently, a more powerful but computationally expensive main language model (i.e., a target model) is used to quickly verify and correct these candidate sequences. If the candidate lexical sequence predicted by the sub-language model is accepted by the main language model, multiple predicted lexical tokens can be generated at once, significantly reducing the number of calls to the main language model and accelerating the overall inference process.

[0030] The specific implementation process, based on speculative decoding technology, includes the following steps in text generation:

[0031] 1. Generate predicted probabilities;

[0032] Based on the prompt word information, the sub-language model predicts a list of N first predicted probabilities corresponding to N word positions. Based on the prompt word information and the list of N first predicted probabilities, the main language model predicts a list of N second predicted probabilities corresponding to N word positions.

[0033] Wherein, the first prediction probability list i includes the prediction probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position; the second prediction probability list i includes the prediction probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position; i is a positive integer less than or equal to N.

[0034] 2. The main language model verifies whether the candidate word sequences predicted by the sub-language model are accepted;

[0035] The verification process includes the following two key steps:

[0036] a. Sampling: This refers to sampling the first predicted probability list for each word position according to a sampling strategy, resulting in N first sampled probability sets corresponding to each word position. The candidate words corresponding to each predicted probability in the first sampled probability set for each word position are the candidate words finally output by the sub-language model for each word position. Following the sampling strategy, sampling the second predicted probability list for each word position, resulting in N second sampled probability sets corresponding to each word position. The candidate words corresponding to each predicted probability in the second sampled probability set for each word position are the candidate words finally output by the main language model for each word position.

[0037] The sampling strategies here include greedy search, top-k sampling (selecting the k highest predicted probabilities from the probability list for each word position), and top-p sampling (i.e., kernel sampling, selecting the smallest set of predicted probabilities whose cumulative probability exceeds a preset threshold). Computer devices can choose an appropriate sampling strategy based on requirements such as computational efficiency, the diversity of generated text, and quality.

[0038] b. Rejection sampling can be implemented in the following two ways:

[0039] Implementation Method 1: Based on the first sampled probability set for each word position, determine the first optimal word for each word position. The first optimal word is the candidate word corresponding to the maximum predicted probability output by the sub-language model for each word position. Based on the second sampled probability set for each word position, determine the second optimal word for each word position. The second optimal word is the candidate word corresponding to the maximum predicted probability output by the main language model for each word position. If the first optimal word and the second optimal word are the same for the same word position, the first optimal word of the sub-language model at that word position is determined as the predicted word for that word position; this can also be referred to as the acceptance of the candidate word of the sub-language model. If the first optimal word and the second optimal word are different for the same word position, the first optimal word of the sub-language model at that word position is rejected; that is, the candidate word output by the sub-language model for that word position is rejected.

[0040] Method 2: From the word list, obtain the candidate words corresponding to each predicted probability in the first sampling probability set for each word position. That is, from the word list, obtain the candidate words output by the sub-language model for each word position. From the first predicted probability list corresponding to each word position, obtain the first predicted probability of the candidate words output by the sub-language model for each word position. From the second predicted probability list corresponding to each word position, obtain the second predicted probability of the candidate words output by the sub-language model for each word position. If the probability ratio between the second and first predicted probabilities of the candidate words output by the sub-language model for a certain word position is greater than or equal to a probability ratio threshold, the candidate words output by the sub-language model for that word position can be determined as the predicted word for that word position, or the candidate words of the sub-language model are accepted. If the probability ratio between the second and first predicted probabilities of the candidate words output by the sub-language model for a certain word position is less than a probability ratio threshold, the candidate words output by the sub-language model for that word position are rejected.

[0041] It should be noted that if a candidate word output by the sub-language model for a certain word position is rejected, the candidate words output by the sub-language model for subsequent word positions will no longer be validated. In this case, based on the second sampled probability set of the rejected word position, a predicted word for that rejected word position is determined. This predicted word is the candidate word corresponding to the maximum predicted probability output by the main language model for that rejected word position.

[0042] In one embodiment, if all candidate words output by the sub-language model for each word position are accepted, the main language model can predict a second predicted probability list for the next word position based on the prompt word information and the first and second adjusted probability lists corresponding to each word position. Sampling is performed on the second predicted probability list for this next word position to obtain a second sampled probability set. Based on this second sampled probability set, the second optimal word for the next word position is determined. The second optimal word for this next word position, as determined by the main language model, is then considered the predicted word for that position; this can also be referred to as the main language model rewarding the predicted word for the next word position.

[0043] In practice, it has been found that due to the small model size of sub-language models—that is, their simpler model structure and fewer parameters—they are prone to overfitting on the training data. This results in a large difference between the predicted probabilities in the first predicted probability list obtained by the sub-language model for a certain word position. In other words, some candidate words in the word list have an excessively high predicted probability for that word position, while others have an excessively low predicted probability, leading to a sharp distribution of predicted probabilities in the first predicted probability list.

[0044] Meanwhile, due to the larger model size of the main language model—that is, its more complex model structure and more parameters—it possesses stronger information extraction and generalization capabilities, effectively mitigating overfitting. This results in smaller differences between the predicted probabilities in the second predicted probability list obtained by the main language model for a specific word position; in other words, the predicted probabilities of each candidate word in the word list at that word position are relatively balanced, and the distribution of predicted probabilities in the second predicted probability list is relatively smooth.

[0045] As can be seen, there is a significant difference in the distribution of predicted probabilities between the sub-language model and the main language model, increasing the probability that the first optimal word at each word position obtained by the sub-language model is different from the second optimal word at each word position obtained by the main language model. This also results in a smaller probability ratio between the second and first predicted probabilities of the candidate word output by the sub-language model for each word position. Consequently, the probability of the candidate word finally output by the sub-language model for each word position being accepted in the rejection sampling verification process in step 2 is relatively low. Therefore, multiple calls to the sub-language model and the main language model are required to generate text content, leading to low text content generation efficiency.

[0046] If fixed adjustment parameters are used to adjust the prediction probabilities in the first and second prediction probability lists, resulting in adjusted probability lists for the main language model and the sub-language model respectively, this will only increase or decrease the prediction probabilities in the first prediction probability list corresponding to the sub-language model. It will not change the distribution characteristics of the prediction probabilities in the first prediction probability list corresponding to the main language model; that is, the distribution characteristics of the prediction probabilities in the first prediction probability list are the same as or similar to the distribution characteristics of the adjusted probabilities in the adjusted probability list corresponding to the main language model. Similarly, this will only increase or decrease the prediction probabilities in the second prediction probability list corresponding to the main language model. It will not change the distribution characteristics of the prediction probabilities in the second prediction probability list corresponding to the sub-language model; that is, the distribution characteristics of the prediction probabilities in the second prediction probability list are the same as or similar to the distribution characteristics of the adjusted probabilities in the adjusted probability list corresponding to the sub-language model. Therefore, there is still a significant distribution difference between the adjusted probabilities in the adjusted probability lists corresponding to the sub-language model and the main language model, reducing the probability of the sub-language model being accepted. This necessitates multiple calls to both the sub-language model and the main language model to generate text content, resulting in relatively low text content generation efficiency.

[0047] Based on this, this application proposes a data processing method. Before performing step 2 above, the predicted probability list can be dynamically adjusted based on the probability distribution characteristics of the predicted probability list. Compared with the method of using fixed adjustment parameters, this dynamic adjustment method is more adaptable and has higher adjustment accuracy.

[0048] Specifically, the predicted probabilities in the N first predicted probability lists obtained from the sub-language model are dynamically adjusted based on their probability distribution characteristics, resulting in N first adjusted probability lists. Similarly, the predicted probabilities in the N second predicted probability lists obtained from the main language model are dynamically adjusted based on their probability distribution characteristics, resulting in N second adjusted probability lists. This approach helps to make the probability distribution characteristics corresponding to the adjusted probabilities in the first adjusted probability lists closer to those in the second adjusted probability lists, meaning the difference between the probability distribution characteristics corresponding to the adjusted probabilities in the first and second adjusted probability lists is relatively small.

[0049] Next, for the first and second adjustment probability lists corresponding to the N word positions, step 2 above can be performed; that is, based on the N first adjustment probability lists, the N second adjustment probability lists, and the word list, the text content that satisfies the prompt word information is determined. This helps to increase the probability that the candidate word sequence predicted by the sub-language model (i.e., the word sequence obtained based on the first adjustment probability list) is accepted by the main language model, improve the efficiency of text content generation, and at the same time provide a new technical idea for inference decoding technology.

[0050] First, let's introduce Figure 1 , Figure 1 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application, such as... Figure 1 As shown in the schematic diagram, the data processing system may include a server 10 and a terminal cluster, which may include multiple terminals. Figure 1 As shown, taking a terminal cluster consisting of three terminals as an example, the terminal cluster can specifically include a first terminal, a second terminal, and a third terminal. It can be understood that the first terminal, the second terminal, and the third terminal can all connect to the server via the network so that each terminal can interact with the server through the network connection.

[0051] Understandably, a server can be a single physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud knowledge base, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0052] It is understood that the terminal can specifically refer to smartphones, tablets, laptops, etc., but is not limited to these. Various terminals and servers can be connected directly or indirectly via wired or wireless communication. Furthermore, the number of terminals and servers can be one or at least two; this application does not impose any limitation on this.

[0053] The server 10 can generate text content that satisfies the prompt word information based on the N first predicted probability lists corresponding to the N word positions predicted by the sub-language model and the N second predicted probability lists corresponding to the N word positions predicted by the main language model.

[0054] It should be noted that the N first predicted probability lists corresponding to N word positions are predicted by the sub-language model based on the prompt word information, while the N second predicted probability lists corresponding to N word positions are predicted by the main language model based on the prompt word information and the N first predicted probability lists corresponding to N word positions.

[0055] The aforementioned sub-language model can be deployed on server 10 or on any terminal; the aforementioned main language model can also be deployed on server 10 or on any terminal. Taking the first terminal as an example, when both the sub-language model and the main language model are deployed on the first terminal, server 10 can obtain from the first terminal a list of N first predicted probabilities corresponding to the N word positions predicted by the sub-language model, and a list of N second predicted probabilities corresponding to the N word positions predicted by the main language model.

[0056] It should be noted that the data processing method provided in this application can be applied to multiple scenarios, including intelligent question answering scenarios, knowledge retrieval scenarios, question search scenarios, game interaction scenarios, etc.

[0057] For example, in the context of intelligent question answering, please refer to... Figure 2 , Figure 2 This is a diagram of an intelligent question-and-answer scenario provided in an embodiment of this application. Figure 2 This includes terminal 20 and server 21. Server 21 can refer to... Figure 1 Server 10 and terminal 20 can be... Figure 1 Any terminal in the network.

[0058] Among them, server 21 can refer to the backend device corresponding to the intelligent question and answer platform. Specifically, server 21 can be used to authenticate the objects that log in to the intelligent question and answer platform and provide intelligent question and answer services to legitimate objects.

[0059] Specifically, terminal 20 can use the intelligent question-and-answer page 20a provided by the intelligent question-and-answer platform to send question information (i.e., Figure 2The text box 20b containing the phrase "Recommended delicious hot pot" is sent to the server 21. The server 21 can receive the question information sent by the terminal 20, fill the question information into the prompt word template, and obtain the prompt word information. The server 21 can input the prompt word information into the sub-language model 22, and predict three first prediction probability lists corresponding to the three word positions through the sub-language model 22. The three first prediction probability lists are first prediction probability list 220a, first prediction probability list 220b, and first prediction probability list 220c. The server 21 can input the prompt word information and the above three first prediction probability lists into the main language model 23, and predict three second prediction probability lists corresponding to the three word positions through the main language model 23. The three second prediction probability lists are second prediction probability list 230a, second prediction probability list 230b, and second prediction probability list 230c.

[0060] Next, server 21 can determine the probability distribution features corresponding to the three first prediction probability lists based on the predicted probabilities of each of the three first prediction probability lists. These features are: probability distribution feature 221a corresponding to first prediction probability list 220a, probability distribution feature 221b corresponding to first prediction probability list 220b, and probability distribution feature 221c corresponding to first prediction probability list 220c. Based on the probability distribution features corresponding to the three first prediction probability lists, server 21 adjusts the three first prediction probability lists to obtain three first adjusted probability lists: first adjusted probability list 222a, first adjusted probability list 222b, and first adjusted probability list 222c. Server 21 can then determine the probability distribution features corresponding to the three second prediction probability lists based on the predicted probabilities of each of the three second prediction probability lists. These features are: probability distribution feature 231a corresponding to second prediction probability list 230a, probability distribution feature 231b corresponding to second prediction probability list 230b, and probability distribution feature 231c corresponding to second prediction probability list 230c. Based on the probability distribution characteristics corresponding to the three second prediction probability lists, the three second prediction probability lists are adjusted to obtain three second adjusted probability lists, namely second adjusted probability list 232a, second adjusted probability list 232b and second adjusted probability list 232c.

[0061] Server 21 can, based on the first adjustment probability list 222a and the second adjustment probability list 232a, obtain the predicted word 24a at the first word position from the word list; based on the first adjustment probability list 222b and the second adjustment probability list 232b, obtain the predicted word 24b at the second word position from the word list; based on the first adjustment probability list 222c and the second adjustment probability list 232c, obtain the predicted word 24c at the third word position from the word list. Server 21 can arrange the predicted word 24a, predicted word 24b, and predicted word 24c to obtain text content that satisfies the prompt word information. This text content is the intelligent answer content corresponding to the question information; the intelligent answer content is then sent to terminal 20. The user corresponding to terminal 20 can browse the intelligent answer content (i.e., ...) through the intelligent question answer page 20a provided by the intelligent question answer platform. Figure 2 The text box 20c contains the phrase "Spicy hot pot, beef hot pot, and mutton hot pot are all good choices."

[0062] The data processing method provided in this application can improve the efficiency of text content generation in intelligent question-answering scenarios, effectively shorten the waiting time of objects in intelligent question-answering scenarios, and improve the intelligent question-answering experience of objects.

[0063] Please see Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 .like Figure 3 As shown, this method can be derived from... Figure 1 It can be executed by any terminal in the system, or by... Figure 1 The server in the middle can be used to execute it, or it can be executed by... Figure 1 The terminal and server in the process work together to execute the data processing method. The device used to execute this data processing method in this application can be collectively referred to as a computer device. The method may include the following steps:

[0064] S101. Based on the prompt word information, predict the N first prediction probability lists corresponding to the N word positions using the sub-language model. Based on the prompt word information and the N first prediction probability lists, predict the N second prediction probability lists corresponding to the N word positions using the main language model.

[0065] In this application, the computer device can input the prompt word information into the sub-language model, and use the sub-language model to identify the prompt word information to obtain a list of N first predicted probabilities corresponding to N word positions; input the list of N first predicted probabilities corresponding to N word positions and the prompt word information into the main language model, and use the main language model to obtain a list of N second predicted probabilities corresponding to N word positions.

[0066] It should be noted that cue word information can refer to instructional or guiding text information used as input into the model. Cue word information can be used to indicate the probability list of predicted word positions for the sub-language model and the main language model.

[0067] It should be noted that each word position corresponds to a first prediction probability list and a second prediction probability list; specifically, the i-th word position corresponds to the first prediction probability list i and the second prediction probability list i. The first and second prediction probability lists i corresponding to the i-th word position respectively include the prediction probability that each candidate word in the indicator word list belongs to the predicted word at the i-th word position; N is a positive integer; i is a positive integer less than or equal to N.

[0068] It should be noted that each prediction probability in the first prediction probability list i is used to reflect the probability that the sub-language model will select the corresponding candidate word in the word list as the predicted word at the i-th word position, and the sum of all prediction probabilities in the first prediction probability list i is 1.

[0069] Taking the first prediction probability list i as [0.6, 0.1, ...] as an example, 0.6 represents the probability that the sub-language model selects the first candidate word in the word list as the predicted word for the i-th word position, 0.1 represents the probability that the sub-language model selects the second candidate word in the word list as the predicted word for the i-th word position, and so on. Since the prediction probability corresponding to the second candidate word in the first prediction probability list i is less than the prediction probability corresponding to the first candidate word, it indicates that the sub-language model determines that the first candidate word in the word list is more suitable as the predicted word for the i-th word position.

[0070] Similarly, each prediction probability in the second prediction probability list i is used to reflect the probability that the main language model will select the corresponding candidate word in the word list as the predicted word at the i-th word position. The sum of all prediction probabilities in the second prediction probability list i is 1.

[0071] Taking the second prediction probability list i as [0.9, 0.01, ...] as an example, 0.9 represents the probability that the main language model selects the first candidate word in the word list as the predicted word for the i-th word position, 0.01 represents the probability that the main language model selects the second candidate word in the word list as the predicted word for the i-th word position, and so on. Since the prediction probability corresponding to the second candidate word in the second prediction probability list i is less than the prediction probability corresponding to the first candidate word, it indicates that the main language model determines that the first candidate word in the word list is more suitable as the predicted word for the i-th word position.

[0072] It should be noted that the lexical list may include P candidate lexical units, where P is a positive integer greater than 3; a candidate lexical unit is the smallest text unit in the text content, and a candidate lexical unit can be any of the following: characters, symbols, letters, etc.

[0073] The word list and the prompt word information are related; that is, the candidate words in the word list can belong to the same domain as the prompt word information. For example, if the prompt word information includes text content in the field of science and technology (such as computers, networks, code, etc.), the word list will include candidate words belonging to the field of science and technology.

[0074] In one embodiment, the sub-language model may include multiple prediction layers, and different prediction layers may perform predictions sequentially, such as... Figure 4a As shown. Figure 4a A data processing scenario provided in the embodiments of this application Figure 1 , Figure 4a It includes prompt word information 40, sub-language model 41 and main language model 42; wherein, the sub-language model 41 may include three prediction layers, namely prediction layer 41a, prediction layer 41b and prediction layer 41c.

[0075] Taking the probability list of predicting three word positions as an example, specifically, the computer device can input the prompt word information 40 into the prediction layer 41a, and obtain the first predicted probability list 411 corresponding to the first word position through the prediction layer 41a, which is the output of the prediction layer 41a. The output of the prediction layer 41a is then input into the prediction layer 41b, and obtain the first predicted probability list 412 corresponding to the second word position through the prediction layer 41b, which is the output of the prediction layer 41b. The output of the prediction layer 41b is then input into the prediction layer 41c, and obtain the first predicted probability list 413 corresponding to the third word position through the prediction layer 41c, which is the output of the prediction layer 41c. The outputs of prediction layers 41a, 41b, and 41c, and the prompt word information 40 are input into the main language model 42. The main language model 42 obtains the second prediction probability list 421 corresponding to the first word position, the second prediction probability list 422 corresponding to the second word position, and the second prediction probability list 423 corresponding to the third word position.

[0076] S102. Determine the probability distribution characteristics corresponding to the predicted probabilities in the N first predicted probability lists respectively. Based on the probability distribution characteristics corresponding to the N first predicted probability lists respectively, adjust the predicted probabilities in the N first predicted probability lists to obtain N first adjusted probability lists.

[0077] It should be noted that the probability distribution characteristics of the first predicted probability list can be used to reflect the probability distribution state of the predicted probabilities in the first predicted probability list. The probability distribution state of the first predicted probability list can include a sharp distribution state, a stationary distribution state, etc.; wherein, a sharp distribution state indicates that the difference between the predicted probabilities in the first predicted probability list is greater than or equal to a difference threshold; a stationary distribution state indicates that the difference between the predicted probabilities in the first predicted probability list is less than a difference threshold.

[0078] The probability distribution features of the first predicted probability list may include at least one of the following: probability distribution entropy, probability variance, and head probability sum (the sum of the k highest predicted probabilities in the first predicted probability list). The correspondence between the probability distribution features of the first predicted probability list and the probability distribution state can be determined based on the specific content of the probability distribution features of the first predicted probability list.

[0079] For example, when the probability distribution characteristics of the first prediction probability list include at least one of the probability distribution entropy and probability variance, the larger the probability distribution characteristics of the first prediction probability list, the more the probability distribution of the predicted probabilities in the first prediction probability list tends to be a flat distribution; the smaller the probability distribution characteristics of the first prediction probability list, the more the probability distribution of the predicted probabilities in the first prediction probability list tends to be a sharp distribution.

[0080] For example, when the probability distribution feature of the first predicted probability list includes the sum of the probabilities of the head, the smaller the probability distribution feature of the first predicted probability list, the more gradual the probability distribution of the predicted probabilities in the first predicted probability list tends to be; the larger the probability distribution feature of the first predicted probability list, the more sharp the probability distribution of the predicted probabilities in the first predicted probability list tends to be.

[0081] Based on this, the computer device can determine the probability distribution characteristics of the first predicted probability list i based on the predicted probabilities in the first predicted probability list i, and then dynamically adjust the predicted probabilities in the first predicted probability list i based on the probability distribution characteristics of the first predicted probability list i, thereby adjusting the probability distribution state of the predicted probabilities in the first predicted probability list i to obtain the first adjusted probability list i. Performing the above operation sequentially for N first predicted probability lists yields N first adjusted probability lists. This allows for precise adaptation to the probability distribution states of different first predicted probability lists, enabling individual dynamic adjustments and improving the adjustment accuracy of the first predicted probability lists. Since the probability distribution state corresponding to the first predicted probability list predicted by the sub-language model usually tends towards a sharp distribution state, the probability distribution state of the first predicted probability list can be moderately smoothed using the probability distribution characteristics of the first predicted probability list. It should be noted that one first adjusted probability list corresponds to one word position. Specifically, the first adjusted probability list corresponding to the i-th word position includes the adjusted probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position.

[0082] S103. Determine the probability distribution characteristics corresponding to the predicted probabilities in the N second prediction probability lists respectively. Based on the probability distribution characteristics corresponding to the N second prediction probability lists respectively, adjust the predicted probabilities in the N second prediction probability lists to obtain N second adjusted probability lists.

[0083] It should be noted that the probability distribution characteristics of the second predicted probability list can be used to reflect the probability distribution state of the predicted probabilities in the second predicted probability list. The probability distribution state of the second predicted probability list can include a sharp distribution state, a stationary distribution state, etc.; wherein, a sharp distribution state indicates that the difference between the predicted probabilities in the second predicted probability list is greater than or equal to a difference threshold; a stationary distribution state indicates that the difference between the predicted probabilities in the second predicted probability list is less than a difference threshold.

[0084] The probability distribution features of the second predicted probability list may include at least one of the following: probability distribution entropy, probability variance, and head probability sum (the sum of the k highest predicted probabilities in the second predicted probability list). The correspondence between the probability distribution features of the second predicted probability list and the probability distribution state can be determined based on the specific content of the probability distribution features of the second predicted probability list.

[0085] For example, when the probability distribution characteristics of the second prediction probability list include at least one of the probability distribution entropy and probability variance, the larger the probability distribution characteristics of the second prediction probability list, the more the probability distribution of the predicted probabilities in the second prediction probability list tends to be a flat distribution; the smaller the probability distribution characteristics of the second prediction probability list, the more the probability distribution of the predicted probabilities in the second prediction probability list tends to be a sharp distribution.

[0086] For example, when the probability distribution feature of the second predicted probability list includes the sum of the probabilities of the head, the smaller the probability distribution feature of the second predicted probability list, the more gradual the probability distribution of the predicted probabilities in the second predicted probability list tends to be; the larger the probability distribution feature of the second predicted probability list, the more sharp the probability distribution of the predicted probabilities in the second predicted probability list tends to be.

[0087] Based on this, the computer device can determine the probability distribution characteristics of the second predicted probability list i based on the predicted probabilities in the second predicted probability list i, and then dynamically adjust the predicted probabilities in the second predicted probability list i based on the probability distribution characteristics of the second predicted probability list i, thereby adjusting the probability distribution state of the predicted probabilities in the second predicted probability list i to obtain the second adjusted probability list i. Performing the above operation sequentially for N second predicted probability lists yields N second adjusted probability lists. This allows for precise adaptation to the probability distribution states of different second predicted probability lists, enabling individual dynamic adjustments and improving the accuracy of the adjustment of the second predicted probability lists. Furthermore, since the probability distribution state corresponding to the second predicted probability list predicted by the main language model usually tends to be a flat distribution state, the probability distribution state of the second predicted probability list can be appropriately sharpened based on its probability distribution characteristics.

[0088] This makes the probability distribution characteristics corresponding to the adjusted probabilities in the first adjusted probability list closer to the probability distribution characteristics corresponding to the adjusted probabilities in the second adjusted probability list, effectively solving the problem that the distribution of the predicted probability list predicted by the main language model is relatively flat, while the distribution of the predicted probability list predicted by the sub-language model is relatively sharp.

[0089] It should be noted that a second adjustment probability list corresponds to a word position. Specifically, the second adjustment probability list corresponding to the i-th word position includes the adjustment probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position.

[0090] S104. Based on the N first adjustment probability lists, the N second adjustment probability lists, and the word list, determine the text content that satisfies the prompt word information.

[0091] In this application, by dynamically aligning and adjusting the predicted probabilities in the first and second prediction probability lists, the probability distribution features corresponding to the adjusted probabilities in the first and second adjustment probability lists become closer to those in the second adjustment probability lists. In other words, the difference between the probability distribution features corresponding to the adjusted probabilities in the first and second adjustment probability lists is relatively small. Then, based on N first adjustment probability lists, N second adjustment probability lists, and a word list, the text content that satisfies the prompt word information is determined. This helps increase the probability that the candidate word sequence predicted by the sub-language model (i.e., the word sequence obtained based on the first adjustment probability list) is accepted by the main language model, thereby improving the efficiency of text content generation.

[0092] Specifically, the computer device can select the word to be verified at position j from the word list based on the third probability list j, and verify the word to be verified at position j using both the third and fourth probability lists j. If the word to be verified at position j passes verification, i.e., the word to be verified at position j is accepted, then the word to be verified at position j is determined as the predicted word at position j. Next, the computer device can select the word to be verified at position j+1 from the word list based on the third probability list j+1, and verify the word to be verified at position j+1 using both the third and fourth probability lists j+1. If the word to be verified at position j passes verification, i.e., the word to be verified at position j+1 is accepted, then the word to be verified at position j+1 is determined as the predicted word at position j+1.

[0093] Once the predicted words corresponding to the N word positions are obtained, these predicted words can be arranged to obtain the initial text content. The initial text content includes the predicted words corresponding to the N word positions. If the initial text content conforms to the constraints in the prompt word information, it is determined as the text content that satisfies the prompt word information, meaning the text content includes the predicted words corresponding to the N word positions.

[0094] It should be noted that the constraint information in the prompt word information may include lexical quantity constraint information.

[0095] Among them, the word count constraint information can refer to the range of word counts that limits the number of predicted words included in the text content (such as the initial text content) predicted by the computer device.

[0096] It should be noted that the initial text content conforming to the constraint information in the prompt word information can mean that the number of predicted words included in the initial text content falls within that range. For example, if the range of predicted words is [100, 120], and the number of predicted words included in the initial text content is 114, then the initial text content conforms to the constraint information in the prompt word information.

[0097] In one embodiment, when the lexical unit to be verified at the (j+1)th lexical unit position fails to pass verification, the computer device can continue to generate the predicted lexical units corresponding to the last j lexical unit positions from the N lexical unit positions based on the predicted lexical units, prompt word information, sub-language model, and main language model corresponding to the first j lexical unit positions, and arrange the predicted lexical units corresponding to the N lexical unit positions to obtain the text content that satisfies the prompt word information.

[0098] In one embodiment, after obtaining the predicted words corresponding to N word positions, the predicted words corresponding to the N word positions are arranged to obtain a first initial text content. If the first initial text content does not meet the constraints in the prompt word information, the first initial text content can be added to the prompt word information. Based on the added prompt word information, the sub-language model, and the main language model, a second initial text content that meets the added prompt word information is generated. The first and second initial text contents are then arranged to obtain text content that meets the prompt word information. By predicting the predicted probability list corresponding to each word position in the second initial text content based on the first initial text content, the opportunity for adaptive optimization of the sampling process based on context (i.e., the predicted words at adjacent word positions in the entire text content) is increased.

[0099] Please refer to Figure 4b , Figure 4b A data processing scenario provided in the embodiments of this application Figure 2 , Figure 4b It includes a sub-language model 44, a main language model 45, and a text validation component 47.

[0100] Specifically, the prompt word information 43 is input into the sub-language model 44, which yields a first prediction probability list set 441, containing first prediction probability lists corresponding to N word positions. The first prediction probability list set 441 and the prompt word information 43 are then input into the main language model 45, which yields a second prediction probability list set 451, containing second prediction probability lists corresponding to N word positions. The prediction process for the first and second prediction probability lists can be referred to step S101 above, and will not be elaborated here. Next, the prediction probabilities in the N first prediction probability lists within the first prediction probability list set 441 are dynamically adjusted to obtain a first adjusted probability list set 442, which contains first adjusted probability lists corresponding to N word positions. The predicted probabilities in the N second predicted probability lists in the second predicted probability list set 451 are dynamically adjusted to obtain the second adjusted probability list set 452, which includes the second adjusted probability lists corresponding to the N word positions. The process of determining the first and second adjusted probability lists can refer to steps S102 and S103 above, and will not be repeated here.

[0101] Furthermore, the computer device can select predicted words corresponding to the N word positions from the word list, based on a first adjustment probability list and a second adjustment probability list corresponding to the N word positions respectively. For example, Figure 4b In the text, predicted word A1 is the predicted word at the first word position, predicted word A2 is the predicted word at the second word position, and so on, with predicted word An being the predicted word at the nth word position. The predicted words corresponding to the N word positions are arranged to obtain the initial text content. This initial text content is then input into the text verification component 47. The text verification component 47 can obtain the prompt word information 43, extract the word count range from the prompt word information 43, and analyze the initial text content to obtain the number of words in the initial text content.

[0102] Next, if the number of words in the initial text content falls within the specified range, the initial text content verification is successful. The text verification component 47 then determines the initial text content as text content 48 that satisfies the prompt word information 43 and outputs text content 48. If the number of words in the initial text content exceeds the specified range, the text verification component 47 optimizes the initial text content so that the number of words in the optimized initial text content falls within the specified range. In other words, the optimized initial text content verification is successful, and the optimized initial text content is determined as text content 48 that satisfies the prompt word information 43, and text content 48 is output. If the number of words in the initial text content is less than the specified range, the initial text content verification is unsuccessful. The text verification component 47 then inputs the initial text content into the model prediction process to continue text content prediction until the predicted text content passes verification. In this application, the prediction probabilities in N first predicted probability lists are dynamically adjusted based on the probability distribution features of the N first predicted probability lists obtained by the sub-language model, resulting in N first adjusted probability lists. Similarly, the prediction probabilities in N second predicted probability lists are dynamically adjusted based on the probability distribution features of the N second predicted probability lists obtained by the main language model, resulting in N second adjusted probability lists. This approach helps to make the probability distribution features corresponding to the adjusted probabilities in the first adjusted probability lists closer to those in the second adjusted probability lists. In other words, the difference between the probability distribution features corresponding to the adjusted probabilities in the first and second adjusted probability lists is relatively small. This increases the probability that the candidate word sequences predicted by the sub-language model (i.e., word sequences obtained based on the first adjusted probability lists) are accepted by the main language model, thereby improving the efficiency of text content generation.

[0103] Please see Figure 5 , Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 .like Figure 5 As shown, this method can be derived from... Figure 1 It can be executed by any terminal in the system, or by... Figure 1 The server in the middle can be used to execute it, or it can be executed by... Figure 1 The terminal and server in the process work together to execute the data processing method. The device used to execute this data processing method in this application can be collectively referred to as a computer device. The method may include the following steps:

[0104] S501. Based on the prompt word information, predict the N first prediction probability lists corresponding to the N word positions using the sub-language model. Based on the prompt word information and the N first prediction probability lists, predict the N second prediction probability lists corresponding to the N word positions using the main language model.

[0105] S502. Sort the predicted probabilities in the first predicted probability list i according to the preset sorting rules to obtain the first probability sequence i.

[0106] It should be noted that the preset sorting rules can be the default rules in the computer device or rules set manually; preset sorting rules can include ascending sorting rules, descending sorting rules, etc.

[0107] It should be noted that the first predicted probability list i mentioned above can refer to the first predicted probability list corresponding to the i-th word position.

[0108] In this application, the computer device can obtain a preset sorting rule stored locally, and sort each predicted probability in the first predicted probability list i according to the preset sorting rule, so that the relationship between the sorted predicted probabilities satisfies the preset sorting rule, thereby obtaining the first probability sequence i.

[0109] S503. Extract the K predicted probabilities that are first in the order of the first probability sequence i.

[0110] In this application, the computer device can extract the first K predicted probabilities starting from the first position of the sorting of the first probability sequence i.

[0111] It should be noted that K can be an integer greater than 1 and less than P, P can be the number of predicted probabilities in the first probability sequence i, and P can also be the number of candidate lexical units in the lexical unit list.

[0112] S504. Based on the K predicted probabilities, determine the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list i.

[0113] In this application, a computer device can determine the probability distribution characteristics among the K predicted probabilities based on their probability distribution states, and then define these characteristics as the probability distribution characteristics of the first predicted probability list i. Determining the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list using only a subset of probabilities from the first probability sequence helps reduce the computational complexity of the probability distribution characteristics and improves computational speed.

[0114] In one embodiment, determining the probability distribution characteristics corresponding to the predicted probabilities in the first prediction probability list i based on K predicted probabilities may include: the computer device performing reciprocal or negative logarithmic processing on the K predicted probabilities to obtain the uncertainty of the candidate word units corresponding to the K predicted probabilities; and averaging the uncertainty of the candidate word units corresponding to the K predicted probabilities to obtain the probability distribution entropy among the K predicted probabilities. The computer device can measure the probability distribution entropy of the first prediction probability list i based on the probability distribution entropy among the K predicted probabilities; and determine the probability distribution entropy of the first prediction probability list i as the probability distribution characteristics corresponding to the predicted probabilities in the first prediction probability list i. Quantifying the distribution characteristics of the predicted probabilities in the probability list using the probability distribution entropy value is beneficial for improving the adjustment accuracy of the probability list.

[0115] It should be noted that the uncertainty of candidate words can be used to measure the prediction probability that the candidate word belongs to the i-th word position as predicted by the language model (here, the sub-language model). Among them, the uncertainty of candidate words is negatively correlated with the prediction probability of candidate words. That is, the higher the prediction probability of candidate words, the lower the uncertainty of candidate words; the lower the prediction probability of candidate words, the higher the uncertainty of candidate words.

[0116] It should be noted that the probability distribution entropy of the first predicted probability list i can be used to reflect the probability distribution state of the predicted probabilities in the first predicted probability list i; specifically, the larger the probability distribution entropy of the first predicted probability list i, the more the probability distribution state of the predicted probabilities in the first predicted probability list i tends to be a flat distribution state, and the smaller the probability distribution entropy of the first predicted probability list i, the more the probability distribution state of the predicted probabilities in the first predicted probability list i tends to be a sharp distribution state.

[0117] In one embodiment, a computer device can input K predicted probabilities into a probability distribution entropy formula and calculate the probability distribution entropy of the first predicted probability list i based on the probability distribution entropy formula.

[0118] It should be noted that the probability distribution entropy formula is shown in formula (1).

[0119]

[0120] Where H is the probability distribution entropy of the first predicted probability list i, and P a Let logP be the a-th prediction probability out of K prediction probabilities. a Let be the uncertainty of the candidate word corresponding to the a-th predicted probability out of K predicted probabilities. Here, a is a positive integer less than or equal to K.

[0121] S505. Based on the probability distribution characteristics corresponding to the N first predicted probability lists, adjust the predicted probabilities in the N first predicted probability lists to obtain N first adjusted probability lists.

[0122] In one embodiment, step S505 may include the following steps: the computer device can obtain initial probability adjustment parameters from local memory; based on the probability distribution characteristics corresponding to the first predicted probability list i, the parameter size of the initial probability adjustment parameters is adjusted to obtain the probability adjustment parameters corresponding to the first predicted probability list i. Then, based on the probability adjustment parameters corresponding to the first predicted probability list i, the computer device can adjust some or all of the predicted probabilities in the first predicted probability list i to adjust the distribution characteristics of the predicted probabilities in the first predicted probability list i, thus obtaining the first adjusted probability list i. Using the initial probability adjustment parameters as the basis for adjustment, combined with the probability distribution characteristics of the first predicted probability list, dynamically adjusting the first predicted probability list, while ensuring processing standardization, helps to improve the flexibility and accuracy of probability list adjustment.

[0123] It should be noted that the initial probability adjustment parameter is used to provide a basic reference value for adjusting the predicted probabilities in the probability list. The initial probability adjustment parameter can be a learnable parameter, and the specific learning process can be found in the description below.

[0124] In one embodiment, the process of determining the probability adjustment parameter corresponding to the first predicted probability list i may include: the computer device can obtain the entropy scaling factor and normalization parameter from the probability adjustment parameter formula from local memory, multiply the probability distribution feature, entropy scaling factor, and normalization parameter corresponding to the first predicted probability list i to obtain the normalized probability distribution feature. The computer device can perform exponential processing on the normalized probability distribution feature to obtain the probability adjustment weight corresponding to the first predicted probability list i, and then adjust the initial probability adjustment parameter using the probability adjustment weight corresponding to the first predicted probability list i, that is, multiply the probability adjustment weight corresponding to the first predicted probability list i and the initial probability adjustment parameter to obtain the probability adjustment parameter corresponding to the first predicted probability list i. Calculating the probability adjustment parameter based on the probability distribution feature using the probability adjustment parameter formula is beneficial to improving the calculation efficiency and accuracy of the probability adjustment parameter.

[0125] It's important to note that the entropy scaling factor can be used as an indicator to quantify the relationship between probability distribution characteristics and probability adjustment weights. A larger entropy scaling factor means that small changes in the probability distribution characteristics will trigger more significant changes in the probability adjustment weights; in other words, the probability distribution characteristics are more sensitive to changes in the probability adjustment weights. Conversely, a smaller entropy scaling factor means that small changes in the probability distribution characteristics will not trigger more significant changes in the probability adjustment weights; that is, the change in probability adjustment weights is more gradual, and a larger probability distribution characteristic is required to cause a noticeable change in the probability adjustment weights.

[0126] It should be noted that the probability adjustment parameter formula is shown in formula (2).

[0127] T=T0*exp(θ*H*logN) (2)

[0128] Where T is the probability adjustment parameter, T0 is the initial probability adjustment parameter, θ is the entropy scaling factor, H is the probability distribution feature, and logN is the normalization parameter. “θ*H*logN” means multiplying the probability distribution feature, entropy scaling factor, and normalization parameter to obtain the normalized probability distribution feature; “exp(θ*H*logN)” means exponentially processing the normalized probability distribution feature to obtain the probability adjustment weight.

[0129] It should be noted that the initial probability adjustment parameter, entropy scaling factor, and normalization parameter can all be learnable parameters.

[0130] In one embodiment, the process of determining the probability adjustment parameter formula, i.e., the learning process of the initial probability adjustment parameter, entropy scaling factor, and normalization parameter, may include: the computer device acquiring the initial probability adjustment parameter formula, which includes preset initial probability adjustment parameters, entropy scaling factor, and normalization parameter. The computer device can predict S first sample prediction probability lists corresponding to S sample word positions based on sample prompt word information using a sub-language model, and predict S second sample prediction probability lists corresponding to the S sample word positions using a main language model based on the aforementioned sample prompt word information and the aforementioned S first sample prediction probability lists. Wherein, the first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position; S is a positive integer; i is a positive integer less than or equal to S.

[0131] Furthermore, using the initial probability adjustment parameter formula, the probability distribution characteristics corresponding to the predicted probabilities in the S first sample prediction probability lists are determined. Based on these probability distribution characteristics, the predicted probabilities in the S first sample prediction probability lists are adjusted to obtain S first sample adjusted probability lists. Similarly, using the initial probability adjustment parameter formula, the probability distribution characteristics corresponding to the predicted probabilities in the S second sample prediction probability lists are determined. Based on these probability distribution characteristics, the predicted probabilities in the S second sample prediction probability lists are adjusted to obtain S second sample adjusted probability lists. Finally, based on the S first sample adjusted probability lists, the S second sample adjusted probability lists, and the aforementioned word list, the sample text content that satisfies the aforementioned sample prompt word information is determined.

[0132] Next, the probability distribution feature differences between the first and second sample adjusted probability lists corresponding to the S sample word positions are determined, along with the text quality score of the sample text content. Based on these probability distribution feature differences and text quality scores, the preset initial probability adjustment parameters, entropy scaling factor, and normalization parameters are adjusted to obtain the adjusted initial probability adjustment parameters, entropy scaling factor, and normalization parameters. Replacing the preset initial probability adjustment parameters, entropy scaling factor, and normalization parameters in the initial probability adjustment parameter formula with the adjusted initial probability adjustment parameters, entropy scaling factor, and normalization parameters yields the probability adjustment parameter formula.

[0133] In one embodiment, the computer device can perform exponential processing on each predicted probability in the first predicted probability list i according to the probability adjustment parameters corresponding to the first predicted probability list i, to obtain the processed predicted probability corresponding to the first predicted probability list i. Then, it can perform normalization processing on the processed predicted probability corresponding to the first predicted probability list i to obtain the adjusted probability corresponding to the first predicted probability list i. Based on the adjusted probability corresponding to the first predicted probability list i, a first adjusted probability list i is generated. The normalization processing ensures that the sum of the adjusted probabilities in the first adjusted probability list is 1, conforming to the basic mathematical constraints of the probability list and maintaining statistical rigor. Since the exponential processing and normalization processing do not change the relative magnitudes of the predicted probabilities in the first predicted probability list i, i.e., the relative magnitudes of the predicted probabilities in the first predicted probability list i are consistent with the relative magnitudes of the adjusted probabilities in the first adjusted probability list i, the device can always prioritize selecting candidate words with high probabilities as the words to be verified. This avoids the situation where low-probability candidate words are incorrectly selected due to probability adjustment errors, thereby maintaining the text quality of the generated text content.

[0134] It should be noted that the first adjustment probability list i may include the adjustment probability of each candidate word in the word list belonging to the predicted word at the i-th word position.

[0135] S506. Determine the probability distribution characteristics corresponding to the predicted probabilities in the N second prediction probability lists respectively. Based on the probability distribution characteristics corresponding to the N second prediction probability lists respectively, adjust the predicted probabilities in the N second prediction probability lists to obtain N second adjusted probability lists.

[0136] In one embodiment, step S506 may include: a computer device acquiring a locally stored preset sorting rule, and sorting each predicted probability in the second predicted probability list i according to the preset sorting rule, so that the relationship between the sorted predicted probabilities satisfies the preset sorting rule, thereby obtaining a second probability sequence i. The computer device may extract the first Q predicted probabilities from the first position of the second probability sequence i, determine the probability distribution characteristics between the Q predicted probabilities based on their probability distribution states, and define these probability distribution characteristics as the probability distribution characteristics of the second predicted probability list i. By extracting a portion of the probabilities from the second probability sequence to determine the probability distribution characteristics corresponding to the predicted probabilities in the second predicted probability list, the computational complexity of the probability distribution characteristics is reduced, and the computation speed is improved.

[0137] It should be noted that Q can be an integer greater than 1 and less than P, where P can be the number of predicted probabilities in the second probability sequence i, and the number of candidate lexical units in the lexical unit list.

[0138] It should be noted that the second predicted probability list i can refer to the second predicted probability list corresponding to the i-th word position.

[0139] In one embodiment, determining the probability distribution feature corresponding to the predicted probabilities in the second prediction probability list i based on the Q predicted probabilities may include: the computer device performing reciprocal or negative logarithmic processing on the Q predicted probabilities to obtain the uncertainty of the candidate word units corresponding to the Q predicted probabilities; and averaging the uncertainty of the candidate word units corresponding to the Q predicted probabilities to obtain the probability distribution entropy among the Q predicted probabilities. The computer device can measure the probability distribution entropy of the second prediction probability list i based on the probability distribution entropy among the Q predicted probabilities; and determine the probability distribution entropy of the second prediction probability list i as the probability distribution feature corresponding to the predicted probabilities in the second prediction probability list i.

[0140] It should be noted that the uncertainty of candidate lexical units can be used to measure the predicted probability that a candidate lexical unit belongs to the i-th lexical unit position as predicted by the language model (here, the main language model).

[0141] It should be noted that the probability distribution entropy of the second predicted probability list i can be used to reflect the probability distribution state of the predicted probabilities in the second predicted probability list i; specifically, the larger the probability distribution entropy of the second predicted probability list i, the more the probability distribution state of the predicted probabilities in the second predicted probability list i tends to be a flat distribution state, and the smaller the probability distribution entropy of the second predicted probability list i, the more the probability distribution state of the predicted probabilities in the second predicted probability list i tends to be a sharp distribution state.

[0142] In one embodiment, the computer device can input Q predicted probabilities into the probability distribution entropy formula, i.e., formula (1), to calculate the probability distribution entropy of the second predicted probability list i based on the probability distribution entropy formula.

[0143] In one embodiment, the computer device can retrieve initial probability adjustment parameters from local memory; based on the probability distribution characteristics corresponding to the second predicted probability list i, the parameter size of the initial probability adjustment parameters is adjusted to obtain the probability adjustment parameters corresponding to the second predicted probability list i. Then, based on the probability adjustment parameters corresponding to the second predicted probability list i, the computer device can adjust some or all of the predicted probabilities in the second predicted probability list i to adjust the distribution characteristics of the predicted probabilities in the second predicted probability list i, thus obtaining the second adjusted probability list i. Using the initial probability adjustment parameters as the basis for adjustment, combined with the probability distribution characteristics of the second predicted probability list, dynamically adjusting the second predicted probability list, while ensuring processing standardization, helps to improve the flexibility and accuracy of probability list adjustment.

[0144] In one embodiment, the computer device can retrieve the entropy scaling factor and normalization parameter from the probability adjustment parameter formula in local memory, multiply the probability distribution feature corresponding to the second predicted probability list i, the entropy scaling factor, and the normalization parameter to obtain the normalized probability distribution feature. The computer device can then perform exponential processing on the normalized probability distribution feature to obtain the probability adjustment weight corresponding to the second predicted probability list i. Using this probability adjustment weight, the initial probability adjustment parameter is adjusted; that is, the probability adjustment weight and the initial probability adjustment parameter are multiplied to obtain the probability adjustment parameter corresponding to the second predicted probability list i. Next, based on the probability adjustment parameter corresponding to the first predicted probability list i, the computer device can adjust some or all of the predicted probabilities in the first predicted probability list i to adjust the distribution feature of the predicted probabilities in the first predicted probability list i, thus obtaining the first adjusted probability list i. Using the initial probability adjustment parameter as the adjustment basis, combined with the probability distribution feature of the second predicted probability list, dynamically adjusts the second predicted probability list. This ensures processing standardization while improving the flexibility and accuracy of probability list adjustments.

[0145] It should be noted that the second adjustment probability list i package may include the adjustment probability of each candidate word in the word list belonging to the predicted word at the i-th word position.

[0146] S507. Based on the N first adjustment probability lists, the N second adjustment probability lists, and the word list, determine the text content that satisfies the prompt word information.

[0147] In one embodiment, step S507 may include: the computer device sorting the adjustment probabilities in the first adjustment probability list i to obtain a third probability sequence i; selecting the candidate word corresponding to the adjustment probability that is first in the third probability sequence i from the word list, and using it as the first optimal word at the i-th word position; performing the above operation sequentially for N first adjustment probability lists to obtain the first optimal word corresponding to each of the N word positions. The computer device may sort the adjustment probabilities in the second adjustment probability list i to obtain a fourth probability sequence i; selecting the candidate word corresponding to the adjustment probability that is first in the fourth probability sequence i from the word list, and using it as the second optimal word at the i-th word position; performing the above operation sequentially for N second adjustment probability lists to obtain the second optimal word corresponding to each of the N word positions. The computer device can determine the text content that satisfies the prompt word information based on the first and second optimal word corresponding to each of the N word positions. The process of determining the first and second optimal lexical units is the sampling process mentioned above. The sampling process is based on the first and second adjusted probability lists, which realizes the intelligent coupling between the sampling process and the adjustment process of the probability lists.

[0148] Among them, the first optimal word at the i-th word position is the candidate word corresponding to the maximum adjustment probability in the first adjustment probability list at the i-th word position.

[0149] Among them, the second optimal word at the i-th word position is the candidate word corresponding to the maximum adjustment probability in the second adjustment probability list at the i-th word position.

[0150] In one embodiment, determining the text content that satisfies the prompt word information based on the first and second optimal lexical units corresponding to the N lexical units positions includes: the computer device sequentially comparing the first and second optimal lexical units corresponding to the N lexical units positions; when the first and second optimal lexical units at the j-th lexical unit position are different, i.e., the first and second optimal lexical units do not belong to the same lexical unit, the first optimal lexical units corresponding to the preceding j-1 lexical units positions for which the comparison operation has been performed are determined as the predicted lexical units corresponding to the preceding j-1 lexical units positions; where j is a positive integer greater than 1 and less than or equal to N. The computer device can determine the second optimal lexical unit at the j-th lexical unit position as the predicted lexical unit at the j-th lexical unit position, and arrange the predicted lexical units corresponding to the preceding j lexical units positions for which the comparison operation has been performed to obtain the first initial text content; if the first initial text content satisfies the constraint information in the prompt word information, then the first initial text content can be determined as the text content that satisfies the prompt word information.

[0151] If the first initial text content does not meet the constraints in the prompt word information, predicted words for the unaligned word positions in the N word positions can be generated based on the first initial text content, the prompt word information, the sub-language model, and the main language model. The first initial text content and the predicted words for the unaligned word positions in the N word positions are then arranged to obtain the second initial text content. If the second initial text content meets the constraints in the prompt word information, it is determined to be the text content that meets the prompt word information. If the second initial text content does not meet the constraints in the prompt word information, text content can be generated based on the second initial text content, the prompt word information, the sub-language model, and the main language model until the generated text content meets the constraints in the prompt word information. The text content that meets the constraints in the prompt word information is then determined to be the text content that meets the prompt word information.

[0152] It should be noted that the first optimal word element and the second optimal word element not belonging to the same word element may include any of the following situations: 1. The word element similarity between the first optimal word element and the second optimal word element at the j-th word element position is less than the word element similarity threshold; 2. The word element identifier of the first optimal word element at the j-th word element position in the word element list is different from the word element identifier of the second optimal word element at the j-th word element position in the word element list.

[0153] The word similarity threshold can be a default value in the computer device, or it can be manually set.

[0154] In one embodiment, when the computer device finds that the first optimal word corresponding to each of the N word positions is the same as the corresponding second optimal word, that is, when the first optimal word corresponding to each of the N word positions belongs to the same word, the first optimal word corresponding to each of the N word positions is determined as the predicted word corresponding to each of the N word positions; the predicted word corresponding to each of the N word positions is arranged to obtain the initial text content. If the initial text content satisfies the constraint information in the prompt word information, the initial text content can be determined as the text content that satisfies the prompt word information.

[0155] It should be noted that the first optimal word and the second optimal word belonging to the same word can include any of the following situations: 1. The word similarity between the first optimal word and the second optimal word at the j-th word position is greater than or equal to the word similarity threshold; 2. The word identifier of the first optimal word at the j-th word position in the word list is the same as the word identifier of the second optimal word at the j-th word position in the word list.

[0156] In one embodiment, after the computer device obtains the predicted words corresponding to the N word positions, the predicted words corresponding to the N word positions can be arranged to obtain a first initial text content. The first initial text content can be added to the prompt word information. Based on the added prompt word information, the sub-language model, and the main language model, a second initial text content that satisfies the added prompt word information is generated. The first initial text content and the second initial text content are arranged to obtain text content that satisfies the prompt word information.

[0157] In one embodiment, the computer device can sort the candidate words in the word list according to a descending order, based on the adjustment probabilities corresponding to each candidate word in the first adjustment probability list i, to obtain the i-th word sequence. The adjustment probability of the candidate word at the beginning of the i-th word sequence in the first adjustment probability list i is greater than the adjustment probability of the candidate word at the end of the i-th word sequence in the first adjustment probability list i. The computer device can perform sequence truncation on the i-th word sequence to obtain the i-th word subsequence. Until the computer device obtains N word subsequences, it can filter the predicted word at the i-th word position from the i-th word subsequence according to the first adjustment probability list i and the second adjustment probability list i. This process continues until N predicted word positions are obtained. The predicted word positions are then arranged to obtain the initial text content. If this initial text content satisfies the constraint information in the prompt word information, it can be determined as text content that satisfies the prompt word information.

[0158] It should be noted that the i-th lexical subsequence may include at least two candidate lexicals from the lexical list.

[0159] It should be noted that the first adjustment probability list i can refer to the first adjustment probability list corresponding to the i-th word position.

[0160] In one embodiment, the above-mentioned selection of the predicted word at the i-th word position from the i-th word subsequence according to the first adjustment probability list i and the second adjustment probability list i includes: the computer device can sequentially traverse the word subsequences corresponding to the N word positions respectively; when traversing to the j-th word subsequence corresponding to the j-th word position, the computer device can obtain the adjustment probability of the candidate word in the j-th word subsequence from the first adjustment probability list j, obtain the adjustment probability of the corresponding candidate word from the second adjustment probability list j, and perform a ratio operation on the two obtained adjustment probabilities to obtain the probability ratio of the corresponding candidate word.

[0161] Next, when there is a candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold, the candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold is determined as the predicted word at the j-th word position; the word subsequence corresponding to the (j+1)-th word position is traversed until the predicted words corresponding to the N word positions are obtained; the predicted words corresponding to the N word positions are arranged to obtain the initial text content. If the initial text content satisfies the constraint information in the prompt word information, the initial text content can be determined as the text content that satisfies the prompt word information.

[0162] The probability ratio threshold can be a default value in the computer equipment, or it can be manually set.

[0163] It should be noted that the second adjustment probability list i can refer to the second adjustment probability list corresponding to the i-th word position.

[0164] In one embodiment, when the probability ratios corresponding to candidate words in the j-th word subsequence are all less than a probability ratio threshold, the operation of traversing the word subsequences corresponding to the N word positions is paused. The computer device can select the candidate word corresponding to the maximum adjustment probability in the second adjustment probability list j from the word list and use it as the predicted word at the j-th word position. The computer device can arrange the predicted words corresponding to the first j word positions that have undergone traversal to obtain the first initial text content. If the first initial text content satisfies the constraint information in the prompt word information, then the first initial text content can be determined as the text content that satisfies the prompt word information.

[0165] If the first initial text content does not meet the constraints in the prompt word information, predicted words for the untraversed word positions out of N word positions can be generated based on the first initial text content, prompt word information, sub-language model, and main language model. The first initial text content and the predicted words for the untraversed word positions out of the N word positions are then arranged to obtain the second initial text content. If the second initial text content meets the constraints in the prompt word information, it is determined to be text content that meets the prompt word information. If the second initial text content does not meet the constraints in the prompt word information, text content can continue to be generated based on the second initial text content, prompt word information, sub-language model, and main language model until the generated text content meets the constraints in the prompt word information. The text content that meets the constraints in the prompt word information is then determined to be text content that meets the prompt word information.

[0166] In this application, the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list i are determined based on K predicted probabilities in the first predicted probability list. This means that only a subset of probabilities in the first probability sequence needs to be analyzed to determine the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list, which helps reduce the computational complexity of the probability distribution characteristics and improves computational speed. Furthermore, based on the probability distribution characteristics of the N first predicted probability lists obtained through the sub-language model, the predicted probabilities in the N first predicted probability lists are dynamically adjusted to obtain N first adjusted probability lists; similarly, based on the probability distribution characteristics of the N second predicted probability lists obtained through the main language model, the predicted probabilities in the N second predicted probability lists are dynamically adjusted to obtain N second adjusted probability lists. This helps to make the probability distribution features corresponding to the adjusted probabilities in the first adjusted probability list closer to the probability distribution features corresponding to the adjusted probabilities in the second adjusted probability list. That is, the difference between the probability distribution features corresponding to the adjusted probabilities in the first adjusted probability list and the probability distribution features corresponding to the adjusted probabilities in the second adjusted probability list is relatively small. This helps to increase the probability that the candidate word sequence predicted by the sub-language model (i.e., the word sequence obtained based on the first adjusted probability list) is accepted by the main language model, thereby improving the efficiency of text content generation.

[0167] Please refer to Figure 6 , Figure 6 This is a flowchart illustrating a data processing method provided in this application. Figure 3 ,like Figure 6 As shown, Figure 6 It includes a main language model 60, a sub-language model 61, and a sub-language model 62. The output of the sub-language model is the input of the main language model.

[0168] Specifically, in the process of generating text content, the computer device first uses the sub-language model 61 to predict the first predicted probability list of the first word position (i.e., based on the prompt word information) using the sub-language model 61. Figure 6 The prompt word information and the first prediction probability list C1 are input into the sub-language model 62. The sub-language model 62 then predicts the first prediction probability list for the second word position (i.e., the first prediction probability list C1 in the first prediction probability list C1). Figure 6 The first predicted probability list C2 is input into the main language model 60 along with the prompt word information and the first predicted probability list C1. The main language model 60 then predicts the second predicted probability list (i.e., the first predicted probability list C2) for the position of the first word unit. Figure 6 The second prediction probability list D1 in the first prediction probability list C1 and the first prediction probability list C2 are input into the main language model 60, and the main language model 60 predicts the second prediction probability list (i.e., the second prediction probability list D1) for the second word position. Figure 6 The second predicted probability list D2 in the table.

[0169] The computer device can input the first predicted probability list C1, the first predicted probability list C2, the second predicted probability list D1, and the second predicted probability list D2 into the rejection sampling component 63; the rejection sampling component 63 will perform the following steps based on the first and second predicted probability lists for each word position:

[0170] S61. Determine the probability distribution characteristics of the predicted probability list.

[0171] In this application, the computer device can target the first predicted probability list C1 and the first predicted probability list.

[0172] C2, the second prediction probability list D1 and the second prediction probability list D2, respectively execute the steps described above.

[0173] The operations from S502 to S504 can obtain the probability distribution features corresponding to the first predicted probability list C1, the first predicted probability list C2, the second predicted probability list D1, and the second predicted probability list D2 for the first word position.

[0174] S62. Based on the probability distribution characteristics corresponding to the predicted probability list, determine the probability adjustment parameters corresponding to the predicted probability list.

[0175] In this application, taking the first predicted probability list C1 as an example, the computer device can obtain an initial probability adjustment parameter, and adjust the initial probability adjustment parameter according to the probability distribution characteristics corresponding to the first predicted probability list C1 to obtain the probability adjustment parameter corresponding to the first predicted probability list C1. The above operation is repeated sequentially for the first predicted probability list C2, the second predicted probability list D1, and the second predicted probability list D2 to obtain the probability adjustment parameters corresponding to the first predicted probability list C2, the second predicted probability list D1, and the second predicted probability list D2, respectively.

[0176] S63. Adjust the predicted probability list based on the probability adjustment parameters corresponding to the predicted probability list.

[0177] In this application, the computer device can perform the operation of step S505 above on the first prediction probability list C1, the first prediction probability list C2, the second prediction probability list D1, and the second prediction probability list D2 respectively, to obtain the first adjustment probability list E1 and the second adjustment probability list F1 for the first word position, and the first adjustment probability list E2 and the second adjustment probability list F2 for the second word position.

[0178] S64. Based on the adjusted probability list of word positions, determine the predicted word for each word position.

[0179] In this application, the computer device can select the predicted word at the first word position from the word list according to the first adjustment probability list E1 and the second adjustment probability list F1. If the computer device successfully selects the predicted word at the first word position, the computer device can select the predicted word at the second word position from the word list according to the first adjustment probability list E2 and the second adjustment probability list F2.

[0180] Next, the rejection sampling component 63 can arrange the predicted lemmas at the first lemma position and the predicted lemmas at the second lemma position to obtain the first initial text content (i.e., Figure 6 The initial text content 64 is input into the sub-language model 61, which predicts the first predicted probability list for the third word position (i.e., ...). Figure 6 The first predicted probability list C3 in the first predicted probability list is input into the sub-language model 62, along with the prompt word information, the initial text content 64, and the first predicted probability list C3. The sub-language model 62 then predicts the first predicted probability list for the fourth word position (i.e., the first predicted probability list C3 in the first predicted probability list C3). Figure 6 The first predicted probability list C4 is then input into the main language model 60, along with the prompt word information, initial text content 64, and the first predicted probability list C3. The main language model then predicts the second predicted probability list (i.e., the second predicted probability list for the third lexical position) through the main language model. Figure 6 The second prediction probability list D3 in the first prediction probability list 64, the prompt word information, the initial text content 64, the first prediction probability list C3 and the first prediction probability list C4 are input into the main language model 60, and the main language model 60 predicts the second prediction probability list (i.e., the second prediction probability list D3) for the fourth word position. Figure 6 The second predicted probability list (D4) in the table.

[0181] The computer device can input the first predicted probability list C3, the first predicted probability list C4, the second predicted probability list D3, and the second predicted probability list D4 into the rejection sampling component 63; based on the rejection sampling component 63, the predicted words corresponding to the third and fourth word positions are determined respectively. The initial text content 64 and the predicted words corresponding to the third and fourth word positions are arranged to obtain the second initial text content. If the text length, text format, etc. of the second initial text content meet the constraints in the prompt word information, then the second initial text content is determined to be the text content that meets the prompt word information.

[0182] This dynamic adjustment process of the entire probability list, as a preprocessing step, can be seamlessly integrated into the rejection sampling verification process while maintaining full compatibility with subsequent sampling processes, which is beneficial for improving the text quality and diversity of the text content. Furthermore, since this dynamic adjustment process can be seamlessly integrated into the rejection sampling verification process, it is also seamlessly integrated into the existing inference pipeline (i.e., the existing model framework). Technical personnel only need to load a lightweight module into the existing model framework to deploy this dynamic adjustment process, which is beneficial for improving the inference speed and response speed of the data processing system when handling language inference with billions of parameters.

[0183] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a data processing device provided in this application. Figure 7 As shown, the data processing apparatus may include:

[0184] The prediction module 711 is used to predict N first prediction probability lists corresponding to N word positions based on prompt word information using a sub-language model, and to predict N second prediction probability lists corresponding to the N word positions using a main language model based on the prompt word information and the N first prediction probability lists. The first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability that each candidate word in the indicator word list belongs to the predicted word at the i-th word position. N is a positive integer; i is a positive integer less than or equal to N.

[0185] The first adjustment module 712 is used to determine the probability distribution characteristics corresponding to the prediction probabilities in the above N first prediction probability lists respectively, and adjust the prediction probabilities in the above N first prediction probability lists according to the probability distribution characteristics corresponding to the above N first prediction probability lists respectively, so as to obtain N first adjusted probability lists.

[0186] The second adjustment module 713 is used to determine the probability distribution characteristics corresponding to the prediction probabilities in the above N second prediction probability lists respectively, and adjust the prediction probabilities in the above N second prediction probability lists according to the probability distribution characteristics corresponding to the above N second prediction probability lists respectively, so as to obtain N second adjusted probability lists.

[0187] The processing module 714 is used to determine the text content that satisfies the above prompt word information based on the above N first adjustment probability lists, the above N second adjustment probability lists and the above word list. The above text content includes the predicted word corresponding to the above N word positions.

[0188] Optionally, the first adjustment module 712 described above can also be used to perform the following operations:

[0189] According to the preset sorting rules, the predicted probabilities in the first predicted probability list i are sorted to obtain the first probability sequence i;

[0190] From the first probability sequence i above, extract the K predicted probabilities that are in the first order; K is an integer greater than 1 and less than P, where P is the number of predicted probabilities in the first probability sequence i above.

[0191] Based on the above K predicted probabilities, determine the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list i.

[0192] Optionally, the first adjustment module 712 described above can also be used to perform the following operations:

[0193] Based on the above K predicted probabilities, determine the uncertainty of the candidate word elements corresponding to the above K predicted probabilities;

[0194] The uncertainty of the candidate words corresponding to the above K predicted probabilities is averaged to obtain the probability distribution entropy of the above first predicted probability list i.

[0195] The probability distribution entropy of the first predicted probability list i is determined as the probability distribution feature corresponding to the predicted probability in the first predicted probability list i.

[0196] Optionally, the first adjustment module 712 described above can also be used to perform the following operations:

[0197] Obtain the initial probability adjustment parameters;

[0198] Based on the probability distribution characteristics corresponding to the first predicted probability list i, the above initial probability adjustment parameters are adjusted to obtain the above probability adjustment parameters corresponding to the first predicted probability list i.

[0199] Based on the probability adjustment parameters corresponding to the first predicted probability list i, the predicted probabilities in the first predicted probability list i are adjusted to obtain the first adjusted probability list i.

[0200] Optionally, the first adjustment module 712 described above can also be used to perform the following operations:

[0201] Multiply the probability distribution features, entropy scaling factor, and normalization parameter corresponding to the first predicted probability list i to obtain the normalized probability distribution features;

[0202] The normalized probability distribution features are subjected to exponential processing to obtain the probability adjustment weights corresponding to the first predicted probability list i.

[0203] The probability adjustment weight corresponding to the first predicted probability list i is multiplied by the initial probability adjustment parameter to obtain the probability adjustment parameter corresponding to the first predicted probability list i.

[0204] Optionally, the first adjustment module 712 described above can also be used to perform the following operations:

[0205] Based on the probability adjustment parameters corresponding to the first predicted probability list i, the predicted probabilities in the first predicted probability list i are normalized to obtain the adjusted probabilities corresponding to the first predicted probability list i.

[0206] Based on the adjusted probabilities corresponding to the first predicted probability list i above, generate the first adjusted probability list i.

[0207] Optionally, the processing module 714 described above can also be used to perform the following operations:

[0208] Based on the above N first adjustment probability lists, the first optimal word corresponding to each of the above N word positions is selected from the above word list; the first optimal word at the i-th word position is the candidate word corresponding to the maximum adjustment probability in the first adjustment probability list at the i-th word position.

[0209] Based on the above N second adjustment probability lists, the second optimal word elements corresponding to the above N word element positions are selected from the above word element list; the second optimal word element at the above i-th word element position is the candidate word element corresponding to the maximum adjustment probability in the above second adjustment probability list at the above i-th word element position;

[0210] Based on the first and second optimal lexical units corresponding to the above N lexical units, the text content that satisfies the above prompt word information is determined.

[0211] Optionally, the processing module 714 described above can also be used to perform the following operations:

[0212] The first optimal word and the second optimal word corresponding to the above N word positions are compared in turn;

[0213] When the first optimal word at the j-th word position is different from the second optimal word, the first optimal word corresponding to the first j-1 word positions is determined as the predicted word corresponding to the first j-1 word positions; j is a positive integer greater than 1 and less than or equal to N;

[0214] The second optimal word at the j-th word position is determined as the predicted word at the j-th word position.

[0215] Based on the predicted lexical units corresponding to the positions of the first j lexical units, as well as the aforementioned sub-language model and the aforementioned main language model, the text content that satisfies the aforementioned prompt word information is determined.

[0216] Optionally, the processing module 714 described above can also be used to perform the following operations:

[0217] When the first optimal word corresponding to each of the above N word positions is the same as the second optimal word, the first optimal word corresponding to each of the above N word positions is determined as the predicted word corresponding to each of the above N word positions.

[0218] Based on the predicted lexical units corresponding to the above N lexical units, the above sub-language model, and the above main language model, the text content that satisfies the above prompt word information is determined.

[0219] Optionally, the processing module 714 described above can also be used to perform the following operations:

[0220] Based on the adjustment probabilities of the candidate words in the first adjustment probability list i, the candidate words in the above word list are sorted to obtain the i-th word sequence.

[0221] Extract the i-th word subsequence from the i-th word sequence above;

[0222] Until N word subsequences are obtained, the text content that satisfies the above prompt word information is determined based on the above N word subsequences, the above N first adjustment probability lists, and the above N second adjustment probability lists.

[0223] Optionally, the processing module 714 described above can also be used to perform the following operations:

[0224] Iterate through the subsequences of the word elements corresponding to the above N word element positions in turn;

[0225] When traversing to the j-th word subsequence corresponding to the j-th word position, calculate the ratio between the adjusted probability of the candidate word in the j-th word subsequence in the first adjusted probability list j and the adjusted probability of the corresponding candidate word in the second adjusted probability list j; j is a positive integer greater than 1 and less than or equal to N.

[0226] When there is a candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold, the candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold is determined as the predicted word at the j-th word position.

[0227] Continue iterating through the subsequence of the word element at the (j+1)th word element position until the predicted word elements corresponding to the above N word element positions are obtained;

[0228] Based on the predicted lexical units corresponding to the above N lexical units, the above main language model, and the above sub-language model, the text content that satisfies the above prompt word information is determined.

[0229] Optionally, the processing module 714 described above can also be used to perform the following operations:

[0230] When the probability ratios of the candidate words in the j-th word subsequence are all less than the probability ratio threshold, the traversal of the word subsequences corresponding to the N word positions is paused.

[0231] From the above list of lexical units, select the candidate lexical unit with the highest adjustment probability in the above second adjustment probability list j, and use it as the predicted lexical unit at the above j-th lexical unit position;

[0232] Based on the predicted lexical units corresponding to the positions of the first j lexical units, as well as the aforementioned sub-language model and the aforementioned main language model, the text content that satisfies the aforementioned prompt word information is determined.

[0233] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0234] In this application, the prediction probabilities in N first predicted probability lists are dynamically adjusted based on the probability distribution features of the N first predicted probability lists obtained by the sub-language model, resulting in N first adjusted probability lists. Similarly, the prediction probabilities in N second predicted probability lists are dynamically adjusted based on the probability distribution features of the N second predicted probability lists obtained by the main language model, resulting in N second adjusted probability lists. This approach helps to make the probability distribution features corresponding to the adjusted probabilities in the first adjusted probability lists closer to those in the second adjusted probability lists. In other words, the difference between the probability distribution features corresponding to the adjusted probabilities in the first and second adjusted probability lists is relatively small. This increases the probability that the candidate word sequences predicted by the sub-language model (i.e., word sequences obtained based on the first adjusted probability lists) are accepted by the main language model, thereby improving the efficiency of text content generation.

[0235] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in this application. For example... Figure 8As shown, the aforementioned computer device 800 can refer to a server or terminal, including: a processor 801, a network interface 804, and a memory 805. Furthermore, the aforementioned computer device 800 may also include: a user interface 803, and at least one communication bus 802. The communication bus 802 is used to implement communication between these components. In some embodiments, the user interface 803 may include a display screen and a keyboard; optionally, the user interface 803 may also include a standard wired interface or a wireless interface. The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 805 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. The memory 805 may also optionally be at least one storage device located remotely from the aforementioned processor 801. Figure 8 As shown, the memory 805, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and computer programs.

[0236] exist Figure 8 In the computer device 800 shown, the network interface 804 provides network communication functionality; the object interface 803 is mainly used to provide an input interface; and the processor 801 can be used to call computer programs stored in the memory 805 to execute:

[0237] Based on the prompt word information, the sub-language model predicts N first prediction probability lists corresponding to N word positions. Based on the prompt word information and the N first prediction probability lists, the main language model predicts N second prediction probability lists corresponding to the N word positions. The first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability that each candidate word in the indicator word list belongs to the predicted word at the i-th word position. N is a positive integer; i is a positive integer less than or equal to N.

[0238] Determine the probability distribution characteristics corresponding to the predicted probabilities in the above N first predicted probability lists respectively, and adjust the predicted probabilities in the above N first predicted probability lists according to the probability distribution characteristics corresponding to the above N first predicted probability lists respectively to obtain N first adjusted probability lists.

[0239] Determine the probability distribution characteristics corresponding to the predicted probabilities in the above N second prediction probability lists respectively, and adjust the predicted probabilities in the above N second prediction probability lists according to the probability distribution characteristics corresponding to the above N second prediction probability lists respectively to obtain N second adjusted probability lists.

[0240] Based on the above N first adjustment probability lists, the above N second adjustment probability lists, and the above word list, determine the text content that satisfies the above prompt word information. The above text content includes the predicted words corresponding to the above N word positions.

[0241] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0242] According to the preset sorting rules, the predicted probabilities in the first predicted probability list i are sorted to obtain the first probability sequence i; the first predicted probability list i is the first predicted probability list corresponding to the i-th word position.

[0243] From the first probability sequence i above, extract the K predicted probabilities that are in the first order; K is an integer greater than 1 and less than P, where P is the number of predicted probabilities in the first probability sequence i above.

[0244] Based on the above K predicted probabilities, determine the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list i.

[0245] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0246] Based on the above K predicted probabilities, determine the uncertainty of the candidate word elements corresponding to the above K predicted probabilities;

[0247] The uncertainty of the candidate words corresponding to the above K predicted probabilities is averaged to obtain the probability distribution entropy of the above first predicted probability list i.

[0248] The probability distribution entropy of the first predicted probability list i is determined as the probability distribution feature corresponding to the predicted probability in the first predicted probability list i.

[0249] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0250] Obtain the initial probability adjustment parameters;

[0251] Based on the probability distribution characteristics corresponding to the first predicted probability list i, the above initial probability adjustment parameters are adjusted to obtain the probability adjustment parameters corresponding to the first predicted probability list i; the above first predicted probability list i is the first predicted probability list corresponding to the i-th word position.

[0252] Based on the probability adjustment parameters corresponding to the first predicted probability list i, the predicted probabilities in the first predicted probability list i are adjusted to obtain the first adjusted probability list i.

[0253] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0254] Multiply the probability distribution features, entropy scaling factor, and normalization parameter corresponding to the first predicted probability list i to obtain the normalized probability distribution features;

[0255] The normalized probability distribution features are subjected to exponential processing to obtain the probability adjustment weights corresponding to the first predicted probability list i.

[0256] The probability adjustment weight corresponding to the first predicted probability list i is multiplied by the initial probability adjustment parameter to obtain the probability adjustment parameter corresponding to the first predicted probability list i.

[0257] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0258] Based on the probability adjustment parameters corresponding to the first predicted probability list i, the predicted probabilities in the first predicted probability list i are normalized to obtain the adjusted probabilities corresponding to the first predicted probability list i.

[0259] Based on the adjusted probabilities corresponding to the first predicted probability list i above, generate the first adjusted probability list i.

[0260] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0261] Based on the above N first adjustment probability lists, the first optimal word corresponding to each of the above N word positions is selected from the above word list; the first optimal word at the i-th word position is the candidate word corresponding to the maximum adjustment probability in the first adjustment probability list at the i-th word position.

[0262] Based on the above N second adjustment probability lists, the second optimal word elements corresponding to the above N word element positions are selected from the above word element list; the second optimal word element at the above i-th word element position is the candidate word element corresponding to the maximum adjustment probability in the above second adjustment probability list at the above i-th word element position;

[0263] Based on the first and second optimal lexical units corresponding to the above N lexical units, the text content that satisfies the above prompt word information is determined.

[0264] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0265] The first optimal word and the second optimal word corresponding to the above N word positions are compared in turn;

[0266] When the first optimal word at the j-th word position is different from the second optimal word, the first optimal word corresponding to the first j-1 word positions is determined as the predicted word corresponding to the first j-1 word positions; j is a positive integer greater than 1 and less than or equal to N;

[0267] The second optimal word at the j-th word position is determined as the predicted word at the j-th word position.

[0268] Based on the predicted lexical units corresponding to the positions of the first j lexical units, determine the text content that satisfies the above prompt word information.

[0269] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0270] When the first optimal word corresponding to each of the above N word positions is the same as the second optimal word, the first optimal word corresponding to each of the above N word positions is determined as the predicted word corresponding to each of the above N word positions.

[0271] Based on the predicted lexical units corresponding to the above N lexical units, determine the text content that satisfies the above prompt word information.

[0272] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0273] Based on the adjustment probabilities of candidate words in the first adjustment probability list i, the candidate words in the word list are sorted to obtain the i-th word sequence; the first adjustment probability list i is the first adjustment probability list corresponding to the position of the i-th word.

[0274] Extract the i-th word subsequence from the i-th word sequence above;

[0275] Until N word subsequences are obtained, the text content that satisfies the above prompt word information is determined based on the above N word subsequences, the above N first adjustment probability lists, and the above N second adjustment probability lists.

[0276] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0277] Iterate through the subsequences of the word elements corresponding to the above N word element positions in turn;

[0278] When traversing to the j-th word subsequence corresponding to the j-th word position, calculate the ratio between the adjusted probability of the candidate word in the j-th word subsequence in the first adjusted probability list j and the adjusted probability of the corresponding candidate word in the second adjusted probability list j; j is a positive integer greater than 1 and less than or equal to N.

[0279] When there is a candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold, the candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold is determined as the predicted word at the j-th word position.

[0280] Continue iterating through the subsequence of the word element at the (j+1)th word element position until the predicted word elements corresponding to the above N word element positions are obtained;

[0281] Based on the predicted lexical units corresponding to the above N lexical units, determine the text content that satisfies the above prompt word information.

[0282] Optionally, the processor 801 described above can be used to call a computer program stored in the memory 805 to execute:

[0283] When the probability ratios of the candidate words in the j-th word subsequence are all less than the probability ratio threshold, the traversal of the word subsequences corresponding to the N word positions is paused.

[0284] From the above list of lexical units, select the candidate lexical unit with the highest adjustment probability in the above second adjustment probability list j, and use it as the predicted lexical unit at the above j-th lexical unit position;

[0285] Using the aforementioned sub-language model and the aforementioned main language model, based on the predicted lexical units corresponding to the positions of the first j lexical units and the aforementioned prompt word information, the text content that satisfies the aforementioned prompt word information is determined.

[0286] In this application, the prediction probabilities in N first predicted probability lists are dynamically adjusted based on the probability distribution features of the N first predicted probability lists obtained by the sub-language model, resulting in N first adjusted probability lists. Similarly, the prediction probabilities in N second predicted probability lists are dynamically adjusted based on the probability distribution features of the N second predicted probability lists obtained by the main language model, resulting in N second adjusted probability lists. This approach helps to make the probability distribution features corresponding to the adjusted probabilities in the first adjusted probability lists closer to those in the second adjusted probability lists. In other words, the difference between the probability distribution features corresponding to the adjusted probabilities in the first and second adjusted probability lists is relatively small. This increases the probability that the candidate word sequences predicted by the sub-language model (i.e., word sequences obtained based on the first adjusted probability lists) are accepted by the main language model, thereby improving the efficiency of text content generation.

[0287] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned data processing apparatus. This computer program includes program instructions, which, when executed by the processor, enable the execution of the data processing method described in the corresponding embodiments above. Therefore, further details will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0288] As an example, the above program instructions can be deployed and executed on a computer device, or deployed and executed on at least two computer devices in one location, or executed on at least two computer devices distributed in at least two locations and interconnected by a communication network. At least two computer devices distributed in at least two locations and interconnected by a communication network can form a blockchain network.

[0289] The aforementioned computer-readable storage medium can be a data processing apparatus provided in any of the foregoing embodiments or a central storage unit of the aforementioned computer device, such as a hard disk or central storage unit of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium may include both the central storage unit and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0290] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish content in different media, rather than to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0291] In practice, the collection and processing of data in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the data subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the data subject.

[0292] This application also provides a computer program product, including a computer program. When executed by a processor, the computer program implements the data processing method and decoding method described in the preceding embodiments, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiments of this application.

[0293] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0294] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable network-connected device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable network-connected device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable network-connected device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable network-connected device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0295] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, include: Based on the prompt word information, the sub-language model predicts N first prediction probability lists corresponding to N word positions, and the main language model predicts N second prediction probability lists corresponding to the N word positions based on the prompt word information and the N first prediction probability lists. The first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position; N is a positive integer; i is a positive integer less than or equal to N; Determine the probability distribution characteristics corresponding to the predicted probabilities in the N first predicted probability lists respectively, and adjust the predicted probabilities in the N first predicted probability lists according to the probability distribution characteristics corresponding to the N first predicted probability lists respectively to obtain N first adjusted probability lists; Determine the probability distribution characteristics corresponding to the predicted probabilities in the N second prediction probability lists respectively, and adjust the predicted probabilities in the N second prediction probability lists according to the probability distribution characteristics corresponding to the N second prediction probability lists respectively to obtain N second adjusted probability lists; Based on the N first adjustment probability lists, the N second adjustment probability lists, and the word list, determine the text content that satisfies the prompt word information, wherein the text content includes the predicted words corresponding to the N word positions.

2. The method according to claim 1, characterized in that, Determining the probability distribution features corresponding to the predicted probabilities in the N first predicted probability lists includes: According to the preset sorting rules, the predicted probabilities in the first predicted probability list i are sorted to obtain the first probability sequence i; the first predicted probability list i is the first predicted probability list corresponding to the i-th word position; from the first probability sequence i, the K predicted probabilities that are in the first order are extracted; K is an integer greater than 1 and less than P, where P is the number of predicted probabilities in the first probability sequence i. Based on the K predicted probabilities, determine the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list i.

3. The method according to claim 2, characterized in that, The step of determining the probability distribution characteristics corresponding to the predicted probabilities in the first predicted probability list i based on the K predicted probabilities includes: Based on the K predicted probabilities, determine the uncertainty of the candidate word corresponding to the K predicted probabilities; The uncertainty of the candidate word corresponding to the K predicted probabilities is averaged to obtain the probability distribution entropy of the first predicted probability list i. The probability distribution entropy of the first predicted probability list i is determined as the probability distribution feature corresponding to the predicted probability in the first predicted probability list i.

4. The method according to claim 1, characterized in that, The step involves adjusting the predicted probabilities in the N first predicted probability lists based on their respective probability distribution characteristics, resulting in N first adjusted probability lists, including: Obtain the initial probability adjustment parameters; Based on the probability distribution characteristics corresponding to the first predicted probability list i, the initial probability adjustment parameters are adjusted to obtain the probability adjustment parameters corresponding to the first predicted probability list i; the first predicted probability list i is the first predicted probability list corresponding to the i-th word position; Based on the probability adjustment parameter corresponding to the first predicted probability list i, the predicted probabilities in the first predicted probability list i are adjusted to obtain the first adjusted probability list i.

5. The method according to claim 4, characterized in that, The step of adjusting the initial probability adjustment parameters according to the probability distribution characteristics corresponding to the first predicted probability list i to obtain the probability adjustment parameters corresponding to the first predicted probability list i includes: Multiply the probability distribution features, entropy scaling factor, and normalization parameter corresponding to the first predicted probability list i to obtain the normalized probability distribution features; The normalized probability distribution features are subjected to exponential processing to obtain the probability adjustment weights corresponding to the first predicted probability list i. The probability adjustment weight corresponding to the first predicted probability list i is multiplied by the initial probability adjustment parameter to obtain the probability adjustment parameter corresponding to the first predicted probability list i.

6. The method according to claim 4, characterized in that, The step of adjusting the predicted probabilities in the first predicted probability list i according to the probability adjustment parameter corresponding to the first predicted probability list i to obtain the first adjusted probability list i includes: Based on the probability adjustment parameter corresponding to the first prediction probability list i, normalize each prediction probability in the first prediction probability list i to obtain the adjusted probability corresponding to the first prediction probability list i. Based on the adjustment probabilities corresponding to the first predicted probability list i, generate the first adjusted probability list i.

7. The method according to claim 1, characterized in that, The step of determining the text content that satisfies the prompt word information based on the N first adjustment probability lists, the N second adjustment probability lists, and the word character list includes: Based on the N first adjustment probability lists, the first optimal word elements corresponding to the N word element positions are selected from the word element list; the first optimal word element at the i-th word element position is the candidate word element corresponding to the maximum adjustment probability in the first adjustment probability list at the i-th word element position; Based on the N second adjustment probability lists, the second optimal word elements corresponding to the N word element positions are selected from the word element list; the second optimal word element at the i-th word element position is the candidate word element corresponding to the maximum adjustment probability in the second adjustment probability list at the i-th word element position; Based on the first and second optimal lexical units corresponding to the N lexical units, the text content that satisfies the prompt word information is determined.

8. The method according to claim 7, characterized in that, The step of determining the text content that satisfies the prompt word information based on the first optimal word and the second optimal word corresponding to the N word positions respectively includes: The first optimal word and the second optimal word corresponding to the N word positions are compared sequentially; When the first optimal word at the j-th word position is different from the second optimal word, the first optimal word corresponding to the first j-1 word positions is determined as the predicted word corresponding to the first j-1 word positions; j is a positive integer greater than 1 and less than or equal to N; The second optimal word at the j-th word position is determined as the predicted word at the j-th word position; By using the sub-language model and the main language model, and based on the predicted lexical units corresponding to the positions of the first j lexical units and the prompt word information, the text content that satisfies the prompt word information is determined.

9. The method according to claim 8, characterized in that, The method further includes: When the first optimal word corresponding to each of the N word positions is found to be the same as the second optimal word, the first optimal word corresponding to each of the N word positions is determined as the predicted word corresponding to each of the N word positions. Based on the predicted lexical units corresponding to the N lexical units, determine the text content that satisfies the prompt word information.

10. The method according to claim 1, characterized in that, The step of determining the text content that satisfies the prompt word information based on the N first adjustment probability lists, the N second adjustment probability lists, and the word character list includes: Based on the adjustment probabilities of candidate words in the first adjustment probability list i, the candidate words in the word list are sorted to obtain the i-th word sequence; the first adjustment probability list i is the first adjustment probability list corresponding to the position of the i-th word. Extract the i-th word subsequence from the i-th word sequence; Until N word subsequences are obtained, the text content that satisfies the prompt word information is determined based on the N word subsequences, the N first adjustment probability lists, and the N second adjustment probability lists.

11. The method according to claim 10, characterized in that, The step of determining the text content that satisfies the prompt word information based on the N word subsequences, the N first adjustment probability lists, and the N second adjustment probability lists includes: Iterate through the subsequences of the N word positions in turn; When traversing to the j-th word subsequence corresponding to the j-th word position, calculate the ratio between the adjusted probability of the candidate word in the j-th word subsequence in the first adjusted probability list j and the adjusted probability of the corresponding candidate word in the second adjusted probability list j; j is a positive integer greater than 1 and less than or equal to N; When there is a candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold, the candidate word in the j-th word subsequence with a probability ratio greater than or equal to the probability ratio threshold is determined as the predicted word at the j-th word position. Continue traversing the subsequence of the word element at the (j+1)th word element position until the predicted word elements corresponding to the N word element positions are obtained; Based on the predicted lexical units corresponding to the N lexical units, determine the text content that satisfies the prompt word information.

12. The method according to claim 11, characterized in that, The method further includes: When the probability ratios of the candidate words in the j-th word subsequence are all less than the probability ratio threshold, the traversal of the word subsequences corresponding to the N word positions is paused. From the word list, select the candidate word with the highest adjustment probability in the second adjustment probability list j, and use it as the predicted word at the j-th word position; By using the sub-language model and the main language model, and based on the predicted lexical units corresponding to the positions of the first j lexical units and the prompt word information, the text content that satisfies the prompt word information is determined.

13. A data processing apparatus, characterized in that, include: The prediction module is used to predict N first prediction probability lists corresponding to N word positions based on prompt word information using a sub-language model, and to predict N second prediction probability lists corresponding to the N word positions based on the prompt word information and the N first prediction probability lists using a main language model. The first and second prediction probability lists corresponding to the i-th word position respectively include the prediction probability of each candidate word in the indicator word list belonging to the predicted word at the i-th word position; N is a positive integer; i is a positive integer less than or equal to N; The first adjustment module is used to determine the probability distribution characteristics corresponding to the prediction probabilities in the N first prediction probability lists respectively, and adjust the prediction probabilities in the N first prediction probability lists according to the probability distribution characteristics corresponding to the N first prediction probability lists respectively, to obtain N first adjusted probability lists. The second adjustment module is used to determine the probability distribution characteristics corresponding to the prediction probabilities in the N second prediction probability lists respectively, and adjust the prediction probabilities in the N second prediction probability lists according to the probability distribution characteristics corresponding to the N second prediction probability lists respectively, to obtain N second adjusted probability lists. The processing module is used to determine the text content that satisfies the prompt word information based on the N first adjustment probability lists, the N second adjustment probability lists, and the word list, wherein the text content includes the predicted words corresponding to the N word positions.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 12.

15. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, performs the method as described in any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

Images