Model training method, text processing method, device and electronic equipment

Abstract

This application provides a model training method, a text processing method, an apparatus, and an electronic device. The model training method includes: acquiring a language model to be trained and training samples; the training samples include: multiple text segments to be processed, and corresponding preferred and non-preferred processing results for each text segment; acquiring adaptive weight values ​​for each text segment; the adaptive weight values ​​are correlated with the degree of difference between the preferred and non-preferred processing results; determining the prediction deviation of each text segment based on a first prediction score of the language model to be trained for the preferred processing results and a second prediction score for the non-preferred processing results; fusing the prediction deviations of each text segment based on its adaptive weight values ​​to obtain a model training loss value; and adjusting the parameters of the language model to be trained based on the model training loss value. This solution can meet the processing preference requirements for local text segments.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model training method, a text processing method, an apparatus, an electronic device, and a computer storage medium. Background Technology

[0002] Generative Language Model (GLM) is a machine learning model that learns the probability distribution of natural language. It can model the generation process of sequences and predict the next token based on the preceding context in an autoregressive manner, thereby generating coherent natural language text that conforms to grammatical and semantic rules.

[0003] Currently, in practical applications, generative language models can perform many types of text processing tasks with high quality, such as news translation and everyday conversation translation. However, in the field of text processing, there exists a type of task that involves "multiple fragment (line) outputs with fragment (line) level local preferences." This type of task is characterized by the fact that the text to be processed typically consists of multiple short sentence fragments with contextual dependencies, but the user's preference for the processing results is concentrated on a few key short sentence fragments. For this type of processing task, the generative language models provided by related technologies perform poorly and cannot meet people's requirements for processing local text fragments.

[0004] Taking subtitle translation as an example, the text to be translated typically includes: simple, conventional text fragments with no room for optimization or rewriting, and a small number of key text fragments with room for rewriting. Users' preferences for translation results usually focus on these few key text fragments: besides the requirement for accurate translation, people also have a preference for "free translation," that is, for key text fragments, people prefer translations that are vivid enough and can flexibly adapt to the localized style of different languages ​​and themes (the translation results should have a certain degree of vividness and naturalness), rather than simply "literal translation." However, current generative language models produce poor subtitle translation results and cannot meet people's preference for "free translation" of key sentences. Summary of the Invention

[0005] In view of this, embodiments of this application provide a model training scheme and a text processing scheme to at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of this application, a model training method is provided, comprising: Obtain the language model to be trained and training samples; the training samples include: multiple text segments to be processed, and the preference processing results and non-preference processing results corresponding to each text segment to be processed; Obtain adaptive weight values ​​for each text segment to be processed; the adaptive weight values ​​are related to the degree of difference between the preferred processing result and the non-preferred processing result; Based on the first predicted score of the language model to be trained for the preferred processing result and the second predicted score for the non-preferred processing result, the prediction deviation of each text segment to be processed is determined; wherein, the first predicted score represents the probability that the language model to be trained generates the preferred processing result, the second predicted score represents the probability that the language model to be trained generates the non-preferred processing result, and the prediction deviation represents the degree of difference between the first predicted score and the second predicted score. Based on the adaptive weight values ​​of each text segment to be processed, the prediction deviation of each text segment to be processed is fused to obtain the model training loss value, and the parameters of the language model to be trained are adjusted according to the model training loss value.

[0007] According to a second aspect of the embodiments of this application, a text processing method is provided, including: Obtain the text to be processed; the text to be processed includes multiple text fragments to be processed. The multiple text fragments to be processed are input into the target language model, and the target language model generates the processing results corresponding to each text fragment to be processed. The target language model is obtained through the model training method described in the first aspect above.

[0008] According to a third aspect of the embodiments of this application, a model training apparatus is provided, comprising: The first acquisition module is used to acquire the language model to be trained and training samples; the training samples include: multiple text segments to be processed, and the preference processing results and non-preference processing results corresponding to each text segment to be processed. The second acquisition module is used to acquire the adaptive weight value of each text segment to be processed; the adaptive weight value is related to the degree of difference between the preferred processing result and the non-preferred processing result; The determining module is used to determine the prediction deviation of each text segment to be processed based on the first prediction score of the language model to be trained for the preferred processing result and the second prediction score for the non-preferred processing result; wherein, the first prediction score represents the probability that the language model to be trained generates the preferred processing result, the second prediction score represents the probability that the language model to be trained generates the non-preferred processing result, and the prediction deviation represents the degree of difference between the first prediction score and the second prediction score; The parameter adjustment module is used to obtain the model training loss value by fusing the prediction deviation of each text segment based on the adaptive weight value of each text segment to be processed, and to adjust the parameters of the language model to be trained according to the model training loss value.

[0009] According to a fourth aspect of the embodiments of this application, a text processing apparatus is provided, comprising: The third acquisition module is used to acquire the text to be processed; the text to be processed includes multiple text fragments to be processed. The processing module is used to input the multiple text fragments to be processed into the target language model, and generate the processing results corresponding to each text fragment through the target language model; The target language model is obtained through the model training method described in the first aspect above.

[0010] According to a fifth aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first or second aspect.

[0011] According to a sixth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first or second aspect.

[0012] According to a seventh aspect of the embodiments of this application, a computer program product is provided, which includes computer instructions that instruct a computing device to perform an operation corresponding to the method described in the first or second aspect.

[0013] According to the model training method provided in the embodiments of this application, the acquired training samples include multiple text segments to be processed, and pairs of good and bad processing results (preferred processing results and non-preferred processing results) corresponding to each text segment to be processed; and adaptive weight values ​​associated with the degree of difference between the pairs of good and bad processing results are also acquired; then, after determining the prediction deviation of each text segment to be processed according to the language model to be trained, the prediction deviation of each text segment to be processed is fused based on the above adaptive weight values, and the fusion result is used as the model training loss value to adjust the model parameters, thus completing the model training process.

[0014] In the embodiments described above, when performing preference optimization training on the language model to be trained, the process is not performed at the granularity of the entire text to be processed, but at a finer granularity of text segments. In other words, preference alignment optimization is performed at the "segment" level. Starting from the fine-grained level of "segment", the text segments to be processed with room for optimization and those without room for optimization are effectively distinguished based on the degree of difference between the good and bad processing results.

[0015] More importantly, for different text segments to be processed, this embodiment sets adaptive weight values ​​for the degree of difference between them based on the processing results of superior and inferior processing, and when calculating the model training loss value, the prediction deviation of each text segment to be processed is adaptively fused and adjusted according to the above adaptive weight values.

[0016] The gradient value during model training represents the rate of change of the model training loss function relative to the model parameters. In other words, it represents the amount of parameter updates the model makes to reduce the training loss value; specifically, it represents the direction and magnitude of adjustments needed to reduce the training loss value. Since the model training loss value is obtained by fusing the prediction deviations of each text segment in this embodiment, each text segment contributes a corresponding gradient component to the total gradient value. Furthermore, since the model training loss value is obtained by adaptively fusing the prediction deviations of each text segment based on adaptive weights, the magnitude of the adaptive weights for each text segment determines the proportion of the gradient component corresponding to each text segment in the total gradient value.

[0017] Therefore, this embodiment sets gradient values ​​for different types of text segments to be processed during model training by associating adaptive weight values ​​with the degree of difference between the good and bad processing result pairs. This allows the training resources during the training process to be better applied to text segments to be processed that have a high degree of difference between the good and bad processing result pairs and have room for optimization, rather than text segments to be processed that have a low degree of difference between the good and bad processing result pairs and have no room for optimization. This avoids the problem of reward signal dilution caused by preference optimization at the granularity of the entire text to be processed, and achieves local preference alignment for text segments to be processed that have room for optimization.

[0018] In summary, the embodiments of this application improve the model's performance in text processing tasks that require "multiple fragment (line) outputs and have fragment (line) level local preferences," thus better meeting people's preferences for processing local text fragments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0020] Figure 1 This is a flowchart illustrating the steps of a model training method according to an embodiment of this application. Figure 2 This is a schematic diagram of a model training framework according to an embodiment of this application; Figure 3 This is a flowchart illustrating the steps of a text processing method according to an embodiment of this application. Figure 4 This is a structural block diagram of a model training device according to an embodiment of this application; Figure 5 This is a structural block diagram of a text processing apparatus according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0022] Reference Figure 1 , Figure 1 This is a flowchart illustrating the steps of a model training method according to an embodiment of this application. The model training method may include the following steps: Step 102: Obtain the language model to be trained and training samples; the training samples include: multiple text segments to be processed, and the preference processing results and non-preference processing results corresponding to each text segment to be processed.

[0023] Indicatively, the language model to be trained can be any generative language model. For example, it can be a pre-trained general generative language model provided by relevant technologies, or a task-specific generative language model that is specifically designed or fine-tuned for a certain type of text processing task.

[0024] The training samples contain text to be processed, which can consist of multiple text fragments with contextual dependencies. In this embodiment, the specific content of the text processing task is not limited; for example, it could be a subtitle translation task, a multi-turn text reasoning task, a multi-turn dialogue task, etc. Regardless of the type of text processing task, users typically have certain preferences for the processing results. The preferred processing result corresponding to the text fragment can be a result that satisfies the user's preferences, while the non-preferred processing result can be a result that does not satisfy the user's preferences.

[0025] In the text processing category of subtitle translation, the text to be processed can be a complete subtitle to be translated, or a text fragment can be a sentence or a line of subtitle from the complete subtitle to be translated. For subtitle translation, in addition to the requirement of accuracy, people often have a preference for "free translation." They tend to want the translation to be vivid enough and flexible enough to adapt to the localized style of different languages ​​and themes. Therefore, the preferred processing result for the text fragment can be a "free translation" that meets the user's requirements of "accuracy, naturalness, and vividness," while the non-preferred processing result can be a "literal translation" that does not meet the above requirements.

[0026] Step 104: Obtain the adaptive weight value of each text segment to be processed; the adaptive weight value is related to the degree of difference between the preferred processing result and the non-preferred processing result.

[0027] To illustrate, after obtaining the preferred processing result and the non-preferred processing result through step 102, for the text segment to be processed, an adaptive weight value for the text segment to be processed can be configured based on the degree of difference between the preferred processing result and the non-preferred processing result.

[0028] The degree of difference between the preference processing result and the non-preference processing result can be represented as the difference between the degree to which the preference processing result satisfies the user's preference needs and the degree to which the non-preference processing result does so. To facilitate obtaining the adaptive weight value of the text segment to be processed, the degree of difference between the preference processing result and the non-preference processing result can be quantified first. Then, based on the quantified degree of difference between the preference processing result and the non-preference processing result, the adaptive weight value of the text segment to be processed can be obtained. For example, the adaptive weight value can be positively correlated with the degree of difference, that is, the greater the degree of difference, the greater the adaptive weight value; conversely, the smaller the degree of difference, the smaller the adaptive weight value.

[0029] Step 106: Determine the prediction deviation of each text segment to be processed based on the first prediction score of the language model to be trained for the preference processing result and the second prediction score for the non-preference processing result; the prediction deviation characterizes the degree of difference between the first prediction score and the second prediction score.

[0030] Schematic, the first predicted score of the language model to be trained for the preference processing result can characterize the probability that the language model to be trained will generate or output the preference processing result. Correspondingly, the second predicted score of the language model to be trained for the non-preference processing result can characterize the probability that the language model to be trained will generate or output the non-preference processing result. For example, the first and second predicted scores can be represented in probabilistic form, that is, the first predicted score is the probability that the language model to be trained will generate the preference processing result, and the second predicted score is the probability that the language model to be trained will generate the non-preference processing result. Further, the probability of each word segmentation unit included in the preference processing result generated by the language model to be trained can be calculated separately, and the probability of each word segmentation unit included in the non-preference processing result generated by the language model to be trained can be multiplied to obtain the probability of the language model to be trained generating the non-preference processing result. Specifically, for a single word segmentation unit contained in the preference processing result, the probability of the model generating that word segmentation unit can be obtained as follows: input the preference processing result into the language model to be trained for forward computation, and then output the original output score (Logits) corresponding to that word segmentation unit through the model; then normalize the original output score to obtain the probability that the language model to be trained generates that word segmentation unit contained in the preference processing result. Similarly, for a single word segmentation unit contained in the non-preference processing result, the probability of the model generating that word segmentation unit can be obtained as follows: input the non-preference processing result into the language model to be trained for forward computation, and then output the original output score corresponding to that word segmentation unit through the model; then normalize the original output score to obtain the probability that the language model to be trained generates that word segmentation unit contained in the non-preference processing result.

[0031] For a specific segment to be processed, the prediction deviation in this embodiment can refer to the degree of inconsistency in the output distribution or score space of the language model to be trained for processing different preference attributes (i.e., preference processing results and non-preference processing results). The prediction deviation can be used to quantify the gap between the model's current prediction logic and the user's preferences. In some optional implementations, the prediction deviation can be determined by calculating the difference term (such as difference, ratio, or log-odds difference) between the first predicted score of the preference processing result and the second predicted score of the non-preference processing result; in other implementations, the prediction deviation can further include the result after performing a nonlinear mapping (such as processing through a Sigmoid function or Softmax function) on the above difference term, to characterize the model's achievement of the preference alignment goal.

[0032] Furthermore, to avoid the model output distribution gradually deviating from the initial distribution of the language model to be trained due to preference optimization, thus affecting the text processing quality, a reference model (e.g., obtained by initializing the parameters of the language model to be trained) can be set as a baseline. By comparing the output of the language model to be trained with the output of the reference model, the text processing behavior of the language model to be trained can be controlled, thereby avoiding the aforementioned distribution shift problem.

[0033] Based on the above, when determining the prediction deviation of the text segment to be processed, the first prediction score of the language model to be trained for the preferred processing result and the second prediction score for the unpreferred processing result can be determined respectively, and the third prediction score of the reference model for the preferred processing result and the fourth prediction score for the unpreferred processing result can be determined respectively; then the first ratio of the first prediction score to the third prediction score and the second ratio of the second prediction score to the fourth prediction score can be calculated, and the prediction deviation of the text segment to be processed can be obtained based on the difference between the first ratio and the second ratio.

[0034] For example, for a specific text fragment to be processed, the above process can be represented by the following formula:

[0035] in, This represents the i-th text segment to be processed. The corresponding prediction deviation; This represents the first predicted score mentioned above; This indicates the third predicted score mentioned above; This represents the second predicted score mentioned above; This represents the fourth prediction score mentioned above; This represents the first ratio mentioned above; This represents the second ratio mentioned above; , All of these are pre-set hyperparameters.

[0036] Step 108: Based on the adaptive weight values ​​of each text segment to be processed, the prediction deviation of each text segment to be processed is fused to obtain the model training loss value, and the parameters of the language model to be trained are adjusted according to the model training loss value.

[0037] Indicatively, after obtaining the adaptive weight values ​​of each text segment to be processed, the prediction deviation of each text segment to be processed can be weighted and fused according to their respective adaptive weight values. The fusion result is then used as the model training loss value to adjust the parameters of the language model to be trained, thereby obtaining the trained language model.

[0038] Taking the above formula as an example, step 108 yields the model training loss value expressed by the following formula:

[0039] That is:

[0040]

[0041] in, This represents the model training loss value; This represents the i-th text segment to be processed. Adaptive weight values; This represents the result of a weighted sum of the prediction deviations of each text segment to be processed based on adaptive weight values; it can also be called the prediction deviation of the text to be processed. This represents the expected value of the prediction deviation for each text to be processed in the same training batch.

[0042] In the embodiments described above, when optimizing preferences for the language model being trained, the optimization is not performed at the granularity of the entire text to be processed, but at a finer granularity of text segments. In other words, preference alignment optimization is performed at the "segment" level. Starting from the fine-grained level of "segment," the text segments with optimization potential and those without optimization potential are effectively distinguished based on the degree of difference between the superior and inferior processing results.

[0043] More importantly, for different text segments to be processed, adaptive weight values ​​are set according to the degree of difference between the processing results, and when calculating the model training loss value, the prediction deviation of each text segment to be processed is adaptively fused and adjusted according to the above adaptive weight values.

[0044] The gradient value during model training represents the rate of change of the model training loss function relative to the model parameters. In other words, it represents the amount of parameter updates the model makes to reduce the training loss value; specifically, it represents the direction and magnitude of adjustments needed to reduce the training loss value. Since the model training loss value is obtained by fusing the prediction deviations of each text segment in this embodiment, each text segment contributes a corresponding gradient component to the total gradient value. Furthermore, since the model training loss value is obtained by adaptively fusing the prediction deviations of each text segment based on adaptive weights, the magnitude of the adaptive weights for each text segment determines the proportion of the gradient component corresponding to each text segment in the total gradient value.

[0045] Therefore, this embodiment sets gradient values ​​for different types of text segments to be processed during model training by associating adaptive weight values ​​with the degree of difference between the good and bad processing result pairs. This allows the training resources during the training process to be better applied to text segments to be processed that have a high degree of difference between the good and bad processing result pairs and have room for optimization, rather than text segments to be processed that have a low degree of difference between the good and bad processing result pairs and have no room for optimization. This avoids the problem of reward signal dilution caused by preference optimization at the granularity of the entire text to be processed, and achieves local preference alignment for text segments to be processed that have room for optimization.

[0046] In summary, the embodiments of this application improve the model's performance in text processing tasks that require "multiple fragment (line) outputs and have fragment (line) level local preferences," thus better meeting people's preferences for processing local text fragments.

[0047] The model training method provided in this embodiment can be executed by any suitable electronic device, including but not limited to: servers, PCs, cloud computing platforms, workstations, etc.

[0048] Optionally, in some embodiments, the process of obtaining adaptive weight values ​​for each text segment to be processed may include: For a single text segment to be processed, obtain the first evaluation value of the preferred processing result in the preset evaluation dimension, and the second evaluation value of the non-preferred processing result in the preset evaluation dimension; based on the evaluation difference between the first evaluation value and the second evaluation value, obtain the adaptive weight value of the single text segment to be processed.

[0049] Schematic, in the above embodiments, an evaluation dimension is first preset according to the user's preference requirements, and the evaluation values ​​of the preference processing result and the non-preference processing result in the preset evaluation dimension are obtained respectively. The difference between the two evaluation values ​​is then used as the degree of difference between the preference processing result and the non-preference processing result. For ease of distinction, in this embodiment, the evaluation value of the preference processing result in the preset evaluation dimension is referred to as the first evaluation value, and the evaluation value of the non-preference processing result in the preset evaluation dimension is referred to as the second evaluation value. The first evaluation value can be regarded as a quantification of the degree to which the preference processing result satisfies the user's preference requirements, and the second evaluation value can be regarded as a quantification of the degree to which the non-preference processing result satisfies the user's preference requirements.

[0050] In this embodiment, the specific method for obtaining the first and second evaluation values ​​is not limited and can be customized according to the actual situation. For example, a traditional rule engine-based automatic evaluation algorithm can be used. The processing can be as follows: first, a formalized set of rules (such as regular expressions, keyword lists, logical conditions, or scoring criteria, etc.) can be constructed according to the preset evaluation dimensions. Then, the preference processing results and non-preference processing results are respectively input into the rule engine for rule matching and condition judgment to obtain the first and second evaluation values.

[0051] For example, artificial intelligence models, such as LLM (Large Language Model), can be used to input the results of preference processing and non-preference processing into the model, which will then automatically output the corresponding evaluation values. Taking subtitle translation as an example, human translators and multiple LLMs simultaneously evaluate and score various translation candidates for the same original sentence. The evaluation dimensions include the following three aspects: accuracy (whether the meaning is translated accurately), naturalness (whether it sounds like authentic spoken language), and vividness (whether it contains atmosphere and emotional information). By comparison, it was found that the difference between the evaluation values ​​of LLM and human evaluation is within a reasonable range. This shows that LLM can reliably act as an "automatic review / reward model" to obtain relatively accurate evaluation values ​​for both preference-processed and non-preference-processed results.

[0052] In the above process, corresponding evaluation dimensions were set according to user preference requirements. Then, the degree of preference for the preferred and unpreferred processing results was evaluated separately, resulting in specific quantified evaluation values. Subsequently, the quantified adaptive weight values ​​were obtained by calculating the difference between these quantified evaluation values. By introducing an evaluation mechanism, the degree of difference between the preferred and unpreferred processing results was accurately quantified. This allows the final adaptive weight values ​​to more accurately reflect the optimizable space of the text segment to be processed. Consequently, during subsequent model training, training resources are better allocated to the text segment with a larger optimization space, ultimately improving the model's text processing efficiency and better meeting users' processing preferences for local text segments.

[0053] Optionally, in some embodiments, the number of preset evaluation dimensions can be multiple.

[0054] Correspondingly, the process of obtaining the first evaluation value of the preference processing result in the preset evaluation dimension and the second evaluation value of the non-preference processing result in the preset evaluation dimension may include: Obtain the evaluation weights corresponding to each preset evaluation dimension; obtain the single-dimensional evaluation value of the preference processing result in a single preset evaluation dimension; and, based on the evaluation weights corresponding to each preset evaluation dimension, sum the single-dimensional evaluation values ​​of the preference processing result in a single preset evaluation dimension to obtain the first evaluation value of the preference processing result in the preset evaluation dimension; obtain the single-dimensional evaluation value of the non-preference processing result in a single preset evaluation dimension; and, based on the evaluation weights corresponding to each preset evaluation dimension, sum the single-dimensional evaluation values ​​of the non-preference processing result in a single preset evaluation dimension to obtain the second evaluation value of the non-preference processing result in the preset evaluation dimension.

[0055] To illustrate, in text processing tasks, people's preferences for processing results can be complex and comprehensive. Based on this, the above embodiments of this application configure multiple different evaluation dimensions and set corresponding evaluation weights for each evaluation dimension. When evaluating the processing results, the single-dimensional evaluation value of each individual dimension is first calculated, and then, based on the aforementioned evaluation weights of each evaluation dimension, the single-dimensional evaluation values ​​are weighted and summed to obtain a comprehensive evaluation value that can characterize the preference attributes of the processing results.

[0056] Compared to single-dimensional evaluation methods, the above-mentioned multi-dimensional comprehensive evaluation method can deconstruct user preferences into several independent attributes, enabling a fine distinction between preference processing results and non-preference processing results, and avoiding evaluation bias or information loss caused by missing dimensions.

[0057] Optionally, in some embodiments, the model training method may further include: Receive preference level information; preference level information represents the degree of preference for different preset evaluation dimensions; Based on the preference level information, the evaluation weights corresponding to each preset evaluation dimension are adjusted. Based on the adjusted evaluation weights of each preset evaluation dimension, the first evaluation value of the preference processing result in the preset evaluation dimension and the second evaluation value of the non-preference processing result in the preset evaluation dimension are obtained.

[0058] In illustratively, embodiments of this application also support users to configure different preference levels, and then adjust the evaluation weights of different preset evaluation dimensions according to the user-configured preference level information. The evaluation values ​​of preference processing results and non-preference processing results are obtained according to the adjusted evaluation weights, so as to adjust the adaptive weight values ​​according to the above evaluation values, and to train the model based on the adjusted adaptive weight values, thereby obtaining a target language model that matches the user's preference level information.

[0059] In other words, the above embodiments of this application can adjust the combination of evaluation weights corresponding to each evaluation dimension by receiving the user's preference level information, thereby achieving level adjustment of the model's processing style to adapt to the preference tendencies in different application scenarios. For example, in the case of subtitle translation tasks, the preset evaluation dimensions may include accuracy, naturalness, and vividness. When the user wants the model's processing result to be more inclined towards "faithful literal translation," preference level information can be provided (in this embodiment of the application, the specific form of preference level information and the specific method of providing preference level information are not limited). This information indicates that the user's preference for the accuracy evaluation dimension is higher than that for other evaluation dimensions, and thus the evaluation weight corresponding to accuracy can be increased, while the evaluation weights of other evaluation dimensions can be decreased. Alternatively, when the user wants the model's processing result to be more inclined towards "vivid localization and easy conversational style," preference level information can be provided. This information indicates that the user's preference for the naturalness and vividness evaluation dimensions is higher than that for the accuracy evaluation dimension, and thus the evaluation weights corresponding to vividness and naturalness can be increased, while the evaluation weight corresponding to accuracy can be decreased.

[0060] Optionally, in some embodiments, the process of obtaining training samples may include: obtaining a text to be processed containing multiple original text segments, and obtaining corresponding candidate processing results for a single original text segment; filtering text segments from multiple original text segments to obtain multiple text segments to be processed; and determining the preferred processing result and non-preferred processing result corresponding to each text segment from the candidate processing results corresponding to each text segment to be processed, based on the candidate evaluation value of each candidate processing result in a preset evaluation dimension.

[0061] The text fragment to be processed must meet the following conditions: the number of candidate processing results after deduplication is greater than a preset number threshold, and / or, among the candidate evaluation values, the evaluation difference between the largest and smallest candidate evaluation values ​​is greater than a preset difference threshold.

[0062] The method for determining candidate evaluation values ​​can be similar to the method for determining the first and second evaluation values ​​described above. Exemplary methods for obtaining candidate processing results will be illustrated later.

[0063] To illustrate, the text to be processed typically includes simple, regular text segments with no room for optimization and rewriting, as well as key text segments with room for optimization and rewriting. During model training, to improve training efficiency and effectiveness, training resources are more likely to be allocated to the key text segments with room for optimization and rewriting, rather than wasted on simple, regular text segments with no room for optimization and rewriting.

[0064] For simple, conventional text fragments with no room for optimization or rewriting, they typically have the following characteristics: the number of candidate processing results after deduplication is small, or the difference in evaluation values ​​between different candidate processing results is small (i.e., the difference between different candidate processing results is not significant).

[0065] Therefore, this embodiment of the application selects text segments from the perspective of the number of deduplicated candidate processing results and the evaluation difference between the largest and smallest candidate evaluation values. Segments with a large number of deduplicated candidate processing results or significant differences in evaluation values ​​between different candidate processing results are used as training samples to participate in the subsequent model training process. Conversely, segments with a small number of deduplicated candidate processing results or small differences in evaluation values ​​between different candidate processing results are not included in the candidate training process. Through this process, training resources can be better allocated to key text segments with rewriting potential, thus effectively improving the efficiency and effectiveness of model training.

[0066] Furthermore, for the text segments selected through text segment filtering and selected for subsequent model training, the corresponding preferred and unpreferred processing results can be determined from among the candidate processing results based on the candidate evaluation values. In this embodiment, the specific method for determining the preferred and unpreferred processing results is not limited. For example, the candidate processing result with the largest candidate evaluation value can be determined as the preferred processing result, and the candidate processing result with the smallest candidate evaluation value can be determined as the unpreferred processing result; alternatively, one can be selected from multiple candidate processing results with a candidate evaluation value greater than a preset value as the preferred processing result, and one can be selected from multiple candidate processing results with a candidate evaluation value less than another preset value as the unpreferred processing result, and so on.

[0067] Optionally, in some embodiments, the process of obtaining corresponding candidate processing results for a single original text segment may include: Obtain the text to be processed, and the processing result segments corresponding to each preceding text segment; wherein, the preceding text segment is the original text segment located before a single original text segment; input the text to be processed and the processing result segments corresponding to each preceding text segment into the language model to be trained, and output the candidate processing result corresponding to a single original text segment through the language model to be trained; repeat the step of inputting the text to be processed and the processing result segments corresponding to each preceding text segment into the language model to be trained until the preset stopping condition is met.

[0068] To illustrate, suppose that for the i-th original text segment in the text to be processed, the process of obtaining the candidate processing result corresponding to the original text segment can be as follows: input the complete text to be processed, as well as the processing result segments corresponding to the 0th original text segment, the 1st original text segment, ..., the (i-1)th original text segment, into the language model to be trained, so that the language model to be trained can output a candidate processing result corresponding to the i-th original text segment; then, repeat the above process to obtain different candidate processing results until the preset stopping condition is met, and then stop the above repetition process.

[0069] Those skilled in the art will understand that, for the first original text segment in the text to be processed, since there is no preceding text segment before the original text segment, it is impossible for there to be a processing result segment corresponding to the preceding text segment. Therefore, the process of obtaining the candidate processing result corresponding to the first original text segment may include: inputting the complete text to be processed and the first original text segment into the language model to be trained, and outputting a candidate processing result corresponding to the first original text segment through the language model to be trained; then, repeating the above process to obtain different candidate processing results until the preset stopping condition is met, and then stopping the above repetition process.

[0070] In this application, the specific content of the preset stopping condition is not limited. For example, the preset stopping condition can be determined to be met when the above-mentioned repeated process is executed a preset number of times; or the preset stopping condition can be determined to be met when the time taken for the above-mentioned candidate processing result generation process reaches a preset duration, and so on.

[0071] For the entire text to be processed, the above process can be performed on each original text segment in sequence to obtain the candidate processing results corresponding to each original text segment.

[0072] In the embodiments described above, without relying on manual intervention, the language model to be trained can automatically and efficiently generate candidate processing results corresponding to each original text segment, and then select high-quality "segment"-level preference data for subsequent preference optimization training. Therefore, the effectiveness and efficiency of model training can be effectively improved.

[0073] Furthermore, in the above embodiments, for the preceding text segment, any suitable selection method can be used to determine a processing result from among the candidate processing results as the processing result segment corresponding to the preceding text segment. For example, the candidate processing result with the largest candidate evaluation value can be determined as the processing result segment corresponding to the preceding text segment.

[0074] Optionally, in some embodiments, the process of obtaining the processing result segment corresponding to the preceding text segment may include: A preset random selection strategy is adopted to select one candidate processing result from the candidate processing results corresponding to the preceding text segment as the processing result segment corresponding to the preceding text segment.

[0075] Schematic, a random selection strategy, also known as a nondeterministic selection strategy, refers to a process where, instead of deterministically selecting based solely on evaluation values, random perturbations or probability distributions are introduced when determining the processing result segment. This allows other candidate processing results with non-maximum evaluation values ​​to also have a certain probability of being selected as the processing result segment corresponding to the preceding text segment. In this application, the specific implementation of the random selection strategy is not limited, and may include, but is not limited to, greedy algorithms, probability sampling based on the Softmax function, roulette wheel algorithm, or Thompson sampling, etc.

[0076] In the above model training process, if the candidate processing result with the largest evaluation value is always determined as the processing result fragment corresponding to the preceding text fragment to participate in the generation of candidate processing results corresponding to subsequent text fragments, this will lead to a significant impact on the processing quality of other text fragments to be processed if the quality of the processing result fragment corresponding to the preceding text fragment is poor during the model inference stage.

[0077] To avoid the above situation, in the above embodiments of this application, when determining the processing result segment corresponding to the preceding text segment, the candidate processing result with the largest evaluation value is not always determined as the processing result segment to participate in the generation of candidate processing results corresponding to the subsequent text segment. Instead, the candidate processing result with the largest evaluation value is replaced with other candidate processing results with a certain probability, so that the trained language model also maintains a certain robustness to "non-most preferred candidate processing results".

[0078] Optionally, in some embodiments, the language model to be trained can be a language model used for subtitle translation. Correspondingly, for the text processing task of subtitle translation, the process of obtaining the above-mentioned language model to be trained may include: Obtain the initial language model and the fine-tuning sample set; the fine-tuning sample set includes: the subtitle fragment to be translated and the corresponding translated text fragment; based on the fine-tuning sample set, perform supervised fine-tuning on the initial language model to obtain the language model to be trained.

[0079] Schematic illustration: The initial language model can be any generative language model. For example, it can be a pre-trained general generative language model provided by relevant technologies. The fine-tuning sample set can include text pairs with semantic alignment at the "fragment" level: the subtitle fragment to be translated, and the corresponding translated fragment.

[0080] After obtaining the initial language model and the fine-tuning sample set, the initial language model can be fine-tuned in a supervised manner using the text pairs in the fine-tuning sample set to obtain the basic subtitle translation language model (i.e., the language model to be trained in the embodiments of this application).

[0081] The above embodiments first employ supervised fine-tuning to enable the model to learn the basic capabilities and specifications of subtitle translation, thereby acquiring the fundamental ability to "translate accurately and speak fluently." Furthermore, supervised fine-tuning allows for the establishment of a reasonable parameter initialization distribution for the model. Subsequent preference optimization training based on this distribution allows the model to focus on distinguishing subtle differences between high-quality preference pairs, avoiding the drift or non-convergence of preference optimization training objectives caused by insufficient initial capabilities. In summary, the above embodiments of this application improve the efficiency of model training and also enhance the robustness of the final model.

[0082] Optionally, in some embodiments, the process of obtaining the fine-tuned sample set described above may include: The process involves obtaining the subtitle text to be translated and the translated text. The subtitle text includes multiple subtitle segments to be translated and their corresponding playback timestamps. The translated text includes multiple translated subtitle segments and their corresponding playback timestamps. Based on the playback timestamps of each subtitle segment, the search timestamp range for each subtitle segment is determined. For a single subtitle segment, based on the search timestamp range and the playback timestamps of each translated subtitle segment, candidate translated subtitle segments with overlapping playback durations with the search timestamp range of the single subtitle segment are searched. Based on the candidate translated subtitle segments, the translated subtitle segment corresponding to the single subtitle segment is determined.

[0083] To illustrate, the field of subtitle translation typically involves various content formats such as movies, TV series, documentaries, and animation, and also involves multilingual (e.g., English, German, French, Korean, Chinese, Thai, etc.) and multidirectional translation. For a specific film or television work, or a particular scene within it, a corresponding translated text (target subtitle) can usually be assigned to the text to be translated (source subtitle) through methods such as human translation. Furthermore, the text to be translated usually contains multiple text segments (sentences) to be translated, along with the playback timestamp information for each segment. Similarly, the translated text also includes multiple translated text segments, along with the playback timestamp information for each segment.

[0084] Actual analysis reveals that film and television subtitle translation often involves significant interpretation and rewriting based on visuals and cultural context. The same text segment to be translated may be abbreviated, split, or rewritten in the translated text segment. Therefore, the relationship between the text to be translated and the corresponding translated text is usually not a simple one-to-one correspondence; that is, the i-th text segment to be translated does not necessarily semantically match the i-th translated text segment. The relationship between the text segment to be translated and the translated text segment may be one-to-one, one-to-many, or many-to-one, and there may be a difference in the number of lines between them.

[0085] Considering the above, the embodiments of this application determine a searchable timestamp range based on the playback timestamp information corresponding to the searchable timestamp range; then, within this searchable timestamp range, they search for corresponding translated text segments for the text segment to be translated, thereby ensuring the correct correspondence between the text segment to be translated and the translated text segment in the fine-tuning sample set, forming bilingual data with "segment"-level alignment that can be used for supervised fine-tuning training. Furthermore, the above alignment process utilizes the playback timestamp information corresponding to the text segment to be translated, the playback timestamp information corresponding to the translated text segment, and the line number difference between the text segment to be translated and the translated text segment. The alignment process has low processing complexity (linear complexity) and high processing efficiency.

[0086] The process of determining the searchable timestamp range for each text segment based on its playback timestamp information can include: determining the playback timestamp range of the text segment to be translated based on its playback timestamp information, and then determining the searchable timestamp range based on that playback timestamp range. For example, the playback timestamp range can be determined as the searchable timestamp range; or, the timestamp range can be extended by a certain amount, such as advancing the start timestamp of the playback timestamp range by a certain duration and delaying the end timestamp of the playback timestamp range by a certain duration, thereby obtaining the searchable timestamp range.

[0087] In addition, when searching for candidate translation fragments, translation text fragments with overlapping playback durations with the search timestamp range of a single text fragment to be translated can be identified as candidate translation fragments. Alternatively, translation text fragments with overlapping playback durations with the search timestamp range of a single text fragment to be translated that are greater than a preset duration threshold can also be identified as candidate translation fragments.

[0088] For example, regarding the process of determining the translation fragment corresponding to a single text fragment to be translated based on candidate translation fragments: If there are multiple candidate translation fragments, and the time interval between any two adjacent candidate translation fragments (arranged in chronological order according to timestamps) is less than a preset interval threshold (this is a time continuity verification; to ensure that multiple candidate translation fragments are continuous or closely connected on the playback timeline, multiple candidate translation fragments that are continuous in time usually correspond to a complete sentence), then each candidate translation fragment can be determined as a translation fragment corresponding to a single text fragment to be translated. In this case, the text fragment to be translated and the candidate translation fragments have a "one-to-many" relationship.

[0089] If the number of candidate translation fragments is 1, and there are no other text fragments to be translated that have an overlapping playback duration with the candidate translation fragment (that is, there are no other text fragments to be translated that satisfy the following condition: the search timestamp range of the other text fragments to be translated has an overlapping playback duration with the playback timestamp range of the candidate translation fragment), then the candidate translation fragment can be determined as the translation fragment corresponding to a single text fragment to be translated. In this case, there is a "one-to-one" relationship between the text fragment to be translated and the candidate translation fragment.

[0090] For any target text segment to be translated, if the number of candidate translation segments is 1, and there are other text segments to be translated that have overlapping playback time with the candidate translation segment; for the target text segment to be translated and the other text segments to be translated, it can be further determined whether the time interval between two adjacent text segments to be translated is less than a preset interval threshold (same as above, belonging to time continuity verification); if so, the target text segment to be translated and the other text segments to be translated are merged to obtain the merged text segment to be translated, and the candidate translation segment is determined as the translation segment corresponding to the merged text segment to be translated. At this time, the text segment to be translated and the candidate translation segment have a "many-to-one" relationship.

[0091] See Figure 2 , Figure 2 This is a schematic diagram of a model training framework according to an embodiment of this application. To facilitate a better understanding of the model training method provided in this application embodiment, the following is combined with... Figure 2 An exemplary illustration of the complete optimization process is provided below: Step 1, Model Fine-tuning. A generic LLM is obtained as the initial language model. Additionally, a fine-tuning sample set is acquired, including the text fragments to be processed and their corresponding processing results. The generic LLM is then subjected to supervised fine-tuning using the fine-tuning sample set, resulting in the fine-tuned LLM: π sft , will π sft As a language model to be trained.

[0092] Step 2, "Fragment" Level Sampling. Obtain a sample set, which includes the text to be processed containing multiple original text fragments. For the text to be processed in this sample set, repeatedly process the text... Text to be processed The i-th original text fragment Previous text fragments ... Corresponding processing result fragment …… And, the i-th original text fragment Input π sft , through π sft Output the i-th original text fragment respectively The corresponding k candidate processing results: , ... Then, the above k candidate processing results are deduplicated. (Furthermore, in order to further improve the model training effect, in addition to generating candidate processing results through the language model to be trained, other methods can be used to obtain higher quality candidate processing results.) (Examples include: manual processing, generation by an optimized language model, etc.) An evaluation model πe (e.g., a general LLM) is used to evaluate each candidate processing result on a preset evaluation dimension, obtaining a candidate evaluation value for each result. A preset random selection strategy is then used to select one candidate processing result from among the candidate results as the i-th original text fragment. Corresponding processing result fragment After that, you can Add it to the input of the language model to be trained to obtain the (i+1)th original text fragment. Corresponding processing result fragment Similarly, for the text to be processed, the following local preference data can be formed: "each original text fragment + candidate processing result corresponding to each original text fragment + evaluation value of each candidate processing result".

[0093] Step 3, Text Segment Filtering. Based on the results obtained in Step 2, the multiple original text segments contained in the text to be processed are filtered. For ease of distinction, the filtered original text segments can be referred to as the text segments to be processed. Specific filtering strategies can be as follows: 1) Filter out original text segments whose number of deduplicated candidate processing results is greater than a preset threshold as text segments to be processed; 2) Filter out original text segments whose evaluation difference between the highest and lowest candidate evaluation values ​​is greater than a preset difference threshold; 3) Filter out original text segments whose number of candidate processing results is greater than a preset threshold, and whose evaluation difference between the highest and lowest candidate evaluation values ​​is greater than a preset difference threshold. After filtering out the text segments to be processed, the preferred processing result and the non-preferred processing result corresponding to each text segment can be determined based on the evaluation value of each candidate processing result in the text segment to be processed. Thus, the training samples for subsequent preference optimization training are obtained: multiple text segments to be processed, and the preferred processing result and the non-preferred processing result corresponding to each text segment to be processed.

[0094] Step 4, Adaptive Local Preference Optimization Training. For each text segment to be processed, an adaptive weight value is calculated based on the difference between the preference-processed and non-preference-processed results. Based on the first predicted score of the language model for the preference-processed result and the second predicted score for the non-preference-processed result, and referring to the third and fourth predicted scores of the model for the preference-processed result and the non-preference-processed result, the prediction deviation of each text segment is determined. Based on the adaptive weight values ​​of each text segment, the prediction deviations of each text segment are fused to obtain the model training loss value. and according to The parameters of the language model to be trained are adjusted to obtain the final trained target language model.

[0095] Reference Figure 3 , Figure 3 This is a flowchart illustrating the steps of a text processing method according to an embodiment of this application. The text processing method may include the following steps: Step 302: Obtain the text to be processed; the text to be processed includes multiple text fragments.

[0096] Step 304: Input multiple text fragments to be processed into the target language model, and generate the processing results corresponding to each text fragment through the target language model.

[0097] The target language model is obtained through the model training method provided in the aforementioned model training embodiments.

[0098] Indicatively, a generative language model is a machine learning model that learns the probability distribution of natural language. During the model inference stage, it can model the generation process of sequences and predict the next word segment based on the preceding text in an autoregressive manner, thereby generating coherent natural language text that conforms to grammatical and semantic rules.

[0099] In this embodiment of the application, after inputting the text to be processed, which includes multiple text fragments, into the target language model, the target language model will generate the processing result corresponding to each text fragment: For the first text segment to be processed, the generation process includes: the target language model generates the processing result corresponding to the first text segment to be processed based on the text to be processed and the first text segment to be processed; for non-first text segments to be processed, the generation process includes: the target language model generates the processing result corresponding to the non-first text segment to be processed based on the text to be processed and the processing results corresponding to each text segment to be processed before the non-first text segment to be processed.

[0100] The text processing method provided in this embodiment can be executed by any suitable electronic device, including but not limited to: servers, PCs, etc.

[0101] According to the text processing method provided in this application embodiment, during the training process of the target language model, starting from the fine-grained level of "segment" and based on the degree of difference between the good and bad processing results, it effectively distinguishes between text segments with optimization space and text segments without optimization space. More importantly, for different text segments to be processed, this application embodiment sets adaptive weight values ​​based on the degree of difference between the good and bad processing results, and adaptively fuses and adjusts the prediction deviation of each text segment to be processed according to the above adaptive weight values ​​when calculating the model training loss value. The gradient value during the model training process represents the rate of change of the model training loss function relative to the model parameters, that is, the amount of parameter updates generated by the model to reduce the model training loss value, that is, the direction and magnitude of adjustment needed to reduce the above model training loss value. Since the model training loss value is obtained by fusing the prediction deviation of each text segment to be processed in this application embodiment, each text segment to be processed contributes a corresponding gradient component to the total gradient value. Furthermore, since this application uses the result of adaptively fusing the prediction deviation of each text segment based on adaptive weight values ​​as the model training loss value, the magnitude of the adaptive weight value of each text segment will determine the proportion of the gradient component corresponding to each text segment in the total gradient value.

[0102] Therefore, this embodiment sets gradient values ​​for different types of text segments to be processed during model training by associating adaptive weight values ​​with the degree of difference between the good and bad processing result pairs. This allows the training resources during the training process to be better applied to text segments to be processed that have a high degree of difference between the good and bad processing result pairs and have room for optimization, rather than text segments to be processed that have a low degree of difference between the good and bad processing result pairs and have no room for optimization. This avoids the problem of reward signal dilution caused by preference optimization at the granularity of the entire text to be processed, and achieves local preference alignment for text segments to be processed that have room for optimization.

[0103] Therefore, through the embodiments of this application, the processing effect of the model on text processing tasks with "multiple fragment (line) outputs and fragment (line) level local preferences" can be improved, better meeting people's processing preferences for local text fragments.

[0104] Figure 4 This is a structural block diagram of a model training apparatus according to an embodiment of this application. The model training apparatus provided in this application includes: The first acquisition module 402 is used to acquire the language model to be trained and training samples; the training samples include: multiple text segments to be processed, and the preference processing results and non-preference processing results corresponding to each text segment to be processed. The second acquisition module 404 is used to acquire the adaptive weight value of each text segment to be processed; the adaptive weight value is related to the degree of difference between the preferred processing result and the non-preferred processing result; The determination module 406 is used to determine the prediction deviation of each text segment to be processed based on the first prediction score of the language model to be trained for the preferred processing result and the second prediction score for the non-preferred processing result; wherein, the first prediction score represents the probability that the language model to be trained generates the preferred processing result, the second prediction score represents the probability that the language model to be trained generates the non-preferred processing result, and the prediction deviation represents the degree of difference between the first prediction score and the second prediction score. The parameter adjustment module 408 is used to obtain the model training loss value by fusing the prediction deviation of each text segment based on the adaptive weight value of each text segment to be processed, and to adjust the parameters of the language model to be trained according to the model training loss value.

[0105] Optionally, in some embodiments, the second acquisition module 404 is specifically used for: For a single text segment to be processed, obtain the first evaluation value of the preferred processing result in the preset evaluation dimension, and the second evaluation value of the non-preferred processing result in the preset evaluation dimension; based on the evaluation difference between the first evaluation value and the second evaluation value, obtain the adaptive weight value of the single text segment to be processed.

[0106] Optionally, in some embodiments, the number of preset evaluation dimensions is multiple; the second acquisition module 404, when acquiring the first evaluation value of the preference processing result in the preset evaluation dimension and the second evaluation value of the non-preference processing result in the preset evaluation dimension, is specifically used to: acquire the evaluation weight corresponding to each preset evaluation dimension respectively. Obtain the single-dimensional evaluation value of the preference processing result in a single preset evaluation dimension; and according to the evaluation weights corresponding to each preset evaluation dimension, sum the single-dimensional evaluation values ​​of the preference processing result in a single preset evaluation dimension to obtain the first evaluation value of the preference processing result in the preset evaluation dimension; obtain the single-dimensional evaluation value of the non-preference processing result in a single preset evaluation dimension; and according to the evaluation weights corresponding to each preset evaluation dimension, sum the single-dimensional evaluation values ​​of the non-preference processing result in a single preset evaluation dimension to obtain the second evaluation value of the non-preference processing result in the preset evaluation dimension.

[0107] Optionally, in some embodiments, the model training apparatus further includes: A preference level adjustment module is used to receive preference level information; the preference level information represents the degree of preference for different preset evaluation dimensions; according to the preference level information, the evaluation weights corresponding to each preset evaluation dimension are adjusted, so as to obtain the first evaluation value of the preference processing result in the preset evaluation dimension and the second evaluation value of the non-preference processing result in the preset evaluation dimension based on the adjusted evaluation weights of each preset evaluation dimension.

[0108] Optionally, in some embodiments, the first acquisition module 402, when acquiring training samples, is specifically used for: The process involves obtaining a text to be processed containing multiple original text fragments, and for each original text fragment, obtaining corresponding candidate processing results; filtering the text fragments from the multiple original text fragments to obtain multiple text fragments to be processed; determining the preferred and unpreferred processing results for each text fragment from the candidate processing results corresponding to each text fragment, based on the candidate evaluation values ​​of each candidate processing result in a preset evaluation dimension; wherein, the deduplicated text fragments to be processed satisfy the following conditions: the number of corresponding candidate processing results is greater than a preset number threshold, and / or, among the candidate evaluation values, the evaluation difference between the largest and smallest candidate evaluation values ​​is greater than a preset difference threshold.

[0109] Optionally, in some embodiments, the first acquisition module 402, when acquiring the corresponding candidate processing result for a single original text fragment, is specifically used for: Obtain the text to be processed, and the processing result segments corresponding to each preceding text segment; wherein, the preceding text segment is the original text segment located before a single original text segment; input the text to be processed and the processing result segments corresponding to each preceding text segment into the language model to be trained, and output the candidate processing result corresponding to a single original text segment through the language model to be trained; repeat the step of inputting the text to be processed and the processing result segments corresponding to each preceding text segment into the language model to be trained until the preset stopping condition is met.

[0110] Optionally, in some embodiments, the first acquisition module 402 is specifically used to: acquire the processing result segment corresponding to the preceding text segment. A preset random selection strategy is adopted to select one candidate processing result from the candidate processing results corresponding to the preceding text segment as the processing result segment corresponding to the preceding text segment.

[0111] Optionally, in some embodiments, the language model to be trained is a language model used for subtitle translation; the first acquisition module 402, when acquiring the language model to be trained, is specifically used for: Obtain the initial language model and the fine-tuning sample set; the fine-tuning sample set includes: the subtitle fragment to be translated and the corresponding translated text fragment; based on the fine-tuning sample set, perform supervised fine-tuning on the initial language model to obtain the language model to be trained.

[0112] Optionally, in some embodiments, the first acquisition module 402, when acquiring the fine-tuning sample set, is specifically used for: acquiring the subtitle text to be translated and the translated text; the subtitle text to be translated includes multiple text segments to be translated and playback timestamp information corresponding to each text segment to be translated; the translated text includes multiple translated text segments and playback timestamp information corresponding to each translated text segment; based on the playback timestamp information corresponding to each text segment to be translated, determining the search timestamp range corresponding to each text segment to be translated; for a single text segment to be translated, based on the search timestamp range corresponding to the single text segment to be translated and the playback timestamp information corresponding to each translated text segment, searching for candidate translated segments with overlapping playback durations with the search timestamp range of the single text segment to be translated; and based on the candidate translated segments, determining the translated segment corresponding to the single text segment to be translated.

[0113] The model training device of this embodiment is used to implement the corresponding model training methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the model training device of this embodiment can be referred to the description of the corresponding part in the foregoing method embodiments, which will also not be repeated here.

[0114] Figure 5 This is a structural block diagram of a text processing apparatus according to an embodiment of this application. The text processing apparatus provided in this application includes: The third acquisition module 502 is used to acquire the text to be processed; the text to be processed includes multiple text fragments to be processed. Processing module 504 is used to input multiple text fragments to be processed into the target language model and generate the processing results corresponding to each text fragment through the target language model; The target language model is obtained through the model training method provided in the above-described model training method embodiments.

[0115] The text generation device of this embodiment is used to implement the corresponding text processing methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the text processing device of this embodiment can be referred to the description of the corresponding part in the foregoing method embodiments, which will also not be repeated here.

[0116] Reference Figure 6 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0117] like Figure 6 As shown, the electronic device may include: a processor 602, a communications interface 604, a memory 606, and a communications bus 608.

[0118] in: The processor 602, communication interface 604, and memory 606 communicate with each other via communication bus 608.

[0119] Communication interface 604 is used for communication with other electronic devices.

[0120] The processor 602 is used to execute program 610, specifically to perform the relevant steps in the above method embodiments.

[0121] Indicatively, program 610 may include program code that includes computer operation instructions.

[0122] The processor 602 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0123] Memory 606 is used to store program 610. Memory 606 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0124] Program 610 may include multiple computer instructions. Specifically, program 610 may use multiple computer instructions to cause processor 602 to perform the operation corresponding to any of the methods described in the foregoing multiple method embodiments.

[0125] The specific implementation of each step in program 610 can be found in the corresponding steps and units described in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0126] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0127] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0128] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0129] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0130] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0131] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0132] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A model training method, comprising: Obtain the language model to be trained and training samples; The training samples include: multiple text segments to be processed, and the preference processing results and non-preference processing results corresponding to each text segment to be processed; Obtain adaptive weight values ​​for each text segment to be processed; the adaptive weight values ​​are related to the degree of difference between the preferred processing result and the non-preferred processing result; Based on the first predicted score of the language model to be trained for the preferred processing result and the second predicted score for the non-preferred processing result, the prediction deviation of each text segment to be processed is determined; wherein, the first predicted score represents the probability that the language model to be trained generates the preferred processing result, the second predicted score represents the probability that the language model to be trained generates the non-preferred processing result, and the prediction deviation represents the degree of difference between the first predicted score and the second predicted score. Based on the adaptive weight values ​​of each text segment to be processed, the prediction deviation of each text segment to be processed is fused to obtain the model training loss value, and the parameters of the language model to be trained are adjusted according to the model training loss value.

2. The method according to claim 1, wherein, The process of obtaining the adaptive weight values ​​for each text segment to be processed includes: For a single text segment to be processed, obtain the first evaluation value of the preference processing result in a preset evaluation dimension, and the second evaluation value of the non-preference processing result in a preset evaluation dimension; An adaptive weight value for the individual text segment to be processed is obtained based on the evaluation difference between the first evaluation value and the second evaluation value.

3. The method according to claim 2, wherein, The number of preset evaluation dimensions is multiple; The step of obtaining the first evaluation value of the preference processing result in a preset evaluation dimension and the second evaluation value of the non-preference processing result in a preset evaluation dimension includes: Obtain the evaluation weights corresponding to each preset evaluation dimension; Obtain the single-dimensional evaluation value of the preference processing result in a single preset evaluation dimension; and according to the evaluation weights corresponding to each preset evaluation dimension, perform a weighted summation of the single-dimensional evaluation values ​​of the preference processing result in a single preset evaluation dimension to obtain the first evaluation value of the preference processing result in the preset evaluation dimension. Obtain the single-dimensional evaluation value of the non-preference processing result in a single preset evaluation dimension; and according to the evaluation weights corresponding to each preset evaluation dimension, sum the single-dimensional evaluation values ​​of the non-preference processing result in a single preset evaluation dimension to obtain the second evaluation value of the non-preference processing result in the preset evaluation dimension.

4. The method according to claim 3, wherein, The method further includes: Receive preference level information; the preference level information represents the degree of preference for different preset evaluation dimensions; Based on the preference level information, the evaluation weights corresponding to each preset evaluation dimension are adjusted, so as to obtain the first evaluation value of the preference processing result in the preset evaluation dimension and the second evaluation value of the non-preference processing result in the preset evaluation dimension based on the adjusted evaluation weights of each preset evaluation dimension.

5. The method according to any one of claims 1-4, wherein, The process of obtaining training samples includes: Obtain the text to be processed, which contains multiple original text fragments, and for a single original text fragment, obtain the corresponding candidate processing result; The text fragments are filtered from the multiple original text fragments to obtain multiple text fragments to be processed. Based on the candidate evaluation values ​​of each candidate processing result in the preset evaluation dimension, the preferred processing result and the non-preferred processing result corresponding to each text segment to be processed are determined from the candidate processing results corresponding to each text segment to be processed. The text fragment to be processed satisfies the following conditions: the number of candidate processing results after deduplication is greater than a preset number threshold, and / or, among the candidate evaluation values, the evaluation difference between the largest and smallest candidate evaluation values ​​is greater than a preset difference threshold.

6. The method according to claim 5, wherein, The step of obtaining the corresponding candidate processing results for a single original text fragment includes: Obtain the text to be processed, and the processing result fragments corresponding to each preceding text fragment; wherein, the preceding text fragment is the original text fragment located before the single original text fragment; The text to be processed and the processing result fragments corresponding to each preceding text fragment are input into the language model to be trained, and the candidate processing result corresponding to the single original text fragment is output by the language model to be trained. Repeat the steps of inputting the text to be processed and the processing result fragments corresponding to each preceding text fragment into the language model to be trained until the preset stopping condition is met.

7. The method according to claim 6, wherein, The process of obtaining the processing result fragment corresponding to the preceding text fragment includes: A preset random selection strategy is adopted to select one candidate processing result from the candidate processing results corresponding to the preceding text segment as the processing result segment corresponding to the preceding text segment.

8. The method according to any one of claims 1-4, wherein, The language model to be trained is a language model used for subtitle translation. The process of obtaining the language model to be trained includes: Obtain the initial language model and the fine-tuning sample set; the fine-tuning sample set includes: the subtitle fragment to be translated and the corresponding translated text fragment; Based on the fine-tuning sample set, the initial language model is subjected to supervised fine-tuning to obtain the language model to be trained.

9. The method according to claim 8, wherein, The process of obtaining the fine-tuned sample set includes: The system obtains the subtitle text to be translated and the translated text; the subtitle text to be translated includes multiple text segments to be translated and the playback timestamp information corresponding to each text segment; the translated text includes multiple translated text segments and the playback timestamp information corresponding to each translated text segment. Based on the playback timestamp information corresponding to each text segment to be translated, determine the range of timestamps to be searched for each text segment to be translated. For a single text segment to be translated, based on the search timestamp range corresponding to the single text segment to be translated and the playback timestamp information corresponding to each translated text segment, search for candidate translated text segments that have an overlapping playback duration with the search timestamp range of the single text segment to be translated; Based on the candidate translation fragments, the translation fragment corresponding to a single text fragment to be translated is determined.

10. A text processing method, comprising: Get the text to be processed; The text to be processed includes multiple text fragments to be processed; The multiple text fragments to be processed are input into the target language model, and the target language model generates the processing results corresponding to each text fragment to be processed. The target language model is obtained by the model training method described in any one of claims 1-9.

11. A model training device, comprising: The first acquisition module is used to acquire the language model to be trained and training samples; The training samples include: multiple text segments to be processed, and the preference processing results and non-preference processing results corresponding to each text segment to be processed; The second acquisition module is used to acquire the adaptive weight value of each text segment to be processed; the adaptive weight value is related to the degree of difference between the preferred processing result and the non-preferred processing result; The determining module is used to determine the prediction deviation of each text segment to be processed based on the first prediction score of the language model to be trained for the preferred processing result and the second prediction score for the non-preferred processing result; wherein, the first prediction score represents the probability that the language model to be trained generates the preferred processing result, the second prediction score represents the probability that the language model to be trained generates the non-preferred processing result, and the prediction deviation represents the degree of difference between the first prediction score and the second prediction score; The parameter adjustment module is used to obtain the model training loss value by fusing the prediction deviation of each text segment based on the adaptive weight value of each text segment to be processed, and to adjust the parameters of the language model to be trained according to the model training loss value.

12. A text processing apparatus, comprising: The third acquisition module is used to acquire the text to be processed; The text to be processed includes multiple text fragments to be processed; The processing module is used to input the multiple text fragments to be processed into the target language model, and generate the processing results corresponding to each text fragment through the target language model; The target language model is obtained by the model training method described in any one of claims 1-9.

13. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform an operation corresponding to the method as described in any one of claims 1-10.

14. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-10.

15. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods described in claims 1-10.

Images