Method and apparatus for optimizing machine learning models for text expansion

Abstract

The application discloses an optimization method and device for a machine learning model for text expansion, the method comprising: inputting a first text into a preset first machine learning model for expansion to obtain a second text; obtaining a sample set of repeated texts in the second text, and comparing the sample set with a first preset text to obtain a paired data set; training a reward model according to the paired data set, and optimizing the first machine learning model according to the reward model to obtain a second machine learning model; evaluating whether the repeated problem of the output text of the second machine learning model is alleviated, and determining the second machine learning model as a target machine learning model in the case that the repeated problem of the output text of the second machine learning model is alleviated. The method can significantly improve the quality of the expanded text, reduce the problem of low quality of the expanded text caused by repeated content, improve the user experience, and improve the efficiency.

Description

Technical Field

[0001] This invention belongs to the field of text processing technology, and particularly relates to an optimization method for a machine learning model for text expansion, an optimization device for a machine learning model for text expansion, a text expander, and a computer-readable storage medium. Background Technology

[0002] In related technologies, when users expand short text into long text using large models, sometimes the output long text contains large sections of repetitive content, making the expanded long text unusable directly. Users still need to check and modify the long text themselves, resulting in a poor user experience and low efficiency. Summary of the Invention

[0003] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, a first objective of this invention is to propose an optimization method for a machine learning model used for text expansion.

[0004] To achieve the above objectives, a first aspect of the present invention provides an optimization method for a machine learning model for text expansion. The method includes: inputting a first text into a preset first machine learning model for expansion to obtain a second text; obtaining a sample set of repeated text in the second text and comparing the sample set with a first preset text to obtain a paired dataset, wherein the first preset text is the standard text corresponding to the first text; training a reward model based on the paired dataset and optimizing the first machine learning model based on the reward model to obtain a second machine learning model; evaluating whether the repetition problem of the output text of the second machine learning model is mitigated, and determining the second machine learning model as the target machine learning model if the repetition problem of the output text of the second machine learning model is mitigated.

[0005] According to an embodiment of the present invention, an optimization method for a machine learning model for text expansion firstly inputs a first text into a preset first machine learning model for expansion to obtain a second text. Then, a sample set of duplicate text in the second text from the first machine learning model is obtained, and the sample set of the first machine learning model is compared with the first preset text to obtain a paired dataset. Next, a reward model is trained based on the paired dataset of the first machine learning model, and the first machine learning model is optimized based on the reward model to obtain a second machine learning model. Finally, the duplication problem in the output text of the first and second machine learning models is evaluated. If the duplication problem in the output text of the first and second machine learning models is reduced, the first and second machine learning models are determined as the target machine learning model. Therefore, this method can significantly improve the quality of expanded text, reduce the problem of low-quality expanded text caused by duplicate content, improve the user experience, and increase efficiency.

[0006] In addition, the optimization method for the machine learning model for text expansion according to the above embodiments of the present invention may also have the following additional technical features:

[0007] According to an embodiment of the present invention, after determining that the second machine learning model is the target machine learning model, the method further includes: replacing the first machine learning model with the target machine learning model to perform iterative optimization on the target machine learning model.

[0008] According to one embodiment of the present invention, optimizing the first machine learning model based on the reward model to obtain a second machine learning model includes: training and generating the second machine learning model using an RLHF (Reward Learning from Human Feedback) process or a DPO (Differentiable Policy Optimization) process.

[0009] According to one embodiment of the present invention, evaluating whether the repetition problem of the output text of the second machine learning model is mitigated includes: obtaining the repetition rate of the output text of the second machine learning model; determining whether the repetition rate is less than a preset repetition rate threshold; and if the repetition rate is less than the preset repetition rate threshold, determining that the repetition problem of the output text of the second machine learning model is mitigated.

[0010] According to one embodiment of the present invention, the repeating text is a series of consecutive words or phrases.

[0011] According to one embodiment of the present invention, the first text is the claims and the second text is the description.

[0012] According to one embodiment of the present invention, the first machine learning model is trained through an SFT (Supervised Fine-Tuning) process.

[0013] To achieve the above objectives, a second aspect of the present invention provides an optimization apparatus for a machine learning model for text expansion. The apparatus includes: a text expansion module for expanding a first text by inputting it into a preset first machine learning model to obtain a second text; an acquisition module for acquiring a sample set of repeated text in the second text and comparing the sample set with a first preset text to obtain a paired dataset, wherein the first preset text is a standard text corresponding to the first text; a model optimization module for training a reward model based on the paired dataset and optimizing the first machine learning model based on the reward model to obtain a second machine learning model; and a model evaluation module for evaluating whether the repetition problem in the output text of the second machine learning model is mitigated, and determining the second machine learning model as the target machine learning model if the repetition problem in the output text of the second machine learning model is mitigated.

[0014] According to an embodiment of the present invention, an optimization apparatus for a machine learning model for text expansion includes a text expansion module that inputs first text into a preset first machine learning model for expansion to obtain second text; an acquisition module that acquires a sample set of repeated text in the second text and compares the sample set with the first preset text to obtain a paired dataset; a model optimization module that trains a reward model based on the paired dataset and optimizes the first machine learning model based on the reward model to obtain a second machine learning model; and a model evaluation module that evaluates whether the repetition problem in the output text of the second machine learning model is mitigated, and if the repetition problem in the output text of the second machine learning model is mitigated, the second machine learning model is determined as the target machine learning model. Therefore, this apparatus can significantly improve the quality of expanded text, reduce the problem of low-quality expanded text caused by repeated content, improve user experience, and increase efficiency.

[0015] To achieve the above objectives, a third aspect of the present invention provides a text expander, comprising: a memory and a processor, wherein the memory stores a program executable on the processor, and when the processor executes the program, it implements an optimization method for a machine learning model for text expansion according to the above embodiments.

[0016] According to the text expander of the present invention, by means of the above-described optimization method for the machine learning model for text expansion, the quality of expanded text can be significantly improved, the problem of low quality expanded text caused by duplicate content can be reduced, the user experience can be improved, and efficiency can be increased.

[0017] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing an optimization program for a machine learning model for text expansion, which, when executed by a processor, implements the optimization method for the machine learning model for text expansion according to the above embodiments.

[0018] According to embodiments of the present invention, the computer-readable storage medium, through the above-described optimization method for the machine learning model for text expansion, can significantly improve the quality of expanded text, reduce the problem of low quality expanded text caused by duplicate content, enhance the user experience, and improve efficiency. Attached Figure Description

[0019] Figure 1 A flowchart illustrating an optimization method for a machine learning model for text expansion according to an embodiment of the present invention;

[0020] Figure 2 A block diagram illustrating the optimization of a machine learning model for text expansion according to an embodiment of the present invention;

[0021] Figure 3 This is a block diagram of a text expander according to an embodiment of the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0023] The following description, with reference to the accompanying drawings, outlines an optimization method for a machine learning model for text expansion, an optimization apparatus for a machine learning model for text expansion, a text expander, and a computer-readable storage medium.

[0024] Figure 1 This is a flowchart of an optimization method for a machine learning model for text expansion according to an embodiment of the present invention.

[0025] like Figure 1 As shown, the optimization method for a machine learning model for text expansion according to an embodiment of the present invention may include the following steps:

[0026] S1, input the first text into the preset first machine learning model for expansion to obtain the second text.

[0027] Here, the first text refers to the text to be expanded, and the second text refers to the expanded text. Specifically, the first text can be input into a pre-defined first machine learning model in a sequence-to-sequence manner to expand the second text. It should be noted that the first text is not in the training set of the first machine learning model.

[0028] S2, obtain a sample set of repeated texts in the second text, and compare the sample set with the first preset text to obtain a paired dataset, wherein the first preset text is the standard text corresponding to the first text.

[0029] Specifically, the first pre-selected text can be the standard text corresponding to the first text after manual review. For example, when the first text is the claims of a patent application, the first pre-selected text can be the specification text revised by a patent attorney. The paired dataset (reject, choose) refers to the mapping relationship between the sample set of repeated text in the second text and the first pre-selected text. Specifically, repeated text can be extracted from the second text as a sample set, and then compared with the first pre-selected text to obtain a paired dataset.

[0030] S3. Train a reward model based on the paired dataset, and optimize the first machine learning model based on the reward model to obtain the second machine learning model.

[0031] Specifically, a reward model can be trained based on a paired dataset, and then a pre-defined model can be optimized using this reward model to obtain a new model, also known as a second machine learning model. It's important to note that different techniques can be used to train the reward model, such as policy gradient methods and actor-critic methods. These methods allow an agent to generate text within a given domain and optimize based on human feedback. Different datasets can be used when training the reward model, and various techniques can be employed to improve the model's performance and reliability.

[0032] S4. Evaluate whether the repetition problem in the output text of the second machine learning model is mitigated, and if the repetition problem in the output text of the second machine learning model is mitigated, determine the second machine learning model as the target machine learning model.

[0033] Specifically, after obtaining the second machine learning model, it can be tested. For example, the first text can be input into the second machine learning model to expand it, and the output text of the second machine learning model can be recognized to determine whether the repetition problem in the output text of the second machine learning model is reduced compared to the output text of the first machine learning model. If the test results show that the repetition problem in the output text of the second machine learning model is reduced, it indicates that the second machine learning model is superior to the first machine learning model. At this point, the second machine learning model can be identified as the target machine learning model. Users can use the target machine learning model to expand short texts into long texts, thereby significantly improving the quality of the expanded text by the machine learning model, reducing the problem of low quality expanded text caused by repetitive content, improving the user experience, and increasing efficiency.

[0034] Therefore, the optimization method of the machine learning model for text expansion in this embodiment of the invention first inputs the first text into a preset first machine learning model for expansion to obtain the second text. Then, it obtains a sample set of repeated text in the second text of the first machine learning model and compares the sample set of the first machine learning model with the first preset text to obtain a paired dataset. Then, it trains a reward model based on the paired dataset of the first machine learning model and optimizes the first machine learning model based on the reward model to obtain the second machine learning model. Finally, it evaluates whether the problem of repetition in the output text of the first machine learning model and the second machine learning model is reduced. If the problem of repetition in the output text of the first machine learning model and the second machine learning model is reduced, the first machine learning model and the second machine learning model are determined as the target machine learning model. This can significantly improve the quality of the machine learning model expanded text, reduce the problem of low quality expanded text caused by repeated content, improve the user experience, and improve efficiency.

[0035] According to an embodiment of the present invention, after determining the second machine learning model as the target machine learning model, the above method further includes: replacing the first machine learning model with the target machine learning model to iteratively optimize the target machine learning model.

[0036] Specifically, after determining the second machine learning model as the target machine learning model, the first machine learning model in step S1 is replaced with the target machine learning model. That is, the first text is input into the target machine learning model for expansion to obtain the second text. Then, steps S2-S4 are executed sequentially to continue collecting new data, training a new reward model, and optimizing the target machine learning model, thereby iteratively optimizing the target machine learning model. This allows for better optimization of the machine learning model and improves its performance.

[0037] According to one embodiment of the present invention, optimizing a first machine learning model based on a reward model to obtain a second machine learning model includes: training and generating the second machine learning model using an RLHF process or a DPO process.

[0038] Specifically, the RLHF process is a reinforcement learning algorithm that incorporates human evaluation to guide model training, making the model's behavior more aligned with human expectations and values. The DPO process is an unsupervised learning method that automatically discovers patterns and regularities in data and extracts useful features. Both processes can be used to train a second machine learning model, but their specific implementations differ. In practice, users can choose the process that best suits their needs for training.

[0039] According to one embodiment of the present invention, evaluating whether the repetition problem of the output text of the second machine learning model is mitigated includes: obtaining the repetition rate of the output text of the second machine learning model; determining whether the repetition rate is less than a preset repetition rate threshold; and determining that the repetition problem of the output text of the second machine learning model is mitigated if the repetition rate is less than the preset repetition rate threshold. The preset repetition rate threshold can be calibrated according to actual conditions and is not limited here. For example, the preset repetition rate threshold can be the repetition rate of the output text of the first machine learning model (the second text).

[0040] Specifically, after obtaining the second machine learning model, it can be tested. For example, the first text can be input into the second machine learning model to expand it. Then, the output text of the second machine learning model is identified to detect duplicate content. The number of characters corresponding to the duplicate content in the output text is divided by the total number of characters in the output text to obtain the duplication rate of the output text of the second machine learning model. The duplication rate is compared with a preset duplication rate threshold. When the duplication rate is less than the preset duplication rate threshold, it is determined that the duplication problem of the output text of the second machine learning model has been mitigated; when the duplication rate is greater than or equal to the preset duplication rate threshold, it is determined that the duplication problem of the output text of the second machine learning model has not been mitigated.

[0041] According to one embodiment of the present invention, the repeated text consists of a series of consecutive words or phrases.

[0042] Specifically, the sample set of repeated text in the second text can refer to a series of consecutive words or phrases. For example, if the first text is "abcdefghijk", then the sample set of repeated text in the second text can be consecutive occurrences of "bcdefg" or consecutive occurrences of "abcde", etc.

[0043] According to one embodiment of the present invention, the first text is the claims of the patent application, and the second text is the description of the patent application.

[0044] Specifically, when the first text is the claims of a patent application and the second text is the specification of a patent application, the claims can be input into a preset first machine learning model in a sequence-to-sequence manner for expansion to obtain the specification.

[0045] In other embodiments of the present invention, the first text may be a paper abstract, and the second text may be the full text of the paper. Of course, the first text is not limited to the two cases mentioned above, and may also be any other form of text.

[0046] According to one embodiment of the present invention, the first machine learning model is trained through the SFT process.

[0047] Specifically, the first machine learning model can be an LSTM (Long Short-Term Memory) network, a Transformer, etc., and can be pre-trained using the SFT process. It's important to understand that the SFT process is a common technique in machine learning, particularly suitable for domains lacking large amounts of labeled data. It leverages the knowledge learned by a pre-trained model on a large dataset, and through fine-tuning on limited labeled data, quickly obtains a high-performance model for a specific task. The SFT process can specifically include the following steps:

[0048] Selection of pre-trained model: Select a pre-trained model that is relevant to the task. Usually, this model has been trained on a large-scale dataset and has certain general feature extraction capabilities.

[0049] Data collection and labeling: Collect high-quality data that is closely related to the target task for fine-tuning. Then label the fine-tuning data so that the model can learn the correct output. For example, in a text classification task, the correct category label needs to be assigned to each piece of text data;

[0050] Dataset partitioning for fine-tuning: The collected dataset is divided into a training set, a validation set, and a test set. The training set is used for model training, the validation set is used for hyperparameter tuning and model selection, and the test set is used to evaluate the final performance of the model.

[0051] Preparation before fine-tuning: Make necessary modifications to the model to adapt to the new dataset. For example, adjust the output layer to match the new task requirements (such as the number of categories);

[0052] Model fine-tuning: Fine-tuning a pre-trained model using training set data. This process involves further training the model on the training set so that it learns features specific to the task. During fine-tuning, a smaller learning rate than during pre-training is typically used to avoid excessive interference with the pre-trained model's original knowledge.

[0053] Hyperparameter optimization: Evaluate model performance on the validation set and adjust hyperparameters such as learning rate, batch size, and number of training epochs to obtain optimal performance;

[0054] Model Evaluation: The model is comprehensively evaluated using validation and test sets. Key metrics of interest may include accuracy, recall, and F1 score, depending on the nature of the task.

[0055] Model iteration: Based on the evaluation results, it may be necessary to go back and adjust the model architecture, training strategy, or data preprocessing steps, and then repeat the fine-tuning and evaluation process.

[0056] Model saving and deployment: Once the model performs well on the test set, save the model's parameters and deploy it to the production environment to solve real-world problems;

[0057] Continuous monitoring and updates: After model deployment, continuously monitor model performance, collect feedback, and update and maintain the model as needed.

[0058] In summary, the optimization method for a machine learning model for text expansion according to embodiments of the present invention firstly inputs a first text into a preset first machine learning model for expansion to obtain a second text. Then, it obtains a sample set of duplicate text from the second text in the first machine learning model and compares this sample set with the first preset text to obtain a paired dataset. Next, it trains a reward model based on the paired dataset and optimizes the first machine learning model based on the reward model to obtain a second machine learning model. Finally, it evaluates whether the duplication problem in the output text of the first and second machine learning models is mitigated. If the duplication problem in the output text of the first and second machine learning models is mitigated, the first and second machine learning models are determined as the target machine learning model. Therefore, this method can significantly improve the quality of expanded text, reduce the problem of low-quality expanded text caused by duplicate content, enhance the user experience, and improve efficiency.

[0059] Corresponding to the above embodiments, the present invention also proposes an optimization device for a machine learning model for text expansion.

[0060] Figure 2 This is a block diagram of an optimization apparatus for a machine learning model for text expansion according to an embodiment of the present invention.

[0061] like Figure 2 As shown, the optimization device 100 for a machine learning model for text expansion according to an embodiment of the present invention may include: a text expansion module 110, an acquisition module 120, a model optimization module 130, and a model evaluation module 140.

[0062] The text expansion module 110 is used to expand the first text by inputting it into a preset first machine learning model to obtain the second text. The acquisition module 120 is used to acquire a sample set of repeated text in the second text and compare the sample set with the first preset text to obtain a paired dataset, where the first preset text is the standard text corresponding to the first text. The model optimization module 130 is used to train a reward model based on the paired dataset and optimize the first machine learning model based on the reward model to obtain the second machine learning model. The model evaluation module 140 is used to evaluate whether the repetition problem in the output text of the second machine learning model is mitigated, and if the repetition problem in the output text of the second machine learning model is mitigated, the second machine learning model is determined as the target machine learning model.

[0063] According to one embodiment of the present invention, after determining the second machine learning model as the target machine learning model, the device 100 is further configured to replace the first machine learning model with the target machine learning model to perform iterative optimization on the target machine learning model.

[0064] According to one embodiment of the present invention, the model optimization module 130 optimizes the first machine learning model according to the reward model to obtain a second machine learning model. Specifically, it is used to train and generate the second machine learning model using the RLHF process or the DPO process.

[0065] According to one embodiment of the present invention, the model evaluation module 140 evaluates whether the repetition problem of the output text of the second machine learning model is mitigated. Specifically, it is used to obtain the repetition rate of the output text of the second machine learning model; determine whether the repetition rate is less than a preset repetition rate threshold; and determine that the repetition problem of the output text of the second machine learning model is mitigated if the repetition rate is less than the preset repetition rate threshold.

[0066] It should be noted that for details not disclosed in the optimization apparatus for the machine learning model for text expansion in this embodiment of the invention, please refer to the details disclosed in the optimization method for the machine learning model for text expansion in this embodiment of the invention, which will not be repeated here.

[0067] According to an embodiment of the present invention, an optimization apparatus for a machine learning model for text expansion includes a text expansion module that inputs first text into a preset first machine learning model for expansion to obtain second text; an acquisition module that acquires a sample set of repeated text in the second text and compares the sample set with the first preset text to obtain a paired dataset; a model optimization module that trains a reward model based on the paired dataset and optimizes the first machine learning model based on the reward model to obtain a second machine learning model; and a model evaluation module that evaluates whether the repetition problem in the output text of the second machine learning model is mitigated, and if the repetition problem in the output text of the second machine learning model is mitigated, the second machine learning model is determined as the target machine learning model. Therefore, this apparatus can significantly improve the quality of expanded text, reduce the problem of low-quality expanded text caused by repeated content, improve user experience, and increase efficiency.

[0068] Corresponding to the above embodiments, the present invention also proposes a text expander.

[0069] Figure 3 This is a block diagram of a text expander according to an embodiment of the present invention.

[0070] like Figure 3 As shown, the text expander 200 of this embodiment includes: a memory 210 and a processor 220. The memory 210 stores a program that can run on the processor 220. When the processor 220 executes the program, it implements the optimization method of the machine learning model for text expansion according to the above-described embodiments.

[0071] According to the text expander of the present invention, by means of the above-described optimization method for the machine learning model for text expansion, the quality of expanded text can be significantly improved, the problem of low quality expanded text caused by duplicate content can be reduced, the user experience can be improved, and efficiency can be increased.

[0072] Corresponding to the above embodiments, the present invention also proposes a computer-readable storage medium.

[0073] The computer-readable storage medium of this invention stores an optimization program for a machine learning model for text expansion. When the optimization program for the machine learning model for text expansion is executed by a processor, it implements the optimization method for the machine learning model for text expansion according to the above-described embodiments.

[0074] According to embodiments of the present invention, the computer-readable storage medium, through the above-described optimization method for the machine learning model for text expansion, can significantly improve the quality of expanded text, reduce the problem of low quality expanded text caused by duplicate content, enhance the user experience, and improve efficiency.

[0075] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0076] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0077] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0078] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0079] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0080] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An optimization method for a machine learning model used for text expansion, characterized in that, The method includes: The first text is input into a preset first machine learning model for expansion to obtain the second text; Obtain a sample set of repeated texts in the second text, and compare the sample set with the first preset text to obtain a paired dataset, wherein the first preset text is the standard text corresponding to the first text; A reward model is trained based on the paired dataset, and the first machine learning model is optimized based on the reward model to obtain a second machine learning model; The repetition problem in the output text of the second machine learning model is evaluated to determine whether it is mitigated. If the repetition problem in the output text of the second machine learning model is mitigated, the second machine learning model is determined to be the target machine learning model. After determining the second machine learning model as the target machine learning model, the method further includes: The first machine learning model is replaced with the target machine learning model to iteratively optimize the target machine learning model.

2. The optimization method for the machine learning model for text expansion according to claim 1, characterized in that, The first machine learning model is optimized based on the reward model to obtain a second machine learning model, including: The second machine learning model is generated by training using the RLHF or DPO process.

3. The optimization method for the machine learning model for text expansion according to claim 1, characterized in that, Evaluating whether the repetition problem in the output text of the second machine learning model is mitigated includes: Obtain the repetition rate of the text output by the second machine learning model; Determine whether the repetition rate is less than a preset repetition rate threshold; If the repetition rate is less than a preset repetition rate threshold, it is determined that the repetition problem of the output text of the second machine learning model is mitigated.

4. The optimization method for the machine learning model for text expansion according to any one of claims 1-3, characterized in that, The repeated text consists of a series of consecutive words or phrases.

5. The optimization method for the machine learning model for text expansion according to any one of claims 1-3, characterized in that, The first text is the claims of the patent application, and the second text is the description of the patent application.

6. The optimization method for the machine learning model for text expansion according to any one of claims 1-3, characterized in that, The first machine learning model was trained using the SFT process.

7. An optimization apparatus for a machine learning model used for text expansion, characterized in that, The device includes: The text expansion module is used to expand the first text by inputting it into a preset first machine learning model to obtain the second text. The acquisition module is used to acquire a sample set of repeated texts in the second text, and compare the sample set with a first preset text to obtain a paired dataset, wherein the first preset text is the standard text corresponding to the first text; The model optimization module is used to train a reward model based on the paired dataset, and optimize the first machine learning model based on the reward model to obtain a second machine learning model. The model evaluation module is used to evaluate whether the duplication problem of the output text of the second machine learning model is reduced, and if the duplication problem of the output text of the second machine learning model is reduced, the second machine learning model is determined to be the target machine learning model. The apparatus is further configured to replace the first machine learning model with the target machine learning model to iteratively optimize the target machine learning model.

8. A text expander, characterized in that, include: A memory and a processor, the memory storing a program executable on the processor, wherein when the processor executes the program, it implements the optimization method for a machine learning model for text expansion according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores an optimization program for a machine learning model for text expansion, which, when executed by a processor, implements the optimization method for a machine learning model for text expansion according to any one of claims 1-6.

Images