Method and device for optimizing machine learning model for text extension

Abstract

The invention discloses an optimization method and device for a machine learning model for text extension, and the method comprises the steps: inputting a first text into a preset first machine learning model for extension, and obtaining a second text; obtaining a sample set of repeated texts in the second text, and comparing the sample set with a first preset text to obtain a paired data set; training a reward model according to the paired data set, and optimizing the first machine learning model according to the reward model to obtain a second machine learning model; and evaluating whether the repetition problem of the output text of the second machine learning model is relieved or not, and determining the second machine learning model as a target machine learning model under the condition that the repetition problem of the output text of the second machine learning model is relieved. According to the method, the quality of the extended text can be remarkably improved, the problem of low quality of the extended text caused by repeated content is reduced, the user experience is improved, and the efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of text processing, and in particular relates to an optimization method for a machine learning model for text expansion, an optimization device for a machine learning model for text expansion, a text expander and a computer-readable storage medium. Background Art

[0002] In the related art, when a user uses a large model to expand a short text into a long text, sometimes large sections of repeated content will appear in the output long text, making the expanded long text unable to be used directly. The user is also required to check and modify the long text by himself, resulting in a poor user experience and low efficiency. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to some extent. To this end, a first object of the present invention is to propose an optimization method for a machine learning model for text expansion.

[0004] To achieve the above-mentioned purpose, the first aspect of the present invention proposes an optimization method for a machine learning model for text expansion, the method comprising: inputting a first text into a preset first machine learning model for expansion to obtain a second text; obtaining a sample set of repeated text in the second text, and comparing the sample set with the first preset text to obtain a paired data set, wherein the first preset text is a standard text corresponding to the first text; training a reward model according to the paired data set, and optimizing the first machine learning model according to the reward model to obtain a second machine learning model; evaluating whether the repetition problem of the text output by the second machine learning model is alleviated, and determining the second machine learning model as the target machine learning model when the repetition problem of the text output by the second machine learning model is alleviated.

[0005] According to the optimization method of the machine learning model for text expansion of the embodiment of the present invention, firstly, the first text is input into the preset first machine learning model for expansion to obtain the second text, then the sample set of repeated text in the second text of the first machine learning model is obtained, and the sample set of the first machine learning model is compared with the first preset text to obtain the paired data set, and then the reward model is trained according to the paired data set of the first machine learning model, and the first machine learning model is optimized according to the reward model of the first machine learning model to obtain the second machine learning model, and finally, it is evaluated whether the duplication problem of the output text of the first machine learning model and the second machine learning model is alleviated, and when the duplication problem of the output text of the first machine learning model and the second machine learning model is alleviated, the first machine learning model and the second machine learning model are determined as the target machine learning model. Therefore, the method can significantly improve the quality of the extended text, reduce the problem of low quality of the extended text caused by repeated content, enhance the user experience, and improve efficiency.

[0006] In addition, the optimization method of the machine learning model for text expansion according to the above embodiment of the present invention may also have the following additional technical features: According to one embodiment of the present invention, after determining that the second machine learning model is the target machine learning model, the method further includes: replacing the first machine learning model with the target machine learning model to iteratively optimize the target machine learning model.

[0007] According to one embodiment of the present invention, the first machine learning model is optimized according to the reward model to obtain a second machine learning model, including: using an RLHF (Reward Learning from Human Feedback) process or a DPO (Differentiable Policy Optimization) process to train and generate the second machine learning model.

[0008] According to one embodiment of the present invention, evaluating whether the repetition problem of the text output by the second machine learning model is alleviated includes: obtaining the repetition rate of the text output by the second machine learning model; determining whether the repetition rate is less than a preset repetition rate threshold; and determining that the repetition problem of the text output by the second machine learning model is alleviated when the repetition rate is less than the preset repetition rate threshold.

[0009] According to an embodiment of the present invention, the repeated text is a plurality of consecutive words or phrases.

[0010] According to an embodiment of the present invention, the first text is a claim, and the second text is a specification.

[0011] According to one embodiment of the present invention, the first machine learning model is trained through an SFT (Supervised Fine-Tuning) process.

[0012] To achieve the above-mentioned purpose, the second aspect of the present invention proposes an optimization device for a machine learning model for text expansion, the device comprising: a text expansion module, used to input a first text into a preset first machine learning model for expansion to obtain a second text; an acquisition module, used to obtain a sample set of repeated text in the second text, and compare the sample set with the first preset text to obtain a paired data set, wherein the first preset text is a standard text corresponding to the first text; a model optimization module, used to train a reward model according to the paired data set, and optimize the first machine learning model according to the reward model to obtain a second machine learning model; a model evaluation module, used to evaluate whether the repetition problem of the text output by the second machine learning model is alleviated, and when the repetition problem of the text output by the second machine learning model is alleviated, determine that the second machine learning model is the target machine learning model.

[0013] According to the optimization device of the machine learning model for text expansion of the embodiment of the present invention, the text expansion module inputs the first text into the preset first machine learning model for expansion to obtain the second text; the acquisition module obtains the sample set of repeated text in the second text, and compares the sample set with the first preset text to obtain the paired data set; the model optimization module trains the reward model according to the paired data set, and optimizes the first machine learning model according to the reward model to obtain the second machine learning model; the model evaluation module evaluates whether the duplication problem of the text output by the second machine learning model is alleviated, and when the duplication problem of the text output by the second machine learning model is alleviated, the second machine learning model is determined as the target machine learning model. Thus, the device can significantly improve the quality of the expanded text, reduce the problem of low quality of the expanded text caused by repeated content, enhance the user experience, and improve efficiency.

[0014] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a text expander, comprising: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the optimization method of the machine learning model for text expansion according to the above-mentioned implementation is implemented.

[0015] The text expander according to the embodiment of the present invention can significantly improve the quality of the expanded text, reduce the problem of low quality of the expanded text caused by repeated content, enhance the user experience, and improve efficiency through the above-mentioned optimization method of the machine learning model for text expansion.

[0016] To achieve the above-mentioned objectives, an embodiment of the fourth aspect of the present invention proposes a computer-readable storage medium, on which an optimization program for a machine learning model for text expansion is stored. When the optimization program for the machine learning model for text expansion is executed by a processor, the optimization method for the machine learning model for text expansion according to the above-mentioned embodiment is implemented.

[0017] According to the computer-readable storage medium of an embodiment of the present invention, through the above-mentioned optimization method of the machine learning model for text expansion, the quality of the expanded text can be significantly improved, the problem of low quality of the expanded text caused by repeated content can be reduced, the user experience can be enhanced, and efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of an optimization method for a machine learning model for text expansion according to an embodiment of the present invention; Figure 2 A schematic block diagram of an optimization of a machine learning model for text expansion according to an embodiment of the present invention; Figure 3 FIG. 4 is a block diagram of a text expander according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0020] The following describes, with reference to the accompanying drawings, an optimization method for a machine learning model for text expansion, an optimization device for a machine learning model for text expansion, a text expander, and a computer-readable storage medium proposed in embodiments of the present invention.

[0021] Figure 1 The present invention is a flowchart of an optimization method for a machine learning model for text expansion according to an embodiment of the present invention.

[0022] like Figure 1 As shown, the optimization method of the machine learning model for text expansion in an embodiment of the present invention may include the following steps: S1, inputting the first text into a preset first machine learning model for expansion to obtain a second text.

[0023] The first text refers to the text to be expanded, and the second text refers to the text after expansion. Specifically, the first text can be input into a preset first machine learning model in a sequence-to-sequence manner for expansion to obtain the second text. It should be noted that the first text is not in the training set of the first machine learning model.

[0024] S2, obtaining a sample set of repeated texts in the second text, and comparing the sample set with a first preset text to obtain a paired data set, wherein the first preset text is a standard text corresponding to the first text.

[0025] Specifically, the first preset text may be a standard text corresponding to the first text after manual review. For example, when the first text is the claims of a patent application, the first preset text may be the specification text modified by a patent agent. A paired data set (reject, chosen) refers to the mapping relationship between a sample set of repeated text in the second text and the first preset text. Specifically, repeated text may be extracted from the second text as a sample set, and then the sample set may be compared with the first preset text to obtain a paired data set.

[0026] S3: training a reward model according to the paired data set, and optimizing the first machine learning model according to the reward model to obtain a second machine learning model.

[0027] Specifically, a reward model can be trained based on the paired data set, and then the preset model can be optimized based on the reward model to obtain a new model, which is the second machine learning model. It should be noted that the training of the reward model can adopt different techniques, such as Policy Gradient, Actor-Critic Methods, etc. These methods allow the agent to generate text in a given domain and optimize it based on human feedback. When training the reward model, different data sets can be used, and various technical means can be used to improve the performance and reliability of the model.

[0028] S4, evaluating whether the repetition problem of the text output by the second machine learning model is alleviated, and determining the second machine learning model as the target machine learning model if the repetition problem of the text output by the second machine learning model is alleviated.

[0029] Specifically, after obtaining the second machine learning model, the second machine learning model can be tested. For example, the first text can be input into the second machine learning model to expand the first text, and the output text of the second machine learning model can be recognized to determine whether the duplication problem of the output text of the second machine learning model is reduced compared with the output text of the first machine learning model. If the test results show that the duplication problem of the output text of the second machine learning model is reduced, it indicates that the second machine learning model is better than the first machine learning model. At this time, the second machine learning model can be determined as the target machine learning model. Users can expand short texts into long texts through the target machine learning model, thereby significantly improving the quality of the expanded text of the machine learning model, reducing the problem of low quality of the expanded text caused by repeated content, improving the user experience, and improving efficiency.

[0030] Therefore, the optimization method of the machine learning model for text expansion of the embodiment of the present invention first inputs the first text into the preset first machine learning model for expansion to obtain the second text, then obtains the sample set of repeated text in the second text of the first machine learning model, and compares the sample set of the first machine learning model with the first preset text to obtain a paired data set, and then trains the reward model according to the first machine learning model paired data set, and optimizes the first machine learning model according to the first machine learning model reward model to obtain the second machine learning model, and finally evaluates whether the duplication problem of the output text of the first machine learning model and the second machine learning model is alleviated, and when the duplication problem of the output text of the first machine learning model and the second machine learning model is alleviated, determines the first machine learning model and the second machine learning model as the target machine learning model, thereby significantly improving the quality of the expanded text of the machine learning model, reducing the problem of low quality of the expanded text caused by repeated content, improving the user experience, and improving efficiency.

[0031] According to one embodiment of the present invention, after determining that the second machine learning model is the target machine learning model, the above method also includes: replacing the first machine learning model with the target machine learning model to iteratively optimize the target machine learning model.

[0032] Specifically, after determining that the second machine learning model is the target machine learning model, the first machine learning model in the aforementioned step S1 is replaced with the target machine learning model, that is, the first text is input into the target machine learning model for expansion to obtain the second text, and then the aforementioned steps S2-S4 are performed in sequence, and new data is continuously collected, a new reward model is trained, and the target machine learning model is optimized and generated, so as to iteratively optimize the target machine learning model. In this way, the machine learning model can be better optimized and the performance of the machine learning model can be improved.

[0033] According to one embodiment of the present invention, the first machine learning model is optimized according to the reward model to obtain the second machine learning model, including: using the RLHF process or the DPO process to train and generate the second machine learning model.

[0034] Specifically, the RLHF process is a reinforcement learning algorithm that can guide the training of the model by combining human evaluation, so that the model's behavior is more in line with human expectations and values. The DPO process is an unsupervised learning method that can automatically discover patterns and regularities in data and extract useful features from them. Both processes can be used to train the second machine learning model, but their specific implementation methods will be different. In actual applications, users can choose the process that best suits them for training.

[0035] According to one embodiment of the present invention, evaluating whether the repetition problem of the text output by the second machine learning model is alleviated includes: obtaining the repetition rate of the text output by the second machine learning model; determining whether the repetition rate is less than a preset repetition rate threshold; and determining that the repetition problem of the text output by the second machine learning model is alleviated when the repetition rate is less than the preset repetition rate threshold. The preset repetition rate threshold can be calibrated according to actual conditions and is not limited here. For example, the preset repetition rate threshold can be the repetition rate of the text output by the first machine learning model (the second text).

[0036] Specifically, after obtaining the second machine learning model, the second machine learning model can be tested. For example, the first text can be input into the second machine learning model to expand the first text, and then the output text of the second machine learning model can be identified to identify the repeated content in the output text, and then the number of words corresponding to the repeated content in the output text is divided by the total number of words in the output text to obtain the repetition rate of the output text of the second machine learning model. The repetition rate is compared with the preset repetition rate threshold. When the repetition rate is less than the preset repetition rate threshold, it is determined that the repetition problem of the output text of the second machine learning model has been alleviated; when the repetition rate is greater than or equal to the preset repetition rate threshold, it is determined that the repetition problem of the output text of the second machine learning model has not been alleviated.

[0037] According to an embodiment of the present invention, the repeated text is a plurality of consecutive words or phrases.

[0038] Specifically, the sample set of repeated text in the second text may refer to multiple consecutive words or phrases. For example, if the first text is "abcdefghijk", the sample set of repeated text in the second text may be consecutively appearing "bcdefg" or consecutively appearing "abcde".

[0039] According to one embodiment of the present invention, the first text is the claims of the patent application, and the second text is the description of the patent application.

[0040] Specifically, when the first text is the claims of a patent application and the second text is the description of the patent application, the claims can be input into a preset first machine learning model in a sequence-to-sequence manner for expansion to obtain the description.

[0041] In other embodiments of the present invention, the first text may be a paper abstract, and the second text may be the full text of the paper. Of course, the first text is not limited to the above two situations, and may also be any other form of text.

[0042] According to one embodiment of the present invention, the first machine learning model is trained through the SFT process.

[0043] Specifically, the first machine learning model can be LSTM (Long Short-Term Memory), Transformer, etc., and the first machine learning model can be pre-trained through the SFT process. It should be understood that the SFT process is a common technology in machine learning, especially suitable for those fields that lack a large amount of labeled data. The knowledge learned by the pre-trained model on a large amount of data can be used to quickly obtain a high-performance model for a specific task by fine-tuning on limited labeled data. The SFT process can specifically include the following steps: Selection of pre-trained model: Choose a pre-trained model related to the task. Usually, this model has been trained on a large-scale dataset and has certain general feature extraction capabilities. Data collection and annotation: Collect data for fine-tuning. The data for fine-tuning is of high quality and closely related to the target task. Then, annotate the data for fine-tuning so that the model can learn the correct output. For example, in a text classification task, the correct category label needs to be assigned to each piece of text data. Fine-tuning dataset division: The collected dataset is divided into training set, validation set and test set. The training set is used for model training, the validation set is used for hyperparameter adjustment and model selection, and the test set is used to evaluate the final performance of the model.

[0044] Preparation before fine-tuning: Make necessary modifications to the model to adapt it to the new dataset. For example, adjust the output layer to match the new task requirements (such as the number of categories); Model fine-tuning: Fine-tune the pre-trained model using the training set data. This process involves further training the model on the training set so that the model learns the characteristics of the specific task. During fine-tuning, a smaller learning rate is usually used than during pre-training to avoid causing too much interference with the original knowledge of the pre-trained model; Hyperparameter optimization: Evaluate model performance on the validation set and adjust hyperparameters such as learning rate, batch size, number of training rounds, etc. to achieve the best performance; Model evaluation: Use validation and test sets to fully evaluate the model. Key metrics of interest may include accuracy, recall, F1 score, etc., depending on the nature of the task; Model iteration: Based on the evaluation results, you may need to go back and adjust the model architecture, training strategy, or data preprocessing steps, and then repeat the fine-tuning and evaluation process; Model preservation and deployment: Once the model performs well on the test set, save the model parameters and deploy it to the production environment to solve practical problems; Continuous monitoring and updating: After the model is deployed, continuously monitor model performance, collect feedback, and update and maintain the model as needed.

[0045] In summary, according to the optimization method of the machine learning model for text expansion of the embodiment of the present invention, firstly, the first text is input into the preset first machine learning model for expansion to obtain the second text, then the sample set of repeated text in the second text of the first machine learning model is obtained, and the sample set of the first machine learning model is compared with the first preset text to obtain a paired data set, and then the reward model is trained according to the paired data set of the first machine learning model, and the first machine learning model is optimized according to the reward model of the first machine learning model to obtain the second machine learning model, and finally, it is evaluated whether the duplication problem of the output text of the first machine learning model and the second machine learning model is alleviated, and when the duplication problem of the output text of the first machine learning model and the second machine learning model is alleviated, the first machine learning model and the second machine learning model are determined as the target machine learning model. Therefore, the method can significantly improve the quality of the extended text, reduce the problem of low quality of the extended text caused by repeated content, enhance the user experience, and improve efficiency.

[0046] Corresponding to the above embodiment, the present invention also proposes an optimization device for a machine learning model for text expansion.

[0047] Figure 2 A block diagram of an optimization device for a machine learning model for text expansion according to an embodiment of the present invention.

[0048] like Figure 2As shown, the optimization device 100 for the machine learning model for text expansion according to an embodiment of the present invention may include: a text expansion module 110, an acquisition module 120, a model optimization module 130 and a model evaluation module 140.

[0049] Among them, the text expansion module 110 is used to input the first text into a preset first machine learning model for expansion to obtain a second text. The acquisition module 120 is used to obtain a sample set of repeated text in the second text, and compare the sample set with the first preset text to obtain a paired data set, wherein the first preset text is the standard text corresponding to the first text. The model optimization module 130 is used to train a reward model based on the paired data set, and optimize the first machine learning model based on the reward model to obtain a second machine learning model. The model evaluation module 140 is used to evaluate whether the repetition problem of the text output by the second machine learning model is alleviated, and when the repetition problem of the text output by the second machine learning model is alleviated, determine the second machine learning model as the target machine learning model.

[0050] According to one embodiment of the present invention, after determining that the second machine learning model is the target machine learning model, the device 100 is also used to replace the first machine learning model with the target machine learning model to iteratively optimize the target machine learning model.

[0051] According to one embodiment of the present invention, the model optimization module 130 optimizes the first machine learning model according to the reward model to obtain a second machine learning model, and is specifically used to train and generate the second machine learning model using the RLHF process or the DPO process.

[0052] According to one embodiment of the present invention, the model evaluation module 140 evaluates whether the repetition problem of the text output by the second machine learning model is alleviated, and is specifically used to obtain the repetition rate of the text output by the second machine learning model; determine whether the repetition rate is less than a preset repetition rate threshold; when the repetition rate is less than the preset repetition rate threshold, determine that the repetition problem of the text output by the second machine learning model is alleviated.

[0053] It should be noted that for details not disclosed in the optimization device for the machine learning model for text expansion in the embodiment of the present invention, please refer to the details disclosed in the optimization method for the machine learning model for text expansion in the embodiment of the present invention, and the details will not be repeated here.

[0054] According to the optimization device of the machine learning model for text expansion of the embodiment of the present invention, the text expansion module inputs the first text into the preset first machine learning model for expansion to obtain the second text; the acquisition module obtains the sample set of repeated text in the second text, and compares the sample set with the first preset text to obtain the paired data set; the model optimization module trains the reward model according to the paired data set, and optimizes the first machine learning model according to the reward model to obtain the second machine learning model; the model evaluation module evaluates whether the duplication problem of the text output by the second machine learning model is alleviated, and when the duplication problem of the text output by the second machine learning model is alleviated, the second machine learning model is determined as the target machine learning model. Thus, the device can significantly improve the quality of the expanded text, reduce the problem of low quality of the expanded text caused by repeated content, enhance the user experience, and improve efficiency.

[0055] Corresponding to the above embodiment, the present invention also proposes a text expander.

[0056] Figure 3 FIG. 4 is a block diagram of a text expander according to an embodiment of the present invention.

[0057] like Figure 3 As shown, the text expander 200 of the embodiment of the present invention includes: a memory 210 and a processor 220. The memory 210 stores a program that can be run on the processor 220. When the processor 220 executes the program, the optimization method of the machine learning model for text expansion according to the above embodiment is implemented.

[0058] The text expander according to the embodiment of the present invention can significantly improve the quality of the expanded text, reduce the problem of low quality of the expanded text caused by repeated content, enhance the user experience, and improve efficiency through the above-mentioned optimization method of the machine learning model for text expansion.

[0059] Corresponding to the above embodiment, the present invention also proposes a computer-readable storage medium.

[0060] The computer-readable storage medium of an embodiment of the present invention stores thereon an optimization program for a machine learning model for text expansion. When the optimization program for the machine learning model for text expansion is executed by a processor, the optimization method for the machine learning model for text expansion according to the above-mentioned embodiment is implemented.

[0061] According to the computer-readable storage medium of an embodiment of the present invention, through the above-mentioned optimization method of the machine learning model for text expansion, the quality of the expanded text can be significantly improved, the problem of low quality of the expanded text caused by repeated content can be reduced, the user experience can be enhanced, and efficiency can be improved.

[0062] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0063] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0064] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0065] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0066] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0067] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for optimizing a machine learning model for text expansion, characterized in that: The method comprises: Inputting the first text into a preset first machine learning model for expansion to obtain a second text; Acquire a sample set of repeated text in the second text, and compare the sample set with a first preset text to obtain a paired data set, wherein the first preset text is a standard text corresponding to the first text; Training a reward model according to the paired data set, and optimizing the first machine learning model according to the reward model to obtain a second machine learning model; Evaluate whether the repetition problem of the text output by the second machine learning model is alleviated, and if the repetition problem of the text output by the second machine learning model is alleviated, determine that the second machine learning model is the target machine learning model.

2. The optimization method for a machine learning model for text expansion according to claim 1, characterized in that: After determining that the second machine learning model is the target machine learning model, the method further includes: Replace the first machine learning model with the target machine learning model to iteratively optimize the target machine learning model.

3. The optimization method for a machine learning model for text expansion according to claim 1, characterized in that: Optimizing the first machine learning model according to the reward model to obtain a second machine learning model, including: The second machine learning model is generated by training using the RLHF process or the DPO process.

4. The optimization method for a machine learning model for text expansion according to claim 1, characterized in that: Evaluate whether the duplication problem of the text output by the second machine learning model is alleviated, including: Obtaining a repetition rate of the text output by the second machine learning model; Determining whether the repetition rate is less than a preset repetition rate threshold; When the repetition rate is less than a preset repetition rate threshold, it is determined that the repetition problem of the output text of the second machine learning model is alleviated.

5. The method for optimizing a machine learning model for text expansion according to any one of claims 1 to 4, characterized in that: The repeated text is a plurality of consecutive words or phrases.

6. The method for optimizing a machine learning model for text expansion according to any one of claims 1 to 4, characterized in that: The first text is the claims of the patent application, and the second text is the description of the patent application.

7. The method for optimizing a machine learning model for text expansion according to any one of claims 1 to 4, characterized in that: The first machine learning model is trained through the SFT process.

8. An optimization device for a machine learning model for text expansion, characterized in that: The device comprises: A text expansion module, used for inputting a first text into a preset first machine learning model for expansion to obtain a second text; an acquisition module, configured to acquire a sample set of repeated texts in the second text, and compare the sample set with a first preset text to obtain a paired data set, wherein the first preset text is a standard text corresponding to the first text; A model optimization module, used for training a reward model according to the paired data set, and optimizing the first machine learning model according to the reward model to obtain a second machine learning model; A model evaluation module is used to evaluate whether the repetition problem of the text output by the second machine learning model is alleviated, and if the repetition problem of the text output by the second machine learning model is alleviated, determine that the second machine learning model is the target machine learning model.

9. A text expander, characterized in that: include: A memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the optimization method of the machine learning model for text expansion according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: An optimization program for a machine learning model for text expansion is stored thereon. When the optimization program for a machine learning model for text expansion is executed by a processor, the optimization method for a machine learning model for text expansion according to any one of claims 1-7 is implemented.

Images