Training method, training sample optimization method and electronic device
By employing clustering and multi-model optimization methods, the problems of complex and costly training of large language models are solved, achieving a more efficient and lower-cost training process and better training results, thereby improving the quality of training samples and model performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2026-03-20
AI Technical Summary
Training large language models is a complex and costly process. The quality and diversity of training data affect model performance. How to complete high-quality training at a lower cost and in a shorter time has become an urgent problem for the industry.
By using cluster feature vectors, representative training samples are extracted and trained and optimized using multiple machine learning models. This process includes clustering algorithms, representative feature vector extraction, external database queries, and multi-model optimization, which reduces the number of training samples and improves sample quality.
Shorten training time, reduce hardware costs, improve training efficiency and enhance model performance by using representative and optimized training samples to achieve a more efficient training process and better training results.
Smart Images
Figure CN121707009A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a training method and a sample optimization method that can accelerate the training process of machine learning models and improve training results. Background Technology
[0002] In recent years, with the rapid development of artificial intelligence technology, large language models (LLMs) have become a core technology in many fields. Leveraging their powerful language understanding and generation capabilities, they have brought significant benefits and technological revolutions to various industries. These models are widely used in scenarios such as production line optimization, administrative efficiency improvement, education and training, user services, game design, and in-vehicle voice control. Through large language models, enterprises can achieve more automated and intelligent processes and services, greatly improving productivity and reducing labor costs.
[0003] However, training these large language models is extremely complex and costly. A typical large language model may contain billions or even hundreds of billions of parameters, requiring enormous amounts of computing resources. Training these models usually requires a large amount of high-performance hardware resources and takes weeks to months, which not only increases hardware costs but also increases the resource investment of enterprises, especially when fine-tuning the model, where resource consumption is particularly significant.
[0004] Besides hardware costs, the quality of training samples is also a key factor affecting model performance. The diversity and accuracy of training data directly impact the final performance of the language model. To ensure the quality of training data, extensive manual cleaning and screening are usually required, which increases development costs and time pressure. As model size increases, how to train high-quality language models at a lower cost and in a shorter time has become an urgent problem to be solved in the industry. Summary of the Invention
[0005] This invention proposes a training method and an optimization method that can accelerate the training process and optimize the quality of training samples to improve training results.
[0006] This invention proposes a method for training a machine learning model, executed by an electronic device. This training method includes: obtaining multiple training samples; inputting each training sample into a first machine learning model to obtain a corresponding feature vector; grouping these feature vectors to obtain multiple groups, each group containing a subset of the feature vectors; extracting a representative feature vector from each group, this representative feature vector corresponding to a representative training sample, wherein the number of representative training samples is less than the number of training samples; and training a second machine learning model based on the representative training samples.
[0007] In one embodiment of the present invention, the second machine learning model described above is a pre-trained model.
[0008] In one embodiment of the present invention, the step of grouping feature vectors to obtain groups includes: calculating the similarity between feature vectors; and if the similarity between two feature vectors is greater than a similarity threshold, grouping the two feature vectors into the same group.
[0009] In one embodiment of the present invention, the step of extracting representative feature vectors from each group includes: constructing a graph for a first group within the group, the graph containing multiple vertices and at least one edge. Vertices correspond to feature vectors in the first group, and edges represent similarities between feature vectors in the first group that are greater than a similarity threshold. Finally, the vertex with the highest number of connections is designated as the representative feature vector.
[0010] In one embodiment of the present invention, each training sample includes a question and an answer. The step of training the second machine learning model based on the representative training samples includes: for a first group, querying an external database to obtain multiple supplementary data based on the questions of the training samples in the first group; inputting the representative training samples of the first group and a first prompt into a third machine learning model to obtain comment data; inputting the representative training samples, supplementary data, comment data, and a second prompt of the first group into a fourth machine learning model to obtain optimized samples corresponding to the representative training samples, wherein the fourth machine learning model is different from the third machine learning model; and training the second machine learning model based on the optimized samples.
[0011] In one embodiment of the present invention, the third machine learning model described above is a language model, and the first prompt is used to indicate the correctness, fluency, and completeness of the evaluation answer.
[0012] In one embodiment of the present invention, the fourth machine learning model is a language model, and the second prompt is used to indicate the answer of the representative training samples of the first group to be adjusted according to supplementary data and comment data.
[0013] From another perspective, embodiments of the present invention propose a method for optimizing training samples, executed by an electronic device. This optimization method includes: (a) obtaining original samples; (b) querying an external database based on the original samples to obtain supplementary data; (c) inputting the original samples and a first prompt into a third machine learning model to obtain comment data; and (d) inputting the original samples, supplementary data, comment data, and a second prompt into a fourth machine learning model to obtain optimized samples corresponding to the original samples, wherein the third machine learning model is different from the fourth machine learning model.
[0014] In one embodiment of the present invention, the above-described optimization method further includes: replacing the original sample with the optimized sample and repeating steps (c) and (d).
[0015] In one embodiment of the present invention, the original sample described above contains a question and an answer, and step (b) includes: querying an external database based on the question to obtain supplementary data.
[0016] From another perspective, embodiments of the present invention provide an electronic device comprising a processor and a memory, wherein the memory is used to store multiple instructions, and the processor is communicatively connected to the memory to execute the instructions to complete the aforementioned training method or optimization method.
[0017] To make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of an electronic device according to one embodiment.
[0019] Figure 2 This is a flowchart illustrating a training method for a machine learning model according to the first embodiment.
[0020] Figure 3 This is a schematic diagram illustrating the creation of a graph based on a group, according to the first embodiment.
[0021] Figure 4 This is a flowchart illustrating the optimization method for training samples according to the second embodiment.
[0022] Figure 5 This is a flowchart illustrating the training method according to the third embodiment. Detailed Implementation
[0023] Some embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Component symbols used in the following description, when appearing in different drawings, are considered to be the same or similar components. These embodiments are only a part of the present invention and do not disclose all possible implementations of the invention. More precisely, these embodiments are merely examples of systems and methods within the scope of the present invention's patent application.
[0024] The terms "first," "second," etc., used in this article do not specifically refer to order or sequence; they are merely used to distinguish components or operations described using the same technical terms.
[0025] Figure 1 This is a schematic diagram illustrating an electronic device according to one embodiment. Please refer to... Figure 1The electronic device 100 can be a smartphone, tablet computer, personal computer, laptop computer, server, distributed computer, cloud server, industrial computer, or various electronic devices with computing capabilities, etc., and this invention is not limited thereto. The electronic device 100 includes a processor 110 and a memory 120. The processor 110 is communicatively connected to the memory 120, and this communication connection can be established through any wired or wireless communication means, or it can be established via the Internet. The processor 110 can be a central processing unit, graphics processing unit, tensor processing unit, application-specific integrated circuit (ASIC), programmable logic device (PLD), etc. The memory 120 can be random access memory, read-only memory, flash memory, floppy disk, hard disk, optical disk, USB flash drive, magnetic tape, or a database accessible via the Internet, storing multiple instructions that the processor 110 executes to perform the methods described below.
[0026] This paper proposes a training method for a machine learning model and an optimization method for training samples. These two methods can be executed together or separately. Several examples will be given below to illustrate this.
[0027] [First Embodiment]
[0028] First, let's explain the training methods for machine learning models. In known techniques, it's generally believed that the more training samples, the better. However, in some cases, the repetition among these training samples may be high, which can actually hinder training results. The proposed method here is to reduce the number of training samples.
[0029] Figure 2 This is a flowchart illustrating the training method of a machine learning model according to the first embodiment. Please refer to... Figure 2 In step 201, multiple training samples are obtained. In some embodiments, each training sample contains a pair of input data and output data for supervised learning. The input data can be images, text, sound signals, data measured by various sensors, etc., and this invention is not limited thereto. Furthermore, the output data can be a label, text, image, sound, etc., and this invention is not limited thereto. In some embodiments, the training sample may also contain only input data for unsupervised learning. In other words, the training method proposed herein is applicable to any type of data.
[0030] In step 202, each training sample is input into the first machine learning model to obtain the corresponding feature vector. In some fields, the feature vector can also be called an embedding. Any known model can be used as the first machine learning model. For example, if the training samples are about images, the first machine learning model can include a convolutional neural network, and the architecture of this network can be LeNet, AlexNet, VGG, GoogLeNet, ResNet, DenseNet, or YOLO (You Only Look Once), etc. If the training samples are about text, the first machine learning model can be BERT (Bidirectional Encoder Representations from Transformers), the GPT (Generative Pre-trained Transformers) series, USE (Universal Sentence Encoder), etc., and this invention is not limited thereto. If the training samples are about sound signals, the first machine learning model can be a Mel-frequency cepstrum model, VGGish, OpenL3, etc., and this invention is not limited thereto. In some embodiments, the training samples include pairs of input data and output data. The feature vector can be generated based on the input data alone, or the input data and output data can be concatenated together before generating the feature vector.
[0031] In step 203, the feature vectors are clustered to obtain multiple groups, where each group contains a subset of the feature vectors. Any clustering algorithm can be used here, such as K-means, Hierarchical Clustering, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), etc.
[0032] In some embodiments, a bottom-up hierarchical clustering algorithm can be used. For example, the similarity between every two feature vectors can be calculated first. This similarity can be cosine similarity, Euclidean distance, etc., and this invention is not limited to this. If the similarity between two feature vectors is greater than a similarity threshold, then these two feature vectors are grouped into the same group. Other feature vectors can also be added to this group through similarity calculation. Each group can be used to build a graph. Figure 3This is a schematic diagram illustrating the creation of a graph based on a group, according to the first embodiment. Figure 3 In the example graph, there are vertices 301-305 and edges 311-314. Vertices 301-305 correspond to feature vectors within the same group, and each edge represents the similarity between two corresponding feature vectors being greater than a similarity threshold. For example, edge 311 connects to vertices 301 and 302, therefore edge 311 indicates that the similarity between the two feature vectors corresponding to vertices 301 and 302 is greater than the similarity threshold.
[0033] Please refer to the reply. Figure 2 In step 204, a representative feature vector is extracted from each group, and this representative feature vector corresponds to a representative training sample. Figure 3 In the example, the vertex with the maximum number of connections among all vertices 301-305 can be set as the representative feature vector. The number of connections is also called the degree of a vertex. In this example, vertex 301 has 1 connection, vertex 302 has 3 connections, vertex 303 has 2 connections, and vertices 304 and 305 have 1 connection. Therefore, vertex 302 has the maximum number of connections, and the feature vector corresponding to vertex 302 will be set as the representative feature vector. In other embodiments, the average vector of all feature vectors in the group can be calculated, and the feature vector closest to this average vector can be taken as the representative feature vector. Alternatively, in other embodiments, principal component analysis can be performed on all feature vectors in the group, and then the principal component with the maximum eigenvalue can be found. The feature vector closest to this principal component can be set as the representative feature vector. Since clustering has been performed first, the number of all representative training samples will be less than the number of original training samples.
[0034] In step 205, a second machine learning model is trained based on representative training samples. Here, the representative training samples can be directly used as input to the second machine learning model for learning, or other processing can be performed on the representative training samples before learning. Since the number of representative training samples is less than the number of original training samples, this training process takes less time, but because the representative training samples contain information from similar training samples, it does not significantly reduce performance.
[0035] In some embodiments, the second machine learning model is a pre-trained model, such as BERT. In this embodiment, the second machine learning model is fine-tuned based on multiple representative training samples. This fine-tuning method includes, for example, low-rank adaptation (LoRA), adapters, and knowledge distillation fine-tuning. The fine-tuned model can be applied to a specific technical field, such as steelmaking, semiconductors, finance, etc., but this invention is not limited thereto. In some embodiments, the aforementioned pre-training is a language model. The pre-trained model already contains a lot of textual information, including common sense and grammar, but contains less information about a specific technical field. Therefore, using representative training samples to train the second machine learning model can incorporate information about that specific technical field. Although the number of samples is reduced, this also removes noise or redundant information, so it does not significantly reduce performance. Alternatively, in some embodiments, the second machine learning model is used to generate videos or images. The pre-trained model lacks information about a particular art style, and the second machine learning model is fine-tuned to generate videos or images with a new art style.
[0036] For example, in some embodiments, a question-answering model about steelmaking is implemented, so each training sample contains a question and an answer. The questions in different training samples may be very similar, such as "What is stainless steel?", "What is stainless steel made of?", and "What is stainless steel made of?". These questions are essentially very similar, and there is some redundant information among these training samples. The above approach groups these training samples into the same group. Furthermore, if there are too many similar training samples, there may be an imbalance problem, such as the number of samples in one category being much greater than the number of samples in another category. In this case, the training of the entire model will be biased towards the category with more samples. However, in the above embodiments, by using representative training samples, the problems of redundant information and imbalance can be solved.
[0037] [Second Embodiment]
[0038] The second embodiment proposes a method for optimizing training samples, which can improve the quality of training samples. Figure 4 This is a flowchart illustrating the optimization method for training samples according to the second embodiment. Please refer to... Figure 4 In step 401, the original sample is obtained. This original sample can be an image, text, sound signal, data measured by various sensors, etc., and this invention is not limited thereto.
[0039] In step 402, an external database is queried based on the original sample to obtain supplementary data. The aforementioned external database may be, for example, a database related to a specific technical field, and may contain text or audio / video data. Any retrieval algorithm can be used to query the database; this invention is not limited to this. For example, the original sample contains text, while the external database contains multiple documents. In some embodiments, the TF-IDF (Term Frequency-Inverse Document Frequency) metric can be used to find relevant paragraphs, articles, or sentences as supplementary data. Alternatively, the original sample can be converted into a feature vector, and then the articles in the external database can also be converted into feature vectors. Relevant supplementary data can be found by calculating the similarity of the feature vectors. In some embodiments, the original sample is an image, and the external database also contains images or videos. Similarly, similar images or videos can be found from the external database through feature vector calculation. In some embodiments, the original sample is used to train a question-answering model, therefore the original sample contains text such as questions and answers. In step 402, the external database can be queried based on the question in the original sample to obtain supplementary data.
[0040] In step 403, the original sample and the first prompt are input into the third machine learning model to obtain comment data. The third machine learning model is used to judge the quality of the original sample. The comment data may contain text or scores representing quality, etc.
[0041] In some embodiments, the original samples are used to train the question-answering model, and therefore contain text such as questions and answers. The third machine learning model is a language model, and the first prompt is used to indicate the correctness, fluency, and completeness of the answer. For example, the question in the original sample is "Why are there un-deoxidized steel ingots, fully-deoxidized steel ingots, and semi-deoxidized steel ingots based on the degree of deoxidation?", and the answer is "Based on the degree of deoxidation of the steel ingot before casting, it can be divided into un-deoxidized steel ingots, fully-deoxidized steel ingots, and semi-deoxidized steel ingots." After providing the question and answer of the original sample to the third machine learning model, the first prompt can be set to "For the given question, please evaluate whether the answer is correct, fluent, and complete." The comment data provided by the third machine learning model is, for example, "This answer is correct, but not complete enough."
[0042] In some embodiments, the original sample is a generated image, with the goal of generating an Impressionist-style image. The third machine learning model is a large language model capable of receiving images. After the original sample is provided to the third machine learning model, the first prompt mentioned above could be "Please evaluate whether this image conforms to the Impressionist style." In some embodiments, the original sample is used to train a customer service model, and the first prompt mentioned above could include "Please evaluate whether this response is concise and polite."
[0043] In step 404, the original sample, the supplementary data obtained in step 402, the comment data obtained in step 403, and the second prompt are input into the fourth machine learning model to obtain an optimized sample corresponding to the original sample. The fourth machine learning model is different from the third machine learning model. In this embodiment, the third machine learning model is used for commenting, while the fourth machine learning model is used to optimize the sample. This is because different machine learning models have different strengths, and separating commenting and optimization into two steps can also avoid the blind spots of a single model.
[0044] Continuing with the question-and-answer model example above, the fourth machine learning model is also a language model. The second prompt is used to instruct on adjusting the answer based on supplementary and comment data. For example, the second prompt could be, "Based on the above question, supplementary data, and comment data, directly refine the answer to make it more accurate and complete." Then, the output of the fourth machine learning model can be used as an optimization sample. Taking the example of steel ingots above, the output of the fourth machine learning model could be, for example, "Different deoxidation levels allow us to distinguish between unrefined, fully refined, and semi-refined steel ingots. They each have their own characteristics and specific uses. Considering the optimal choice for each ingot before casting is also crucial, involving different decarburization processes and mechanical performance requirements. Unrefined steel ingots, also known as 'heartless' ingots, because their surface has not undergone special treatment, are advantageous for casting using the pressure-hollow method when a molten steel distributor is installed on top. Its advantages…", which is more complete than the previous answer.
[0045] In the example of the generated image above, the second prompt could be "Redraw the image directly based on the above image, supplementary data, and comment data to make it closer to the Impressionist style," while the fourth machine learning model can generate a new image as an optimization sample.
[0046] Next, in step 405, it is determined whether to continue optimization. In some embodiments, the decision to continue optimization can be based on the comment data output by the third machine learning model. For example, when the comment data is text, it can be determined whether any of the correctness, fluency, or completeness is insufficient; if so, optimization continues. Alternatively, when the comment data is a score, it can be determined whether the score exceeds a threshold; if so, optimization stops. In some embodiments, it can also be determined whether the number of optimization attempts has reached a threshold; if so, optimization stops.
[0047] If the result of step 405 is yes, then the original sample is replaced with the optimized sample, and steps 403 and 404 are repeated. Continuing with the question-answering model example above, the optimized sample can be used to replace the answer in the original sample. In step 403, the original question and the updated answer can be input into the third machine learning model. In step 404, the original question, the updated answer, new comment data, supplementary data, and the second hint are input into the fourth machine learning model, thereby optimizing the sample again.
[0048] If the result of step 405 is negative, then the process ends. Figure 4 The process involves multiple optimizations to generate higher-quality samples, which can then be used to train a machine learning model.
[0049] [Third Embodiment]
[0050] The third embodiment combines the first and second embodiments described above, which not only reduces the number of training samples but also improves the quality of the samples. Figure 5 This is a flowchart illustrating the training method according to the third embodiment. Please refer to... Figure 5 In step 501, multiple training samples are obtained. In step 502, the number of training samples is reduced to obtain representative training samples. Step 502 may include... Figure 2 Steps 202-204. In other words, these training samples will be divided into multiple groups, and a representative training sample can be extracted from each group.
[0051] Next, proceed to step 503 to optimize the representative training samples. Step 503 may include... Figure 4Steps 402-405 involve replacing the original samples with representative training samples. Specifically, in step 402, an external database can be queried based on all training samples within the group to which the representative training sample belongs. Again, using a question-answering model as an example, each training sample contains a question and an answer. Taking a first group as an example, the external database can be queried based on the questions of all training samples in this group. Each training sample's question yields supplementary data, resulting in multiple sets of supplementary data. In step 403, the representative training samples of the first group and the first hint are input into the third machine learning model to obtain comment data. In step 404, the representative training samples of the first group, multiple sets of supplementary data, comment data, and the second hint are input into the fourth machine learning model to obtain optimized samples corresponding to the representative training samples. In other words, the second hint instructs the adjustment of the answer of the representative training sample of the first group based on the supplementary data and comment data, making the answer more accurate, fluent, and complete. Since each representative training sample can be optimized using multiple sets of supplementary data, the quality improvement is greater than using a single set of supplementary data.
[0052] After performing step 503 on all representative training samples, multiple optimized samples can be obtained. Next, execute... Figure 5 Step 504 involves training a second machine learning model based on optimized samples, for example, fine-tuning a pre-trained model based on optimized samples.
[0053] [Experimental Results]
[0054] Please refer to the process in the third embodiment. In one experiment, the original number of training samples was 42,144, and after clustering, the number of representative training samples was 17,267. The third machine learning model mentioned above is, for example, Breeze-7b, while the fourth machine learning model can be Qwen2-7b. The relevant results are shown in Table 1 below, where correctness, readability, and reasonableness are scored using GPT-4.
[0055]
[0056] Table 1
[0057] In Experiment 1 (Table 1), the question-answering model was trained using only the original training samples, resulting in a sample size of 42,144 and an average score of 8.02. In Experiment 2, the sample size was reduced to 17,267 representative training samples, and the model was trained using these representative samples (without optimization), achieving an average score of 8.78. It can be seen that Experiment 2 used fewer samples but achieved better results than Experiment 1. In Experiment 3, the original training samples were used, and supplementary data was added to the training samples by querying an external database, resulting in an average score of 5.9. In Experiment 4, optimized representative training samples were used for training, resulting in an average score of 9.16 (the highest score among all experiments).
[0058] Furthermore, using a smaller number of representative training samples can shorten the training time, while the optimized representative training samples have more tokens, which increases the training time. The combined effect of the two is shown in Table 2 below.
[0059]
[0060] Table 2
[0061] As shown in Table 2, although the number of symbols is larger when using the optimized representative training samples, the overall training time is still reduced by 22.5%. Therefore, this approach can not only improve training efficiency but also reduce training time.
[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for training a machine learning model, executed by an electronic device, characterized in that, The training method includes: Obtain multiple training samples; Each of the plurality of training samples is input into the first machine learning model to obtain the corresponding feature vector; The multiple feature vectors corresponding to the multiple training samples are grouped to obtain multiple groups, wherein each of the multiple groups contains a portion of the multiple feature vectors; A representative feature vector is extracted from each of the plurality of groups, and the representative feature vector corresponds to a representative training sample in the plurality of training samples, wherein the number of representative training samples corresponding to the plurality of groups is less than the number of the plurality of training samples; as well as The second machine learning model is trained based on the representative training samples.
2. The training method according to claim 1, characterized in that, The second machine learning model is a pre-trained model.
3. The training method according to claim 1, characterized in that, The step of grouping the multiple feature vectors corresponding to the multiple training samples to obtain the multiple groups includes: Calculate the similarity between the multiple feature vectors; and If the similarity between two of the plurality of feature vectors is greater than a similarity threshold, the two of the plurality of feature vectors are grouped into the same group among the plurality of groups.
4. The training method according to claim 3, characterized in that, The step of extracting the representative feature vector from each of the plurality of groups includes: For a first group among the plurality of groups, a graph is constructed, the graph comprising a plurality of vertices and at least one edge, the plurality of vertices corresponding to the plurality of feature vectors in the first group, and the edge representing that the similarity of the plurality of feature vectors in the first group is greater than a similarity threshold; and The vertex with the largest number of connections among the plurality of vertices is set as the representative feature vector.
5. The training method according to claim 1, characterized in that, Each of the plurality of training samples contains a question and an answer, wherein the step of training the second machine learning model based on the representative training samples includes: For the first group among the plurality of groups, external databases are queried based on the plurality of questions of the plurality of training samples in the first group to obtain multiple sets of supplementary data; The representative training samples of the first group and the first prompt are input into the third machine learning model to obtain comment data; The representative training samples of the first group, the multiple supplementary data sets, the comment data, and the second prompt are input into the fourth machine learning model to obtain optimized samples corresponding to the representative training samples, wherein the fourth machine learning model is different from the third machine learning model; and The second machine learning model is trained based on the optimized samples.
6. The training method according to claim 5, characterized in that, The third machine learning model is a language model, and the first prompt is used to indicate the correctness, fluency, and completeness of the answer.
7. The training method according to claim 6, characterized in that, The fourth machine learning model is a language model, and the second prompt is used to instruct the answer of the representative training sample of the first group to be adjusted based on the multiple supplementary data and the comment data.
8. A method for optimizing training samples, executed by an electronic device, characterized in that, The optimization method includes: (a) Obtain the original sample; (b) Query an external database based on the original sample to obtain supplementary data; (c) Inputting the original sample and the first prompt into the first machine learning model to obtain comment data; and (d) Input the original sample, the supplementary data, the comment data and the second prompt into the second machine learning model to obtain an optimized sample corresponding to the original sample, wherein the second machine learning model is different from the first machine learning model.
9. The optimization method according to claim 8, characterized in that, Also includes: Replace the original sample with the optimized sample and repeat steps (c) and (d).
10. The optimization method according to claim 8, characterized in that, The original sample contains a question and an answer, and step (b) includes: The external database is queried based on the question to obtain the supplementary data.
11. The optimization method according to claim 10, characterized in that, The original sample contains text, the first machine learning model is a language model, and the first prompt is used to indicate the correctness, fluency, and completeness of the answer.
12. The optimization method according to claim 10, characterized in that, The second machine learning model is a language model, and the second prompt is used to instruct the answer to be adjusted based on the supplementary data and the comment data.
13. An electronic device, characterized in that, Include: Memory, used to store multiple instructions; The processor, communicatively connected to the memory, executes the plurality of instructions to complete a plurality of steps: Obtain multiple training samples; Each of the plurality of training samples is input into the first machine learning model to obtain the corresponding feature vector; The multiple feature vectors corresponding to the multiple training samples are grouped to obtain multiple groups, wherein each of the multiple groups contains a portion of the multiple feature vectors; A representative feature vector is extracted from each of the plurality of groups, and the representative feature vector corresponds to a representative training sample in the plurality of training samples, wherein the number of representative training samples corresponding to the plurality of groups is less than the number of the plurality of training samples; as well as The second machine learning model is trained based on the representative training samples.
14. The electronic device according to claim 13, characterized in that, The second machine learning model is a pre-trained model.
15. The electronic device according to claim 13, characterized in that, The step of grouping the multiple feature vectors corresponding to the multiple training samples to obtain the multiple groups includes: Calculate the similarity between the multiple feature vectors; and If the similarity between two of the plurality of feature vectors is greater than a similarity threshold, the two of the plurality of feature vectors are grouped into the same group among the plurality of groups.
16. The electronic device according to claim 15, characterized in that, The step of extracting the representative feature vector from each of the plurality of groups includes: For a first group among the plurality of groups, a graph is constructed, the graph comprising a plurality of vertices and at least one edge, the plurality of vertices corresponding to the plurality of feature vectors in the first group, and the edge representing that the similarity of the plurality of feature vectors in the first group is greater than a similarity threshold; and The vertex with the largest number of connections among the plurality of vertices is set as the representative feature vector.
17. The electronic device according to claim 13, characterized in that, Each of the plurality of training samples contains a question and an answer, wherein the step of training the second machine learning model based on the representative training samples includes: For the first group among the plurality of groups, external databases are queried based on the plurality of questions of the plurality of training samples in the first group to obtain multiple sets of supplementary data; The representative training samples of the first group and the first prompt are input into the third machine learning model to obtain comment data; The representative training samples of the first group, the multiple supplementary data sets, the comment data, and the second prompt are input into the fourth machine learning model to obtain optimized samples corresponding to the representative training samples, wherein the fourth machine learning model is different from the third machine learning model; and The second machine learning model is trained based on the optimized samples.
18. The electronic device according to claim 17, characterized in that, The third machine learning model is a language model, and the first prompt is used to indicate the correctness, fluency, and completeness of the answer.
19. The electronic device according to claim 18, characterized in that, The fourth machine learning model is a language model, and the second prompt is used to instruct the answer of the representative training sample of the first group to be adjusted based on the plurality of supplementary data and the comment data.