Training data processing method, training method, device, equipment and medium
By clustering the training data, the problem of data redundancy and high cost in dialogue model training is solved, and the generalization ability and training efficiency of the model are improved.
Patent Information
- Application Number
- CN202311435529.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-31
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-10-31
AI Technical Summary
The quality of training data in the existing dialogue models during the fine-tuning stage of supervised instructions leads to high labor costs and excessive redundancy of instructions, which easily leads to model overfitting.
By obtaining multiple instruction data sets from the original text data, obtaining cluster seeds using at least two clustering algorithms, clustering to form multiple instruction data clusters, and selecting target instruction data from the cluster as training data, for Fine-tuning of the dialogue model's instructions.
It improves the diversity and coverage of training data, reduces labor costs, avoids model overfitting, and enhances the generalization ability of dialogue models.
Smart Images

Figure CN117436505B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to dialogue models, pre-training and instruction fine-tuning technologies, and specifically to a training data processing method, training method, device, equipment and medium. Background Art
[0002] With the emergence of conversation models such as chatGPT, a large number of open source models have emerged in academia and industry, and the overall ecosystem is extremely prosperous. At present, the training of such conversation models mainly includes three steps, namely large-scale pre-training, supervised instruction fine-tuning, and reinforcement learning alignment.
[0003] In the supervised instruction fine-tuning stage, the quality of training data plays an important role in obtaining a good fine-tuning model. At present, in order to obtain high-quality training data, the industry mainly collects many open source NLP (Natural Language Processing) subtasks, and then manually processes them to obtain different task instructions. However, this approach not only has high labor costs, but also the task instructions obtained are too redundant as a whole, which makes it easy for the model to overfit a certain type of data instructions. Summary of the invention
[0004] The present disclosure provides a training data processing method, a training method, an apparatus, a device and a medium.
[0005] According to one aspect of the present disclosure, a training data processing method is provided, comprising:
[0006] Obtain multiple instruction data sets from the original text data;
[0007] Based on at least two clustering algorithms, respectively obtaining clustering seeds from the instruction data set to obtain a clustering seed set;
[0008] Clustering the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters;
[0009] Target instruction data are respectively selected from the multiple instruction data clusters, wherein the target instruction data are used as training data for the dialogue model in the instruction fine-tuning stage.
[0010] According to another aspect of the present disclosure, a method for training a dialogue model is provided, comprising:
[0011] Acquire training data according to the training data processing method described in any embodiment of the present disclosure;
[0012] Using the training data, fine-tuning the pre-trained dialogue model to obtain a dialogue model after fine-tuning the instructions;
[0013] Reinforcement learning alignment is performed on the dialogue model after fine-tuning the instructions to obtain a trained dialogue model.
[0014] According to another aspect of the present disclosure, a text data processing method is provided, comprising:
[0015] Inputting text data into a trained dialogue model, wherein the dialogue model is trained according to the dialogue model training method described in any embodiment of the present disclosure;
[0016] The dialog model is used to recognize instructions in the text data, and an answer matching the instructions is output.
[0017] According to another aspect of the present disclosure, there is provided a training data processing apparatus, comprising:
[0018] An instruction data set acquisition module is used to acquire multiple instruction data sets from the original text data;
[0019] A clustering seed set acquisition module, used to acquire clustering seeds from the instruction data set based on at least two clustering algorithms to obtain a clustering seed set;
[0020] A clustering module, used for clustering the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters;
[0021] The target instruction data selection module is used to select target instruction data from the multiple instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0022] According to another aspect of the present disclosure, a training device for a dialogue model is provided, comprising:
[0023] A training data acquisition module, used to acquire training data using the training data processing device described in any embodiment of the present disclosure;
[0024] An instruction fine-tuning module, used to use the training data to perform instruction fine-tuning on the pre-trained dialogue model to obtain a dialogue model after instruction fine-tuning;
[0025] The reinforcement learning alignment module is used to perform reinforcement learning alignment on the dialogue model after fine-tuning the instructions to obtain a trained dialogue model.
[0026] According to another aspect of the present disclosure, there is provided a text data processing device, comprising:
[0027] An input module, used to input text data into a trained dialogue model, wherein the dialogue model is trained using the dialogue model training device as described in any embodiment of the present disclosure;
[0028] An output module is used to use the dialogue model to identify instructions in the text data and output answers that match the instructions.
[0029] According to another aspect of the present disclosure, there is provided an electronic device, comprising:
[0030] at least one processor; and
[0031] a memory communicatively connected to the at least one processor; wherein,
[0032] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training data processing method, the dialogue model training method, and the text data processing method described in any embodiment of the present disclosure.
[0033] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the training data processing method, the dialogue model training method, and the text data processing method described in any embodiment of the present disclosure.
[0034] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0036] Figure 1 is a flowchart of a training data processing method according to an embodiment of the present disclosure;
[0037] Figure 2a is a flowchart of a training data processing method according to an embodiment of the present disclosure;
[0038] Figure 2b is another schematic diagram of obtaining an instruction data set in the training data processing method of an embodiment of the present disclosure;
[0039] Figure 3a is a flowchart of a training data processing method according to an embodiment of the present disclosure;
[0040] Figure 3bis another schematic diagram of obtaining a clustering seed set in the training data processing method of an embodiment of the present disclosure;
[0041] Figure 4 is a flowchart of a training data processing method according to an embodiment of the present disclosure;
[0042] Figure 5a is a flowchart of a training data processing method according to an embodiment of the present disclosure;
[0043] Figure 5b is a schematic diagram of low-quality data filtering and semantic aggregation in the training data processing method of an embodiment of the present disclosure;
[0044] Figure 6a is a flowchart of a training data processing method according to an embodiment of the present disclosure;
[0045] Figure 6b is another schematic diagram of selecting target instruction data in the training data processing method of an embodiment of the present disclosure;
[0046] Figure 7 is another flow chart of a training data processing method according to an embodiment of the present disclosure;
[0047] Figure 8 is a schematic diagram of a training data processing device according to an embodiment of the present disclosure;
[0048] Fig. 9 It is a block diagram of an electronic device used to implement the training data processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION
[0049] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0050] Figure 1 This is a flowchart of a training data processing method according to an embodiment of the present disclosure. This embodiment can be applied to the case where the training data required for the supervised instruction fine-tuning stage of the dialogue model is processed to obtain diverse high-quality training data. It relates to the field of artificial intelligence technology, and in particular to dialogue models, pre-training and instruction fine-tuning technology. The method can be executed by a training data processing device, which is implemented in software and / or hardware, and is preferably configured in an electronic device, such as a computer device, a terminal or a server. Figure 1 As shown, the method specifically includes the following:
[0051] S101. Acquire multiple instruction data sets from original text data.
[0052] S102: Based on at least two clustering algorithms, respectively obtain clustering seeds from the instruction data set to obtain a clustering seed set.
[0053] S103 . Clustering the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters.
[0054] S104. Select target instruction data from the multiple instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0055] The supervised instruction fine-tuning stage plays a connecting role in the training process of the dialogue model, and is the key to transforming the basic model into an AI assistant. The effectiveness of a model is closely related to its ability to understand instructions. At present, instruction data mainly includes two categories. One is NLP task-related instructions, that is, instructions related to traditional tasks, such as asking the model to translate data, mine entities, etc.; the other is dialogue instructions, which are expected to let the model learn to understand many commands that are closer to daily life. It is worth mentioning that the main function of the instruction fine-tuning stage is to let the model learn to understand instructions, rather than expecting the model to learn different task-solving capabilities (such as learning more knowledge), which requires that the instruction data has strong diversity and wide coverage. Through the technical solution disclosed in the present invention, instruction data with strong diversity and wide coverage can be obtained.
[0056] First, multiple instruction data sets are obtained from the original text data. The original text data may be open source data that can be used for model training, such as open source NLP task data or dialogue data. However, most of the original text data do not contain clear instructions, so it is necessary to first obtain instruction data from the original text data. In addition to data in text form, in some embodiments, the original data may also exist in the form of voice. At this time, the original voice data may be first converted into original text data, and then multiple instruction data sets may be obtained therefrom. Among them, the specific method for obtaining instruction data may adopt any natural language processing method in the prior art to extract instruction data that meets the requirements from a large amount of original text data. In addition, the number of instruction data is huge and can be divided into multiple instruction data sets. The present disclosure does not limit the specific division method. In one embodiment, the capability attributes of the instruction data set may be divided according to the type of the original text data, that is, in multiple instruction data sets, different instruction data sets have different capability attributes. For example, if the original text data belongs to translation data, then the capability attribute of the corresponding instruction data set is translation, and if the original text data belongs to logical reasoning data, then the capability attribute of the corresponding instruction data set is logical reasoning.
[0057] Then, based on at least two clustering algorithms, clustering seeds are obtained from the instruction data set respectively to obtain a clustering seed set, and then the instruction data in the instruction data set are clustered according to the clustering seed set to obtain multiple instruction data clusters. Finally, target instruction data are selected from multiple instruction data clusters to be used as training data for the dialogue model in the instruction fine-tuning stage. That is to say, after obtaining multiple instruction data sets, clustering is performed for each instruction data set, and then target instruction data are extracted from each obtained instruction data cluster. In this way, the target instruction data can be evenly extracted from different instruction data clusters. Among them, the accuracy of the clustering results directly affects the diversity of the target instruction data. Therefore, it is very important to cluster the instruction data in the instruction data set.
[0058] Specifically, since the plaintext features between the instruction data in the same instruction data set are relatively obvious, that is, the literal features of the text are relatively obvious, for example, in translation tasks, the instruction data can be "Please help me translate the following content", or "Can you help me translate this paragraph?", etc., these instruction data usually use the plaintext expression of translating a certain content. Therefore, plaintext clustering based on text literal information can obtain purer and more accurate cluster data.
[0059] However, the core factor of the plaintext clustering effect is the selection of clustering seeds when starting clustering. In order to obtain more representative clustering seeds, the embodiment of the present disclosure adopts a multi-model integration method to obtain clustering seeds through semantic pre-clustering. That is, based on at least two clustering algorithms, clustering seeds are respectively obtained from the instruction data set to obtain a clustering seed set. During implementation, any clustering algorithm can be selected from the existing clustering algorithms, and clustering seeds can be obtained from the instruction data set based on the clustering algorithm. That is, the instruction data set is semantically pre-clustered based on the clustering algorithm, and clustering seeds are obtained according to the clustering results. The number of clustering seeds obtained is related to the results of semantic pre-clustering. For example, multiple clusters are obtained through semantic pre-clustering, and the center of each cluster can be used as a clustering seed. After obtaining at least one clustering seed based on a clustering algorithm, these clustering seeds can be put into a seed pool. Then, another different clustering algorithm is selected to perform semantic pre-clustering on the instruction data set, and at least one clustering seed is obtained therefrom. If these clustering seeds are different from the clustering seeds in the current seed pool, all of them are put into the seed pool. If there are identical clustering seeds, only the other different clustering seeds are put into the seed pool. In this way, clustering seeds are obtained based on at least two clustering algorithms, and the clustering seeds in the seed pool finally obtained are used as the final clustering seed set. After that, the instruction data in the instruction data set can be clustered according to these clustering seeds to obtain multiple instruction data clusters.
[0060] The technical solution of the embodiment of the present disclosure first obtains multiple instruction data sets from the original text data, and then for each instruction data set, a multi-model integration method is adopted to obtain clustering seeds through semantic pre-clustering, and the clustering seeds obtained based on each clustering algorithm are composed into a clustering seed set, and the instruction data in the instruction data set are clustered according to the clustering seeds in the clustering seed set to obtain multiple instruction data clusters, and then target instruction data are respectively selected from the multiple instruction data clusters to obtain training data for use as a dialogue model in the instruction fine-tuning stage.
[0061] For example, the instructions for instructing translation included in the instruction dataset belonging to translation tasks usually have various forms of expression, such as "Please help me translate the following content into English", "Please help me translate <Hello, who are you?> into English" or "Give you a paragraph, can you help me translate the current Chinese into English?". The semantics of instruction data in different forms of expression are all to translate a certain content. If instruction data in various forms of expression are used as training data, and these training data are used to train the dialogue model in the instruction fine-tuning stage, then the dialogue model can recognize instructions of different forms. Otherwise, the model will overfit a certain type of instruction and fail to recognize other types of instructions. For example, if most of the instruction data for translation tasks used for training are similar to the expression form of "Please help me translate the following content into English", then if "Please help me translate the following content into English" is input into the trained dialogue model, the dialogue model will have a good recognition effect on it because it conforms to the distribution of training data. However, if "Give you a paragraph, can you help me translate the current Chinese into English? And the English should be as short as possible." is input into the dialogue model, since it is rarely distributed in the training data, the dialogue model may not understand this sentence and then cannot output a suitable answer.
[0062] The technical solution of the embodiment of the present disclosure can obtain more representative clustering seeds in each instruction data set, so that the effect of clustering based on these clustering seeds can be improved, and multiple instruction data clusters with diversity and purity can be obtained. For example, each data representing the translation instruction in different expressions has its corresponding instruction data cluster. Therefore, a certain number of target instruction data are selected from each instruction data cluster as training data, and the training data includes target instruction data with multiple expressions under the same task, and the number of target instruction data of each expression form is equal as training data. In this way, the diversity of training data can be improved, so that the instruction fine-tuning stage can be based on diversified and wider-ranging instruction data for training, thereby improving the training effect of the dialogue model in the instruction fine-tuning stage. When instructions with different expressions under the same task are input into the trained dialogue model, the dialogue model can recognize and understand them, and output answers that match the input instructions, so that the dialogue model has stronger generalization.
[0063] Figure 2a FIG. 1 is a flow chart of a training data processing method according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes how to obtain multiple instruction data sets. Figure 2a As shown, the method specifically includes the following:
[0064] S201. Inputting original text data and pre-configured AI prompt words into a pre-trained dialogue model, and outputting instruction data of the original text data through the dialogue model.
[0065] Among them, the AI prompt words include examples of instruction data and instruction extraction rules. The examples of instruction data, instruction extraction rules and original text data are input into the pre-trained dialogue model together, and the dialogue model is used to output the instruction data of the original text data. There can be multiple examples of instruction data, and the specific content of the examples can be configured as needed. The present disclosure does not impose any restrictions on this. The instruction extraction rules can be, for example, "extract instructions from the following inputs based on the reference examples, do not extract real questions and do not answer questions." As a result, the dialogue model will extract instruction data that complies with the rules and is similar to the examples from the original text data based on the rules and examples.
[0066] S202: Determine the capability attribute to which the instruction data belongs according to the category information to which the original text data belongs.
[0067] S203. Acquire multiple instruction data sets with different capability attributes from the instruction data according to the capability attributes.
[0068] After extracting the instruction data, in order to prevent the mixing of instructions between multiple tasks, the instruction data can be classified according to the category information to which the original text data belongs, and the category information can be used as the capability attribute to which the instruction data belongs. Based on the capability attribute, multiple instruction data sets with different capability attributes can be obtained from the instruction data, such as translation instruction data sets or logical reasoning instruction data sets.
[0069] Figure 2b It is another schematic diagram of obtaining an instruction data set in the training data processing method of the embodiment of the present disclosure. As shown in the figure, the pre-trained dialogue model is used to extract instruction data from the original text data, and the instruction data is divided according to the ability according to the category of the original text data, and finally multiple instruction data sets are obtained, namely, ability 1, ability 2...ability n, where n is a natural number. Among them, the specific prompt refers to the AI prompt word, and it is expected that the dialogue model will imitate the given example for extraction, and the instruction extraction rule does not output additional information except for the output instruction.
[0070] In addition, in one embodiment, in the case where the extraction based on the dialogue model fails, the instruction data can be extracted from the original text data based on preset rules and using natural language processing methods to ensure the comprehensiveness of the instruction data extraction.
[0071] S204: Based on at least two clustering algorithms, respectively obtain clustering seeds from the instruction data set to obtain a clustering seed set.
[0072] S205 . Cluster the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters.
[0073] S206. Select target instruction data from the multiple instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0074] In addition, in one embodiment, in the case where the extraction based on the dialogue model fails, the instruction data can be extracted from the original text data based on preset rules and using natural language processing methods to ensure the comprehensiveness of the instruction data extraction.
[0075] The technical solution of the disclosed embodiment can automatically extract instruction data from the original text data using the dialogue model, and divide the instruction data according to the category to which the original text data belongs, so as to obtain instruction data sets with different capability attributes. On the one hand, the instruction data can be extracted quickly and comprehensively, and on the other hand, each instruction data set can be used as an object to perform subsequent clustering operations, which is equivalent to first dividing a large amount of instruction data according to the capability attributes, and then clustering each divided instruction data set separately, thereby improving the clustering effect. For example, the instruction data belonging to the translation task is divided into the corresponding instruction data set, and similarly, the instruction data of the logical reasoning class or the mining entity class is also divided into the corresponding instruction data set. In this way, when further clustering is performed later, it is performed separately in the instruction data sets corresponding to different types of tasks, and the instruction data clusters obtained are instruction data of different forms under the same type of task. Therefore, on the one hand, the accuracy of clustering can be improved, and on the other hand, since multiple instruction data of different forms under the same type of task can be obtained, the diversity of instruction data can be improved, and the overfitting of the dialogue model obtained by training can be avoided.
[0076] Figure 3a FIG. 1 is a flow chart of a training data processing method according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes how to obtain a clustering seed set. Figure 3a As shown, the method specifically includes the following:
[0077] S301. Acquire multiple instruction data sets from original text data.
[0078] S302 , vectorize the instruction data in the instruction data set to obtain an instruction vector data set.
[0079] S303: Select at least two clustering algorithms from the preset algorithm set as target clustering algorithms.
[0080] S304. Based on each target clustering algorithm, obtain at least one target clustering seed from the instruction vector data set.
[0081] S305: Update the cluster seed set according to at least one target cluster seed.
[0082] S306 . Cluster the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters.
[0083] S307. Select target instruction data from the multiple instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0084] First, the instruction data in the instruction data set is vectorized to obtain an instruction vector data set. Data vectorization refers to converting text data into vector data for subsequent semantic clustering. For example, for the instruction data "Please help me translate the following content into English", this sentence of text is converted into a vector to represent it. The preset algorithm set may include, for example, a plurality of different clustering algorithms such as Kmeans (K-means) algorithm, spectral clustering, and density clustering. During implementation, one algorithm is selected as the target clustering algorithm each time, and at least one target clustering seed is obtained from the instruction vector data set based on each target clustering algorithm. In this way, the above operation is implemented based on at least two clustering algorithms, and iterations are continuously performed, and the clustering seed set is updated according to the target clustering seeds obtained in each iteration, wherein repeated clustering seeds may not be put into the clustering seed set. The iteration can be terminated until the final clustering seeds tend to be stable. The clustering seeds tend to be stable means that the newly obtained clustering seeds are repeated with the clustering seeds in the clustering seed set, that is, no new clustering seeds can be obtained. Of course, other iteration end conditions may be set as needed, and the present disclosure does not impose any limitation on this.
[0085] Figure 3b This is another schematic diagram of obtaining a cluster seed set in the training data processing method of the embodiment of the present disclosure. As shown in the figure, for each instruction data set, Bert (a natural language processing model) vectorization is first performed, and then based on multiple clustering algorithms in the clustering ensemble, one is selected each time to obtain the target cluster seed from the instruction vector data set, and this cycle is repeated until convergence, that is, when the cluster seeds in the seed set are no longer updated, the cycle can be stopped, and these seeds 1, seed 2...seed m (natural number) form the final cluster seed set.
[0086] The process of obtaining at least one target clustering seed from the instruction vector data set based on the target clustering algorithm is to perform a clustering process once and obtain multiple clusters, and then obtain a clustering seed from each cluster. For example, the seed can be obtained according to the cluster center of the cluster.
[0087] In one embodiment, based on each target clustering algorithm, obtaining at least one target clustering seed from the instruction vector data set may include:
[0088] Based on each target clustering algorithm, semantic clustering is performed on the instruction vector data set to obtain at least one instruction vector data cluster;
[0089] Performing weighted summation on the instruction vectors in the instruction vector data cluster to obtain a cluster center vector of the instruction vector data cluster;
[0090] According to the vector distance from the cluster center vector, target cluster seeds are selected from the instruction vector data clusters.
[0091] In this embodiment, semantic clustering is first performed on the instruction vector data set based on the target clustering algorithm to obtain at least one instruction vector data cluster. For example, semantic clustering is performed in the instruction data vector data set of the translation task to obtain multiple instruction vector data clusters belonging to the same translation task. Then, the instruction vectors in the instruction vector data cluster are weighted and summed to obtain the cluster center vector of the instruction vector data cluster. The present disclosure does not make any restrictions on the selection of instruction vector weights here, which can be configured according to actual needs, or directly used as the cluster center vector by finding the mean of the instruction vector. Afterwards, the vector distance between each instruction vector data in the instruction vector data cluster and the cluster center vector is calculated, and the instruction vector data with the closest vector distance is selected as the target cluster seed, that is, the instruction vector data with the highest similarity to the cluster center vector is selected as the target cluster seed. In this way, target cluster seeds equal to the number of instruction vector data clusters can be obtained in each instruction vector data set. These target cluster seeds constitute a cluster seed set.
[0092] The technical solution of the disclosed embodiment adopts a multi-model integration approach to obtain clustering seeds through semantic pre-clustering, which can prevent a clustering algorithm from overfitting the data, so that more representative clustering seeds can be obtained in each instruction data set, thereby improving the effect of clustering based on these clustering seeds and obtaining more diverse and pure cluster data.
[0093] Figure 4 FIG. 1 is a flow chart of a training data processing method according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes how to obtain multiple instruction data clusters. Figure 4 As shown, the method specifically includes the following:
[0094] S401. Acquire multiple instruction data sets from original text data.
[0095] S402: Based on at least two clustering algorithms, respectively obtain clustering seeds from the instruction data set to obtain a clustering seed set.
[0096] S403: Calculate the edit distance between each instruction data in the instruction data set and each cluster seed in the cluster seed set.
[0097] S404 , clustering the instruction data in the instruction data set according to the edit distance to obtain a plurality of instruction data clusters.
[0098] S405. Select target instruction data from the multiple instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0099] The technical solution of the embodiment of the present disclosure, after obtaining the cluster seed set, performs clustering by calculating the edit distance. Specifically, the edit distance between each instruction data in the instruction data set and each cluster seed in the cluster seed set is calculated, that is, the edit distance is calculated for each instruction data and cluster seed, and the instruction data in the instruction data set is clustered according to the edit distance. For example, the instruction data whose edit distance with each cluster seed is within the set threshold range is divided into an instruction data cluster, so that multiple instruction data clusters are obtained according to multiple cluster seeds. Thus, for the instruction data set under each capability attribute, through this plaintext clustering based on text literal information, those instruction data with similar plaintexts are placed in an instruction data cluster, so as to improve the clustering effect of the instruction data and obtain purer cluster data. In this way, when selecting target instruction data from these instruction data clusters, more diverse target instruction data can be selected, and these target instruction data can be used to train the dialogue model in the instruction fine-tuning stage, so that the dialogue model can learn diverse instructions, so that the trained dialogue model can understand instructions of different expressions under the same task category, and improve the generalization of the model.
[0100] Figure 5a FIG. 1 is a flow chart of a training data processing method according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes how to remove redundancy from instruction data. Figure 5a As shown, the method specifically includes the following:
[0101] S501. Acquire multiple instruction data sets from original text data.
[0102] S502: Based on at least two clustering algorithms, respectively obtain clustering seeds from the instruction data set to obtain a clustering seed set.
[0103] S503 . Cluster the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters.
[0104] S504 : Calculate a first similarity between the instruction data in the instruction data cluster and the cluster seed of the instruction data cluster.
[0105] S505: Delete the instruction data whose first similarity meets the first preset condition from the instruction data cluster.
[0106] S506. Select target instruction data from the multiple instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0107] In order to further improve the purity of the data in the cluster, before selecting the target instruction data, the embodiment of the present disclosure removes redundancy from the data in the instruction data cluster. Specifically, the first similarity between the instruction data in the instruction data cluster and the cluster seed of the instruction data cluster is calculated, and the instruction data whose first similarity meets the first preset condition is deleted from the instruction data cluster. For example, the query similarity model is used to calculate and score the first similarity between the instruction data in the cluster and the cluster seed of the cluster. If it is determined that the first similarity meets the first preset condition according to the scoring situation, for example, the first similarity is lower than a certain threshold, it indicates that the current instruction data is not similar to the cluster seed, belongs to the noise data in the cluster, and should be deleted from the cluster. For example, in the instruction data cluster corresponding to the translation task, if the instruction data is "Please help me send the translation to the conference room", the semantics of this sentence is not translation. If it appears here, it means that it is noise data and needs to be deleted from the cluster. On the contrary, if the instruction data is not deleted, and the rest of the instruction data in the cluster are used as training data to train the dialogue model, it will affect the accuracy of the trained dialogue model's understanding of such instructions.
[0108] In addition, in another embodiment, before selecting the target instruction data, the central vector of each instruction data cluster can be calculated, and then the second similarity of the central vectors of any two instruction data clusters among the multiple instruction data clusters can be calculated, and the two instruction data clusters whose second similarity meets the second preset condition can be merged. That is, among the many instruction data clusters, there may be clusters with high similarity. Therefore, by calculating the second similarity of the central vectors between the two data clusters, two instruction data clusters whose second similarity is higher than a certain threshold can be merged, thereby removing redundant instruction data clusters and improving the overall diversity of instruction data clusters.
[0109] The technical solution of the disclosed embodiment can eliminate a large number of redundant instructions and ensure that the instruction data cluster is diverse, so that different instructions are distributed more evenly and the weights of different instructions learned in the dialogue model are balanced. Then the trained dialogue model is unlikely to be biased towards a certain type of instruction. At this time, the generalization is strongest and there will be no overfitting problem.
[0110] Figure 5bIt is a schematic diagram of low-quality data filtering and semantic aggregation in the training data processing method of the embodiment of the present disclosure. As shown in the figure, first, for instruction data clusters 1, 2...j (natural numbers), the query similarity model is used to respectively calculate the first similarity between the instruction data therein and the clustering seed of the instruction data cluster, and the instruction data whose first similarity meets the first preset condition is deleted from the instruction data cluster, and the noise data is filtered with a low score to obtain the filtered instruction data clusters 1', 2'...j'. Then, the center vectors of the clusters are calculated in the instruction data clusters 1', 2'...j' respectively, and the second similarity of the center vectors of any two instruction data clusters in the multiple instruction data clusters is calculated. The two instruction data clusters whose second similarity meets the second preset condition are merged. For example, instruction data cluster 1' and instruction data cluster 2" meet the above-mentioned merging conditions, and instruction data cluster 1" is obtained through semantic aggregation, and finally instruction data cluster 1"...instruction data cluster i is obtained.
[0111] Figure 6a FIG. 1 is a flow chart of a training data processing method according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes how to select target instruction data. Figure 6a As shown, the method specifically includes the following:
[0112] S601. Acquire multiple instruction data sets from original text data.
[0113] S602: Based on at least two clustering algorithms, respectively obtain clustering seeds from the instruction data set to obtain a clustering seed set.
[0114] S603 . Cluster the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters.
[0115] S604 , performing semantic clustering on the instruction data in each instruction data cluster respectively to obtain multiple sub-clusters of each instruction data cluster.
[0116] S605. For each instruction data cluster, select a preset number of target instruction data from the corresponding sub-clusters, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0117] After obtaining high-quality instruction data clusters of instruction data sets under different capabilities, N pieces of data are selected from each instruction data cluster as training data for the instruction fine-tuning stage to ensure the capability balance between different instructions to the greatest extent possible. In the embodiment of the present disclosure, in order to ensure the diversity of data within similar instructions, random screening is not performed when selecting data, but pre-clustering is performed first. That is to say, the instruction data is semantically clustered in each instruction data cluster to obtain multiple sub-clusters, and then a preset number of target instruction data are selected from each sub-cluster, and the target instruction data selected from all sub-clusters constitute the training data in the instruction data set. Among them, the semantic clustering method can be implemented using any algorithm in the prior art, and the present disclosure does not impose any limitation on this. For example, Figure 6b As shown, the instruction data in the instruction data cluster is first Bert vectorized, and then the vectorized data is clustered using the Kmeans algorithm to obtain multiple sub-clusters 1, 2...k (natural number), and then the training data is evenly selected from each sub-cluster.
[0118] The technical solution of the disclosed embodiment does not select the target instruction data randomly, but first clusters the instruction data cluster to obtain multiple sub-clusters, and then evenly selects from each sub-cluster. In this way, the diversity of the extracted instructions can be further guaranteed, thereby improving the training effect of the dialogue model in the instruction fine-tuning stage.
[0119] Figure 7 It is another flow chart of the training data processing method according to the embodiment of the present disclosure. As shown in the figure, in the first step Step1, after extracting the instruction data from the original text data, the instruction data is segmented based on the category of the original text data to obtain multiple instruction data sets with different ability attributes. The second step Step2 is to perform seed screening. In the embodiment of the present disclosure, clustering seeds are iteratively selected in the form of an integrated algorithm. Each iteration selects a different clustering algorithm from the algorithm set to obtain clustering seeds from the instruction data set, and the duplicate seeds in all the obtained clustering seeds are deleted. After deletion, a clustering seed set is formed, including seeds 1, 2...n' (natural number). The third step Step3 is plaintext clustering. According to all the clustering seeds in the clustering seed set, the instruction data in the instruction data set is clustered based on the text literal to obtain instruction data clusters 1, 2...m' (natural number). In the fourth step, low-quality filtering and semantic aggregation are performed on the instruction data clusters, that is, the noise data in each instruction data cluster is removed, and then two or more instruction data clusters with high similarity are merged to obtain the final instruction data cluster 1”'…t (natural number). By uniformly selecting target instruction data from each cluster, the required training data can be obtained.
[0120] As can be seen from the above, the technical solution of the embodiment of the present disclosure first performs capability segmentation on the instruction data and performs initial division on the instruction data. Subsequent operations are all for each instruction data set obtained by the division, which effectively solves the problem that similar tasks under different data sets cannot be distinguished, and analyzes and cleans the overall data from the perspective of capability. Next, a multi-stage clustering method is used to remove redundant data and obtain target instruction data with strong diversity. The first stage is seed screening, which adopts a multi-model integration method and uses multiple clustering algorithms to obtain more representative clustering seeds. The second stage is plaintext clustering, which has a better clustering effect for instruction data with obvious plaintext features. The third stage is low-quality filtering and semantic aggregation, which deletes redundant instructions in the cluster and merges clusters with higher similarity. The above, based on multi-stage clustering, not only a lot of manpower costs are saved, but also the clustering effect is improved through plaintext clustering on the basis of obtaining more representative clustering seeds. Finally, the instruction clusters are further integrated through low-quality filtering and semantic aggregation to improve the diversity of instruction clusters. Therefore, the diversity of target instruction data evenly selected from each instruction cluster can also meet the training requirements, allowing the model to evenly learn instructions of different categories, improving the training effect of the model in the instruction fine-tuning stage, and making the generalization performance of the trained dialogue model stronger.
[0121] The disclosed embodiment also provides a method for training a dialogue model, which can be applied to training a dialogue model. The trained dialogue model can be used to output corresponding answers according to input instructions.
[0122] The method includes:
[0123] Acquire training data according to the training data processing method described in any embodiment of the present disclosure;
[0124] Using the training data, fine-tuning the pre-trained dialogue model to obtain a dialogue model after fine-tuning the instructions;
[0125] Perform reinforcement learning alignment on the dialogue model after fine-tuning the instructions to obtain a trained dialogue model.
[0126] Specifically, the pre-trained dialogue model has fully learned world knowledge. In the instruction fine-tuning stage, according to the training data processing method described in any embodiment of the present disclosure, a variety of instruction training data can be obtained, and the number of instruction data in each expression form is equal. Using these training data to train the dialogue model makes the dialogue model more generalizable, avoiding overfitting of a certain type of instruction and affecting the instruction fine-tuning effect of the dialogue model. Afterwards, the dialogue model after instruction fine-tuning is further aligned by reinforcement learning to obtain a trained dialogue model, which can give answers corresponding to the input instructions and align with human preferences, and its answers are more comprehensive and accurate.
[0127] In addition, the embodiment of the present disclosure also provides a text data processing method, which can be used to process input text instructions and output corresponding answers. The method includes:
[0128] Inputting text data into a trained dialogue model, wherein the dialogue model is trained according to the dialogue model training method described in any embodiment of the present disclosure;
[0129] Use the dialogue model to identify instructions in text data and output answers that match the instructions.
[0130] Specifically, in the supervised instruction fine-tuning stage, since the pre-trained dialogue model is trained by the dialogue model training method described in any embodiment of the present disclosure, and the dialogue model training method described in any embodiment of the present disclosure also uses the training data processing method described in any embodiment of the present disclosure to obtain training data, the training data has stronger diversity, and the number of different instruction data is balanced, which avoids overfitting of the dialogue model and improves the generalization performance of the dialogue model. Therefore, the dialogue model can process the input text data more accurately, understand the corresponding instructions, and give answers that match the instructions based on the learned knowledge. For example, when the input text data is "Please help me translate <Hello, who are you?> into English", the dialogue model can output "Hello, Who are you?".
[0131] Figure 8 Schematic diagram of a training data processing device according to an embodiment of the present disclosure. This embodiment can be applied to the case where the training data required for the supervised instruction fine-tuning stage of the dialogue model is processed to obtain diverse high-quality training data. It relates to the field of artificial intelligence technology, and in particular to dialogue models, pre-training and instruction fine-tuning technology. The device can implement the training data processing method described in any embodiment of the present disclosure. Figure 8 As shown, the device 800 specifically includes:
[0132] The instruction data set acquisition module 801 is used to acquire multiple instruction data sets from the original text data;
[0133] A clustering seed set acquisition module 802 is used to acquire clustering seeds from the instruction data set based on at least two clustering algorithms to obtain a clustering seed set;
[0134] A clustering module 803, configured to cluster the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters;
[0135] The target instruction data selection module 804 is used to select target instruction data from the multiple instruction data clusters, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage.
[0136] Optionally, among the multiple instruction data sets, different instruction data sets have different capability attributes.
[0137] Optionally, the instruction data set acquisition module 801 includes:
[0138] An instruction data output unit, configured to input the original text data and pre-configured AI prompt words into the pre-trained dialogue model, and output instruction data of the original text data through the dialogue model, wherein the AI prompt words include examples of instruction data and instruction extraction rules;
[0139] A capability attribute determination unit, used to determine the capability attribute to which the instruction data belongs according to the category information to which the original text data belongs;
[0140] The instruction data set acquisition unit is used to acquire a plurality of instruction data sets with different capability attributes from the instruction data according to the capability attribute.
[0141] Optionally, the cluster seed set acquisition module 802 includes:
[0142] A vectorization submodule, used for vectorizing the instruction data in the instruction data set to obtain an instruction vector data set;
[0143] An algorithm selection submodule, used to select at least two clustering algorithms from a preset algorithm set as target clustering algorithms;
[0144] A cluster seed acquisition submodule, configured to acquire at least one target cluster seed from the instruction vector data set based on each target clustering algorithm;
[0145] The cluster seed set updating submodule is used to update the cluster seed set according to the at least one target cluster seed.
[0146] Optionally, the clustering seed acquisition submodule includes:
[0147] A semantic clustering unit, configured to perform semantic clustering on the instruction vector data set based on each target clustering algorithm to obtain at least one instruction vector data cluster;
[0148] A cluster center vector determination unit, used for performing weighted summation of instruction vectors in the instruction vector data cluster to obtain a cluster center vector of the instruction vector data cluster;
[0149] The cluster seed selection unit is used to select target cluster seeds from the instruction vector data cluster according to the vector distance from the cluster center vector.
[0150] Optionally, the clustering module 803 includes:
[0151] An edit distance calculation unit, used for calculating the edit distance between each instruction data in the instruction data set and each cluster seed in the cluster seed set;
[0152] The instruction data cluster determining unit is used to cluster the instruction data in the instruction data set according to the edit distance to obtain a plurality of instruction data clusters.
[0153] Optionally, the device further includes a first redundancy removal module, the first redundancy removal module including:
[0154] A first similarity calculation unit, configured to calculate a first similarity between the instruction data in the instruction data cluster and a cluster seed of the instruction data cluster before the target instruction data selection module selects target instruction data from the plurality of instruction data clusters respectively;
[0155] The deleting unit is used to delete the instruction data whose first similarity meets the first preset condition from the instruction data cluster.
[0156] Optionally, the device further includes a second redundancy removal module, wherein the second redundancy removal module includes:
[0157] A central vector calculation unit, used for calculating the central vector of each of the instruction data clusters before the target instruction data selection module selects the target instruction data from the plurality of instruction data clusters respectively;
[0158] A second similarity calculation unit, used for calculating a second similarity of the center vectors of any two instruction data clusters among the plurality of instruction data clusters;
[0159] The merging unit is used to merge two instruction data clusters whose second similarity meets the second preset condition.
[0160] Optionally, the target instruction data selection module 804 includes:
[0161] A sub-cluster determination unit, used to perform semantic clustering on the instruction data in each of the instruction data clusters to obtain a plurality of sub-clusters of each of the instruction data clusters;
[0162] The target instruction data selection unit is used to select a preset number of target instruction data from the corresponding sub-clusters for each instruction data cluster.
[0163] In addition, the present disclosure also provides a training device for a dialogue model, including:
[0164] A training data acquisition module, used to acquire training data using the training data processing device described in any embodiment of the present disclosure;
[0165] An instruction fine-tuning module, used to use the training data to perform instruction fine-tuning on the pre-trained dialogue model to obtain a dialogue model after instruction fine-tuning;
[0166] The reinforcement learning alignment module is used to perform reinforcement learning alignment on the dialogue model after fine-tuning the instructions to obtain a trained dialogue model.
[0167] Furthermore, the present disclosure also provides a text data processing device, including:
[0168] An input module, used to input text data into a trained dialogue model, wherein the dialogue model is trained using the dialogue model training device as described in any embodiment of the present disclosure;
[0169] An output module is used to use the dialogue model to identify instructions in the text data and output answers that match the instructions.
[0170] The above-mentioned product can execute the method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
[0171] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0172] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0173] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0174] like Fig. 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0175] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0176] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as training data processing methods. For example, in some embodiments, the training data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the training data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the training data processing method in any other appropriate manner (e.g., by means of firmware).
[0177] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0178] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0179] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0180] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0181] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0182] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services. The server may also be a server of a distributed system, or a server combined with a blockchain.
[0183] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.
[0184] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.
[0185] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions provided by this disclosure can be achieved, and this document does not limit this.
[0186] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A training data processing method, comprising: Obtain multiple instruction data sets from the original text data; Based on at least two clustering algorithms, respectively obtaining clustering seeds from the instruction data set to obtain a clustering seed set; Clustering the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters; Selecting target instruction data from the plurality of instruction data clusters respectively, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage; Wherein, based on at least two clustering algorithms, clustering seeds are respectively obtained from the instruction data set to obtain a clustering seed set, including: Vectorizing the instruction data in the instruction data set to obtain an instruction vector data set; Select at least two clustering algorithms from a preset algorithm set as target clustering algorithms; Based on each target clustering algorithm, obtaining at least one target clustering seed from the instruction vector data set; updating a cluster seed set according to the at least one target cluster seed; The step of clustering the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters includes: Calculating the edit distance between each instruction data in the instruction data set and each cluster seed in the cluster seed set; The instruction data in the instruction data set are clustered according to the edit distance to obtain a plurality of instruction data clusters.
2. The method according to claim 1, wherein: Among the multiple instruction data sets, different instruction data sets have different capability attributes.
3. The method according to claim 1 or 2, wherein: The step of obtaining multiple instruction data sets from the original text data includes: Inputting the original text data and pre-configured artificial intelligence AI prompt words into the pre-trained dialogue model, and outputting instruction data of the original text data through the dialogue model, wherein the AI prompt words include examples of instruction data and instruction extraction rules; Determining the capability attribute to which the instruction data belongs according to the category information to which the original text data belongs; According to the capability attribute, a plurality of instruction data sets with different capability attributes are obtained from the instruction data.
4. The method according to claim 1, wherein: The acquiring at least one target clustering seed from the instruction vector data set based on each target clustering algorithm includes: Based on each target clustering algorithm, semantic clustering is performed on the instruction vector data set to obtain at least one instruction vector data cluster; Taking a weighted sum of the instruction vectors in the instruction vector data cluster to obtain a cluster center vector of the instruction vector data cluster; Target clustering seeds are selected from the instruction vector data clusters according to the vector distance from the cluster center vector.
5. The method according to claim 1, before selecting target instruction data from the plurality of instruction data clusters, further comprising: Calculating a first similarity between the instruction data in the instruction data cluster and a cluster seed of the instruction data cluster; The instruction data whose first similarity meets a first preset condition is deleted from the instruction data cluster.
6. The method according to claim 5, before selecting target instruction data from the plurality of instruction data clusters, further comprising: Calculating the center vector of each of the instruction data clusters; Calculating a second similarity of center vectors of any two instruction data clusters among the plurality of instruction data clusters; The two instruction data clusters whose second similarity meets the second preset condition are merged.
7. The method according to claim 1, wherein: The selecting target instruction data from the plurality of instruction data clusters respectively comprises: Performing semantic clustering on the instruction data in each of the instruction data clusters respectively to obtain a plurality of subclusters of each of the instruction data clusters; For each of the instruction data clusters, a preset number of target instruction data are selected from the corresponding sub-clusters.
8. A method for training a dialogue model, comprising: Acquire training data according to the training data processing method according to any one of claims 1 to 7; Using the training data, fine-tuning the pre-trained dialogue model to obtain a dialogue model after fine-tuning the instructions; Reinforcement learning alignment is performed on the dialogue model after fine-tuning the instructions to obtain a trained dialogue model.
9. A text data processing method, comprising: Inputting text data into a trained dialogue model, wherein the dialogue model is trained according to the dialogue model training method according to claim 8; The dialog model is used to recognize instructions in the text data, and an answer matching the instructions is output.
10. A training data processing device, comprising: An instruction data set acquisition module is used to acquire multiple instruction data sets from the original text data; A clustering seed set acquisition module, used to acquire clustering seeds from the instruction data set based on at least two clustering algorithms to obtain a clustering seed set; A clustering module, used for clustering the instruction data in the instruction data set according to the clustering seed set to obtain a plurality of instruction data clusters; A target instruction data selection module, used to select target instruction data from the multiple instruction data clusters, wherein the target instruction data is used as training data for the dialogue model in the instruction fine-tuning stage; Wherein, the cluster seed set acquisition module includes: A vectorization submodule, used for vectorizing the instruction data in the instruction data set to obtain an instruction vector data set; An algorithm selection submodule, used to select any at least two clustering algorithms from a preset algorithm set as target clustering algorithms; A cluster seed acquisition submodule, configured to acquire at least one target cluster seed from the instruction vector data set based on each target clustering algorithm; A cluster seed set updating submodule, configured to update the cluster seed set according to the at least one target cluster seed; Wherein, the clustering module includes: An edit distance calculation unit, used for calculating the edit distance between each instruction data in the instruction data set and each cluster seed in the cluster seed set; The instruction data cluster determining unit is used to cluster the instruction data in the instruction data set according to the edit distance to obtain a plurality of instruction data clusters.
11. The device according to claim 10, wherein: Among the multiple instruction data sets, different instruction data sets have different capability attributes.
12. The device according to claim 10 or 11, wherein: The instruction data set acquisition module includes: An instruction data output unit, configured to input the original text data and pre-configured AI prompt words into the pre-trained dialogue model, and output instruction data of the original text data through the dialogue model, wherein the AI prompt words include examples of instruction data and instruction extraction rules; A capability attribute determination unit, used to determine the capability attribute to which the instruction data belongs according to the category information to which the original text data belongs; The instruction data set acquisition unit is used to acquire a plurality of instruction data sets with different capability attributes from the instruction data according to the capability attribute.
13. The device according to claim 10, wherein: The clustering seed acquisition submodule includes: A semantic clustering unit, configured to perform semantic clustering on the instruction vector data set based on each target clustering algorithm to obtain at least one instruction vector data cluster; A cluster center vector determination unit, used for performing weighted summation of instruction vectors in the instruction vector data cluster to obtain a cluster center vector of the instruction vector data cluster; The cluster seed selection unit is used to select target cluster seeds from the instruction vector data cluster according to the vector distance from the cluster center vector.
14. The apparatus according to claim 10, further comprising a first redundancy removal module, wherein the first redundancy removal module comprises: A first similarity calculation unit, configured to calculate a first similarity between the instruction data in the instruction data cluster and a cluster seed of the instruction data cluster before the target instruction data selection module selects target instruction data from the plurality of instruction data clusters respectively; The deleting unit is used to delete the instruction data whose first similarity meets the first preset condition from the instruction data cluster.
15. The apparatus according to claim 14, further comprising a second redundancy removal module, wherein the second redundancy removal module comprises: A central vector calculation unit, used for calculating the central vector of each of the instruction data clusters before the target instruction data selection module selects the target instruction data from the plurality of instruction data clusters respectively; A second similarity calculation unit, used for calculating a second similarity of the center vectors of any two instruction data clusters among the plurality of instruction data clusters; The merging unit is used to merge two instruction data clusters whose second similarity meets the second preset condition.
16. The device according to claim 10, wherein: The target instruction data selection module includes: A sub-cluster determination unit, used to perform semantic clustering on the instruction data in each of the instruction data clusters to obtain a plurality of sub-clusters of each of the instruction data clusters; The target instruction data selection unit is used to select a preset number of target instruction data from the corresponding sub-clusters for each instruction data cluster.
17. A training device for a dialogue model, comprising: A training data acquisition module, used to acquire training data using the training data processing device according to any one of claims 10 to 16; An instruction fine-tuning module, used to use the training data to perform instruction fine-tuning on the pre-trained dialogue model to obtain a dialogue model after instruction fine-tuning; The reinforcement learning alignment module is used to perform reinforcement learning alignment on the dialogue model after fine-tuning the instructions to obtain a trained dialogue model.
18. A text data processing device, comprising: An input module, used for inputting text data into a trained dialogue model, wherein the dialogue model is trained using the dialogue model training device according to claim 17; An output module is used to use the dialogue model to identify instructions in the text data and output answers that match the instructions.
19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training data processing method described in any one of claims 1 to 7, the training method of the dialogue model described in claim 8, and the text data processing method described in claim 9.
20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the training data processing method according to any one of claims 1 to 7, the dialogue model training method according to claim 8, and the text data processing method according to claim 9.
21. A computer program product, comprising a computer program, which, when executed by a processor, implements the training data processing method according to any one of claims 1 to 7, the training method of the dialogue model according to claim 8, and the text data processing method according to claim 9.
