Training data generation method, device, computer equipment, storage medium and product

Abstract

The present application is about a training data generation method, device, computer equipment, storage medium and product, and relates to the field of machine learning technology. The method includes: obtaining any target text segment based on the original document; extracting question-answer pairs from the target text segment through multiple large language models set in different prompting modes to obtain multiple question-answer pairs of the target text segment; generating a reorganized question-answer pair based on at least two target question-answer pairs in the case where the multiple question-answer pairs contain at least two target question-answer pairs with the same question text; the reorganized question-answer pair contains question text and reorganized answer text; the reorganized answer text is generated based on the answer text in at least two target question-answer pairs; generating training data based on the reorganized question-answer pair and non-target question-answer pairs in the multiple question-answer pairs; through the above method, the automation of training data generation can be realized, the efficiency of training data generation can be improved, and the data quality and comprehensiveness of training data can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of machine learning technology, and in particular to a training data generation method, apparatus, computer equipment, storage medium and product. Background Art

[0002] In recent years, the application of large language models (LLMs) such as GPT has gradually increased; the quality and diversity of training data play a vital role in improving the performance of large language models.

[0003] In related technologies, training data generation often relies on manual labeling or rule-based template generation, which usually requires semantic recognition and data labeling of the original data. Currently, there is a lack of effective automated means, and most of them are still semi-automated and manually labeled, which makes the generation of training data inefficient and less accurate. Summary of the invention

[0004] The embodiments of the present application provide a training data generation method, apparatus, computer equipment, storage medium and product, which can realize the automation of training data generation, improve the efficiency of training data generation, and improve the data quality and comprehensiveness of training data. The technical solution is as follows.

[0005] On the one hand, a training data generation method is provided. The method comprises: Obtaining a target text segment; the target text segment is any text segment obtained based on the original document; Extracting question-answer pairs from the target text segment through multiple large language models to obtain multiple question-answer pairs of the target text segment; different prompting modes are set between the large language models; In the case where the multiple question-answer pairs include at least two target question-answer pairs with the same question text, a reorganized question-answer pair is generated based on the at least two target question-answer pairs; the reorganized question-answer pair includes a question text and a reorganized answer text; the reorganized answer text is generated based on the answer texts in the at least two target question-answer pairs; Training data is generated based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

[0006] In another aspect, a training data generating device is provided, the device comprising: A text segment acquisition module is used to acquire a target text segment; the target text segment is any text segment obtained based on the original document; A question-answer pair extraction module, used to extract question-answer pairs from the target text segment through multiple large language models to obtain multiple question-answer pairs of the target text segment; the prompt modes of the various large language models are set differently; A question-answer pair generation module, configured to generate a reorganized question-answer pair based on at least two target question-answer pairs having the same question text, when the multiple question-answer pairs include at least two target question-answer pairs having the same question text; the reorganized question-answer pair includes a question text and a reorganized answer text; the reorganized answer text is generated based on the answer texts in at least two target question-answer pairs; A data generation module is used to generate training data based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

[0007] In a possible implementation, the question-answer pair generating module includes: A list generation submodule, used to generate an answer text list corresponding to the question text based on at least two of the target question-answer pairs; The deduplication submodule is used to deduplicate at least two answer texts in the answer text list corresponding to the question text to obtain the reorganized question-answer pair consisting of the deduplicated answer text and the question text.

[0008] In a possible implementation, the deduplication submodule includes: A similarity calculation unit, used for performing similarity calculation on each answer text in the answer text list to obtain the similarity between each answer text; The deduplication unit is used to perform deduplication processing on at least two question and answer texts based on the similarity between the answer texts to obtain the reorganized question and answer pair consisting of the deduplicated answer text and the question text.

[0009] In a possible implementation, the deduplication unit is used to determine the first N pairs of answer texts with the smallest similarity in at least two question and answer texts as the deduplicated answer texts; N is a positive integer.

[0010] In a possible implementation, the deduplication unit is used to determine M pairs of answer texts whose relative degree is less than a similarity threshold among at least two answer texts as deduplicated answer texts; M is a positive integer.

[0011] In a possible implementation manner, the device further includes: An initialization module, used to initialize the template of each large language model based on the prompt mode corresponding to each large language model; The model extraction module is used to randomly extract the multiple large language models from the initialized large language models.

[0012] In a possible implementation, the text segment acquisition module includes: A text extraction submodule, used to extract text from the original document to obtain the original text corresponding to the original document; A segmentation processing submodule, used for segmenting the original text by using a segmentation algorithm to obtain a text segment list corresponding to the original text; The text segment acquisition submodule is used to acquire any one text segment in the text segment list as the target text segment.

[0013] In a possible implementation, the text extraction submodule is used to: In the case where the original document includes image information, performing image recognition on the image information to obtain text content corresponding to the image information; In the case that the original document includes audio information, speech recognition is performed on the audio information to obtain text content corresponding to the audio information.

[0014] In a possible implementation, the segmentation algorithm includes at least one of the following: Segmenting the original text based on the target delimiter; Segmenting the original text based on text semantics; Segmenting the original text based on the number of words in the text; The original text is segmented based on the specified keywords.

[0015] In a possible implementation, the data generation module is used to convert the format of the reorganized question and answer pairs and the non-target question and answer pairs based on the training data format of the target training model to obtain training data corresponding to the target training model.

[0016] On the other hand, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the above-mentioned training data generation method.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the above-mentioned training data generation method.

[0018] On the other hand, a computer program product is provided, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, which, when executed by a computer, cause the computer to execute to implement the training data generation method provided in the above-mentioned various optional implementations.

[0019] The technical solution provided by this application may have the following beneficial effects: The training data generation method provided in the embodiment of the present application is to extract question-answer pairs for any text paragraph obtained based on the original document by setting multiple large language models with different prompting modes to obtain multiple question-answer pairs of the text paragraph. In the case that there are multiple target question-answer pairs with the same question text, a reorganized question-answer pair is generated based on the multiple target question-answer pairs, and the reorganized question-answer pair includes the question text and the reorganized answer text generated based on the answer text in the multiple target question-answer pairs. After that, training data is generated based on the reorganized question-answer pair and the non-target question-answer pairs in the multiple question-answer pairs except the target question-answer pairs. Through the above method, the automation of training data generation can be realized, and the efficiency of training data generation can be improved. In the process, by extracting the same text paragraph information with the help of different prompting modes of multiple large language models, question-answer pairs can be obtained from multiple perspectives, thereby improving the comprehensiveness of information extraction. By reorganizing the question-answer pairs with the same question text, the comprehensiveness and accuracy of the answer text in the question-answer pairs are improved, so that when generating training data based on the reorganized question-answer pairs and the non-target question-answer pairs, the data quality and comprehensiveness of the training data can be improved, thereby improving the effect of model training.

[0020] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] Figure 1 A flowchart of a training data generation method provided by an exemplary embodiment of the present application is shown; Figure 2 A flowchart of a training data generation method provided by another exemplary embodiment of the present application is shown; Figure 3 A schematic diagram of a processing process of a preprocessing tool provided by an exemplary embodiment of the present application is shown; Figure 4 A schematic diagram showing the organizational relationship of question-answer pairs provided by an exemplary embodiment of the present application is shown; Figure 5A schematic diagram showing a training data generation process provided by an exemplary embodiment of the present application is shown; Figure 6 A block diagram of a training data generating device provided by an exemplary embodiment of the present application is shown; Figure 7 A structural block diagram of a computer device shown in an exemplary embodiment of the present application is shown; Figure 8 A structural block diagram of a computer device shown in an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0024] The present invention provides a method for generating training data, which can realize the automation of training data generation and improve the efficiency and accuracy of training data generation; Figure 1 A flowchart of a training data generation method provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device, and the computer device can be implemented as a server or a terminal, such as Figure 1 As shown, the training data generation method may include the following steps.

[0025] Step 110, obtaining a target text segment; the target text segment is any text segment obtained based on the original document.

[0026] When generating training data, the computer device can generate corresponding training data based on the text segments in the original document. By processing different text segments in different original documents, a large amount of training data can be generated to meet the needs of model training. In the embodiment of the present application, any one of the text segments to be processed (i.e., the target text segment) in the text segments generated based on the original document is taken as an example to illustrate the method for generating training data provided in the embodiment of the present application.

[0027] Among them, the original document can be various forms of document materials, such as news reports, academic papers, novels, technical documents, etc. In addition to text information, the original document may also include image information and audio information, etc. This application does not limit the type of original document.

[0028] Step 120, extracting question-answer pairs from the target text segment through multiple large language models to obtain multiple question-answer pairs of the target text segment; the prompting methods of the various large language models are set differently.

[0029] For the target text segment, multiple different large language models are used to mine the question and answer information contained therein. Since different large language models have their own unique characteristics and advantages, each large language model is set with different prompting methods, which can analyze the target text segment from different angles and depths and generate multiple questions and answers; wherein, each large language model can generate a corresponding specified number of question and answer pairs; for example, in the case of extracting question and answer pairs for the same text segment through two large language models, for one of the large language models, the prompting method may be to require focusing on the key concepts and definitions in the text segment and extracting related question and answer pairs, such as the prompt information is "Please generate question and answer pairs about its definition and characteristics based on [key concept name] in the text segment", the model can generate one or more related question and answer pairs based on the prompt information and the text segment; while the prompting method of the other large language model focuses on the causal relationship mining in the text segment, such as the prompt information is "Analyze the causal logic in the text segment and generate corresponding question and answer pairs", the model can also generate one or more related question and answer pairs based on the prompt information and the text segment.

[0030] Step 130, when multiple question-answer pairs include at least two target question-answer pairs with the same question text, generate a reorganized question-answer pair based on the at least two target question-answer pairs; the reorganized question-answer pair includes the question text and the reorganized answer text; the reorganized answer text is generated based on the answer texts in the at least two target question-answer pairs.

[0031] Among the multiple question-answer pairs generated by multiple large language models, there may be question-answer pairs with the same question text (i.e., target question-answer pairs) and question-answer pairs with different question texts (i.e., non-target question-answer pairs). For the target question-answer pairs, the answer text of the question text needs to be integrated to make the answer text corresponding to the question text more comprehensive and rich.

[0032] When generating a reorganized text pair based on at least two target question-answer pairs, in a possible implementation, answer texts from different target question-answer pairs may be summarized under the same question text, that is, the reorganized answer pair includes a question text and multiple answer texts; or, in another possible implementation, answer texts from different target answer pairs may be integrated into an answer text for the question text, that is, the reorganized answer pair includes a question text and an answer text; schematically, if there are three question-answer pairs among multiple question-answer pairs generated by multiple large language models, and the question texts are all "What are the main advantages of solar energy?", and the answer texts are "solar energy is clean energy with no pollution emissions", "solar energy has a wide range of sources and is almost inexhaustible", and "the operation and maintenance costs of solar energy systems are relatively low", the reorganized question-answer pairs generated based on the above three question-answer pairs may be: Question text: What are the main advantages of solar energy? Answer text 1: "Solar energy is a clean energy with no pollution emissions", Answer text 2: "Solar energy has a wide range of sources and is almost inexhaustible" and Answer text 3: "Solar energy system operation and maintenance costs are relatively low" Alternatively, the reorganized question-answer pair could be: Question text: What are the main advantages of solar energy? Answer text: Solar energy is a clean energy with no pollution emissions; solar energy sources are widespread and almost inexhaustible; the operation and maintenance costs of solar energy systems are relatively low.

[0033] By reorganizing the target question-answer pairs, the quality and information content of the question-answer pairs can be improved, providing more comprehensive materials for the generation of training data.

[0034] Step 140, generating training data based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

[0035] The reorganized question-answer pairs and other non-target question-answer pairs are used as the basis for constructing training data to achieve complete application of the target text segment information; since different training models have different training data format requirements, computer equipment can process the reorganized question-answer pairs and other non-target question-answer pairs based on the training data formats required by different training models to obtain training data that meets the model training requirements.

[0036] In summary, the training data generation method provided in the embodiment of the present application, for any text paragraph obtained based on the original document, extracts question and answer pairs respectively by setting multiple large language models with different prompting methods to obtain multiple question and answer pairs of the text segment, and in the case that there are multiple target question and answer pairs with the same question text, generates a reorganized question and answer pair based on the multiple target question and answer pairs, and the reorganized question and answer pair includes the question text and the reorganized answer text generated based on the answer text in the multiple target question and answer pairs, and then generates training data based on the reorganized question and answer pair and the non-target question and answer pairs in the multiple question and answer pairs except the target question and answer pairs; through the above method, the automation of training data generation can be realized, and the efficiency of training data generation can be improved. In the process, by extracting the same text segment information with the help of different prompting methods of multiple large language models, the question and answer pairs can be obtained from multiple perspectives, thereby improving the comprehensiveness of information extraction; by reorganizing the question and answer pairs with the same question text, the comprehensiveness and accuracy of the answer text in the question and answer pair are improved, so that when generating training data based on the reorganized question and answer pairs and the non-target question and answer pairs, the data quality and comprehensiveness of the training data can be improved, thereby improving the effect of model training.

[0037] When multiple large language models extract question-answer pairs from the same text segment, there may be repeated or highly similar answer texts for the same question text. In order to avoid redundancy of the reorganized answer texts caused by repeated or highly similar answer texts, when reorganizing the question-answer pairs, the answer texts for the same question text can be deduplicated before being integrated. Figure 2 A flowchart of a training data generation method provided by another exemplary embodiment of the present application is shown. The method can be executed by a computer device, and the computer device can be implemented as a server or a terminal, such as Figure 2 As shown, the training data generation method may include the following steps.

[0038] Step 210, obtaining a target text segment; the target text segment is any text segment obtained based on the original document.

[0039] The process of extracting text segments based on the original document can be implemented as follows: Perform text extraction on the original document to obtain the original text corresponding to the original document; The original text is segmented by a segmentation algorithm to obtain a text segment list corresponding to the original text; Get any text segment in the text segment list as the target text segment.

[0040] Since the original document may contain table information, image information, audio information, etc. in addition to text information, image recognition technology and speech recognition technology can be used to extract text from the original document. In this case, the process of extracting text from the original document and obtaining the original text corresponding to the original document can be implemented as follows: When the original document includes image information, image recognition is performed on the image information to obtain text content corresponding to the image information; When the original document includes audio information, speech recognition is performed on the audio information to obtain text content corresponding to the audio information.

[0041] Illustratively, in the case where the original document is a medical report, if there are medical images in the medical report, image recognition can be used to extract text information such as patient symptom descriptions and examination indicators. If the original document is a meeting record with accompanying voice content, voice recognition can be used to convert the voice signal in the audio into text information, for example, converting the speaker's speech into a text paragraph.

[0042] After obtaining the original text, the computer device may divide the text into segments using a segmentation algorithm to obtain a text segment list corresponding to the original text, wherein the segmentation algorithm includes at least one of the following: Segment the original text based on the target delimiter; Segment the original text based on text semantics; Segment the original text based on the number of words in the text; The original text is segmented based on the specified keywords.

[0043] Illustratively, the target separator can be a symbol used to mark the end of a sentence or a semantic unit, such as punctuation marks such as a period, question mark, exclamation mark, etc. The target separator can also be based on punctuation marks specified by relevant personnel based on actual needs, and this application does not impose any restrictions on this.

[0044] When the original text is segmented based on text semantics, the computer device can use the semantic analysis technology in natural language processing to input the original text into a pre-trained semantic analysis model to perform semantic understanding and analysis on the original text, and when a semantic conversion, topic switching, or change in logical level is detected, the text is divided into different text segments.

[0045] When performing segmentation based on the word count of the text, the computer device may pre-set a word count threshold, start counting from the first character of the original text, and truncate the text whenever the word count threshold is reached, thereby forming individual text segments.

[0046] When the original text is segmented based on designated keywords, the computer device can perform keyword recognition and use the location of the keywords as the basis for segmentation to truncate the text. The designated keywords can be set by relevant personnel based on actual needs, and this application does not impose any restrictions on this. For example, in scientific and technological literature, the designated keywords can be set as "experimental materials", "experimental steps", "experimental results", etc., so as to divide the text into segments based on the above keywords.

[0047] It should be noted that when segmenting the original text, the above segmentation algorithms can be used separately or in combination, and this application does not impose any restrictions on this.

[0048] In a possible implementation, the above process of processing the original document to obtain the text segment list can be implemented by a preprocessing tool, and the computer device inputs the original document into the preprocessing tool to obtain the text segment list corresponding to the original document output by the preprocessing tool. The preprocessing tool can be built in the computer device, or can also receive a call from the computer device; Figure 3 FIG. 4 shows a schematic diagram of a processing process of a preprocessing tool provided by an exemplary embodiment of the present application. Figure 3 As shown, the preprocessing tool 310 receives the original document; performs text extraction on the original text to obtain the original text; and performs segmentation processing on the original text through a segmentation algorithm to obtain and output a text segment list.

[0049] Step 220, extracting question-answer pairs from the target text segment through multiple large language models to obtain multiple question-answer pairs of the target text segment; the prompting methods of the various large language models are set differently.

[0050] Various types of large language models may be configured in a computer device. When performing question-answer pair extraction, multiple large language models may be randomly extracted from the configured large language models for question-answer pair extraction. That is to say, the number of large language models configured in the computer device may be greater than the number of large language models used when performing question-answer pair extraction. In this case, the computer device may perform template initialization on each large language model based on the prompt method corresponding to each large language model; and randomly extract multiple large language models from each initialized large language model.

[0051] That is to say, when the service is started, the computer device can configure the prompt request template for the large language model in advance to realize the template initialization of the large language model. For example, the prompt request template of one of the large language models is: "Please analyze the following content '[text segment content here]' and classify it according to the news category (politics, economy, culture, sports, technology), and directly reply with the category name without other redundant explanations." Or, "Please judge the emotional tendency of this text '[text segment content here]', and the reply format is 'positive', 'negative' or 'neutral'. Only one of these three answers is given without additional explanation", or "Please combine '[text segment content here]', according to scientific principles and common sense, to answer the question '[For example: How long is the earth's revolution period?]'. It is required to explain the principles and basis clearly, and present the answer content in plain language." And so on. Since the prompt methods of different large language models are different, the corresponding template settings are also different. The above prompt request templates are only illustrative, and this application does not limit this.

[0052] For each text segment in the text segment list, multiple large language models are randomly selected to perform question-answer pair extraction, and multiple question-answer pairs corresponding to each text segment are obtained by the output of each large language model. That is to say, the multiple large language models applied when extracting question-answer pairs for different text segments may partially overlap or not overlap; wherein, for any one of the text segments (i.e., the target text segment), after performing question-answer pair extraction through multiple large language models, multiple question-answer pairs can be obtained, and each question-answer pair contains a question text and a corresponding answer text; further, after obtaining multiple question-answer pairs, the computer device can filter abnormal information for the multiple question-answer pairs to perform preliminary screening of the question-answer pairs, and after deleting the question-answer pairs that are abnormal due to the large model itself or involve sensitive topics, the remaining question-answer pairs are used as the question-answer pair extraction result of the target text segment.

[0053] Step 230: When the multiple question-answer pairs include at least two target question-answer pairs with the same question text, a list of answer texts corresponding to the question texts is generated based on the at least two target question-answer pairs.

[0054] Figure 4 FIG. 4 shows a schematic diagram of the organizational relationship of question-answer pairs provided by an exemplary embodiment of the present application, such as Figure 4 As shown in , for multiple question-answer pairs of the target text segment, each question text may correspond to an answer text list; when the question texts of multiple question-answer pairs are the same, that is, when multiple target question-answer pairs are included, the answer text list corresponding to the question texts of the multiple target question-answer pairs includes multiple answer texts, such as Figure 4The question text 1 and question text 2 in the answer text list contain multiple answer texts; when the question text of the question-answer pair is different from the question texts of other question-answer pairs, the answer text list corresponding to the question text contains one answer text, such as Figure 4 The question text 3 in the answer text list contains an answer text.

[0055] Step 240, deduplicate at least two answer texts in the answer text list corresponding to the question text to obtain a reorganized question-answer pair consisting of the deduplicated answer text and the question text.

[0056] Since the question-answer pairs are obtained through multiple large language models or different processing methods, there may be multiple similar or even identical answer texts for the same question text. For example, for the question "What are the main components of the sun?", the possible answer texts include "hydrogen and helium", "mainly hydrogen and helium", "composed of hydrogen and helium", etc. In order to streamline and optimize the answer texts and avoid the negative impact of repeated information on the quality of training data, in an embodiment of the present application, the computer device will deduplicate multiple answer texts in the answer text list of the same question text.

[0057] In a possible implementation, the computer device may perform deduplication processing based on the similarity between answer texts. The process may be implemented as follows: Calculate the similarity of each answer text in the answer text list to obtain the similarity between each answer text; Based on the similarity between each answer text, at least two question and answer texts are deduplicated to obtain a recombined question and answer pair consisting of the deduplicated answer text and the question text.

[0058] When calculating the similarity of each answer text, the computer device can vectorize each answer text to obtain the vectorization result of each answer text; perform vectorized similarity calculation based on the vectorization result of each answer text to obtain the similarity between each answer text, wherein, when performing the similarity calculation, similarity calculation methods such as cosine similarity and Euclidean distance can be used for calculation, and this application does not limit this.

[0059] Among them, when performing deduplication processing based on similarity, in a possible implementation method, the computer device can determine the top N pairs of answer texts with the smallest similarity in at least two question and answer texts as the deduplicated answer texts; N is a positive integer.

[0060] The computer device can sort based on similarity and determine the top N answer texts with the least similarity as the deduplicated answer texts; further, since the similarity is calculated between each answer text, there may be repeated answer texts in the top N pairs of answer texts. When determining the deduplicated answer texts, the computer device can determine the non-repeated answer texts in the top N pairs of answer texts with the least similarity as the deduplicated answer texts.

[0061] In another possible implementation, the computer device may determine M pairs of answer texts whose relative degree is less than a similarity threshold among at least two answer texts as deduplicated answer texts; M is a positive integer.

[0062] In this case, the computer device can compare the similarity of each pair of answer texts with the similarity threshold, and determine the non-duplicate answer texts among the M pairs of answer texts whose similarity is less than the similarity threshold as the deduplicated answer texts. The value of the similarity threshold can be set based on actual needs, and this application does not impose any restrictions on this.

[0063] Step 250, generating training data based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

[0064] The computer device can obtain the reorganized question-answer pairs and the non-target question-answer pairs as the basis for generating training data. The process of generating training data can be implemented as follows: The reorganized question-answer pairs and non-target question-answer pairs are formatted based on the training data format of the target training model to obtain training data corresponding to the target training model.

[0065] In one possible implementation, a computer device may generate training data through a data generation tool, that is, the reorganized question-answer pairs and non-target question-answer pairs are input into the data generation tool, and the data generation tool converts the format of each question-answer pair based on the training data format of the target training model, thereby obtaining training data suitable for the target training model; the data generation tool may be configured in a computer device, or may be called by a computer device; optionally, the data generation tool may be pre-configured with training data formats corresponding to each training model, and the target training model may be any one of the training models, or, further, the data generation tool may also support customization of the training data format to expand the application scenarios of the data generation tool.

[0066] In summary, the training data generation method provided in the embodiment of the present application, for any text paragraph obtained based on the original document, extracts question and answer pairs respectively by setting multiple large language models with different prompting methods to obtain multiple question and answer pairs of the text segment, and in the case that there are multiple target question and answer pairs with the same question text, generates a reorganized question and answer pair based on the multiple target question and answer pairs, and the reorganized question and answer pair includes the question text and the reorganized answer text generated based on the answer text in the multiple target question and answer pairs, and then generates training data based on the reorganized question and answer pair and the non-target question and answer pairs in the multiple question and answer pairs except the target question and answer pairs; through the above method, the automation of training data generation can be realized, and the efficiency of training data generation can be improved. In the process, by extracting the same text segment information with the help of different prompting methods of multiple large language models, the question and answer pairs can be obtained from multiple perspectives, thereby improving the comprehensiveness of information extraction; by reorganizing the question and answer pairs with the same question text, the comprehensiveness and accuracy of the answer text in the question and answer pair are improved, so that when generating training data based on the reorganized question and answer pairs and the non-target question and answer pairs, the data quality and comprehensiveness of the training data can be improved, thereby improving the effect of model training.

[0067] In addition, when integrating the answer texts corresponding to the same question text, deduplication processing can be performed to reduce the redundancy of the answer texts, improve the refinement of the answer texts, and further improve the data quality of the generated training data.

[0068] Figure 5 A schematic diagram of a training data generation process provided by an exemplary embodiment of the present application is shown. Figure 5 As shown, after the computer device extracts text segments from the original document, multiple text segments are obtained, and question-answer pairs are extracted from each of the text segments through multiple large language models with different prompting methods to obtain multiple question-answer pairs corresponding to each text segment. For target question-answer pairs with the same question text, multiple answer texts are deduplicated and then integrated into a reorganized answer text of the question text to generate a reorganized question-answer pair. Finally, data format conversion is performed based on the reorganized question-answer pairs and non-target question-answer pairs to generate training data.

[0069] It should be noted that when determining the target question and answer pairs and reorganizing the target question and answer pairs, in one possible implementation, the computer device can determine and reorganize the question and answer pairs obtained by extracting the target text segments; in another possible implementation, taking into account the dispersion of information in the document, the computer device can also complete the extraction of question and answer pairs for all text segments corresponding to the original document, and determine the target question and answer pairs based on the question and answer pairs of all text segments and reorganize the target question and answer pairs; further, since different documents may involve different answers to the same question, the computer device can also complete the extraction of question and answer pairs for all text segments of multiple similar documents, and determine the target question and answer pairs based on the question and answer pairs of all text segments and reorganize the target question and answer pairs; the above-mentioned various applications can be selected based on actual needs, and this application does not limit this.

[0070] Figure 6 A block diagram of a training data generating device provided by an exemplary embodiment of the present application is shown. The device can be applied in a computer device to perform the following steps: Figure 1 or Figure 2 All or part of the steps of the embodiment shown, such as Figure 6 As shown, the training data generating device comprises: The text segment acquisition module 610 is used to acquire a target text segment; the target text segment is any text segment obtained based on the original document; A question-answer pair extraction module 620 is used to extract question-answer pairs from the target text segment using multiple large language models to obtain multiple question-answer pairs of the target text segment; different prompt modes are set between the large language models; A question-answer pair generation module 630 is configured to generate a reorganized question-answer pair based on at least two target question-answer pairs with the same question texts when the multiple question-answer pairs include at least two target question-answer pairs with the same question texts; the reorganized question-answer pair includes a question text and a reorganized answer text; and the reorganized answer text is generated based on the answer texts in at least two target question-answer pairs; The data generation module 640 is used to generate training data based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

[0071] In a possible implementation, the question-answer pair generating module 630 includes: A list generation submodule, used to generate an answer text list corresponding to the question text based on at least two of the target question-answer pairs; The deduplication submodule is used to deduplicate at least two answer texts in the answer text list corresponding to the question text to obtain the reorganized question-answer pair consisting of the deduplicated answer text and the question text.

[0072] In a possible implementation, the deduplication submodule includes: A similarity calculation unit, used for performing similarity calculation on each answer text in the answer text list to obtain the similarity between each answer text; The deduplication unit is used to perform deduplication processing on at least two question and answer texts based on the similarity between the answer texts to obtain the reorganized question and answer pair consisting of the deduplicated answer text and the question text.

[0073] In a possible implementation, the deduplication unit is used to determine the first N pairs of answer texts with the smallest similarity in at least two question and answer texts as the deduplicated answer texts; N is a positive integer.

[0074] In a possible implementation, the deduplication unit is used to determine M pairs of answer texts whose relative degree is less than a similarity threshold among at least two answer texts as deduplicated answer texts; M is a positive integer.

[0075] In a possible implementation manner, the device further includes: An initialization module, used to initialize the template of each large language model based on the prompt mode corresponding to each large language model; The model extraction module is used to randomly extract the multiple large language models from the initialized large language models.

[0076] In a possible implementation, the text segment acquisition module 610 includes: A text extraction submodule, used to extract text from the original document to obtain the original text corresponding to the original document; A segmentation processing submodule, used for segmenting the original text by using a segmentation algorithm to obtain a text segment list corresponding to the original text; The text segment acquisition submodule is used to acquire any one text segment in the text segment list as the target text segment.

[0077] In a possible implementation, the text extraction submodule is used to: In the case where the original document includes image information, performing image recognition on the image information to obtain text content corresponding to the image information; In the case that the original document includes audio information, speech recognition is performed on the audio information to obtain text content corresponding to the audio information.

[0078] In a possible implementation, the segmentation algorithm includes at least one of the following: Segmenting the original text based on the target delimiter; Segmenting the original text based on text semantics; Segmenting the original text based on the number of words in the text; The original text is segmented based on the specified keywords.

[0079] In a possible implementation, the data generation module 640 is used to convert the format of the reorganized question-answer pairs and the non-target question-answer pairs based on the training data format of the target training model to obtain training data corresponding to the target training model.

[0080] In summary, the training data generating device provided in the embodiment of the present application, for any text paragraph obtained based on the original document, extracts question and answer pairs respectively by setting multiple large language models with different prompting modes to obtain multiple question and answer pairs of the text segment, and in the case that there are multiple target question and answer pairs with the same question text, generates a reorganized question and answer pair based on the multiple target question and answer pairs, and the reorganized question and answer pair includes the question text and the reorganized answer text generated based on the answer text in the multiple target question and answer pairs, and then generates training data based on the reorganized question and answer pair and the non-target question and answer pairs in the multiple question and answer pairs except the target question and answer pairs; through the above-mentioned device, the automation of training data generation can be realized, and the efficiency of training data generation can be improved. In the process, by extracting the same text segment information with the help of different prompting modes of multiple large language models, the question and answer pairs can be obtained from multiple perspectives, thereby improving the comprehensiveness of information extraction; by reorganizing the question and answer pairs with the same question text, the comprehensiveness and accuracy of the answer text in the question and answer pair are improved, so that when generating training data based on the reorganized question and answer pairs and the non-target question and answer pairs, the data quality and comprehensiveness of the training data can be improved, thereby improving the effect of model training.

[0081] Figure 7 The structural block diagram of a computer device 700 shown in an exemplary embodiment of the present application is shown. The computer device can be implemented as a server in the above-mentioned solution of the present application. The computer device 700 includes a central processing unit (CPU) 701, a system memory 704 including a random access memory (RAM) 702 and a read-only memory (ROM) 703, and a system bus 705 connecting the system memory 704 and the central processing unit 701. The computer device 700 also includes a large-capacity storage device 706 for storing an operating system 709, an application program 710 and other program modules 711. The above-mentioned system memory 704 and the large-capacity storage device 706 can be collectively referred to as a memory.

[0082] According to various embodiments of the present disclosure, the computer device 700 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 700 can be connected to the network 708 through the network interface unit 707 connected to the system bus 705, or the network interface unit 707 can be used to connect to other types of networks or remote computer systems (not shown).

[0083] The memory also includes at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is stored in the memory. The central processing unit 701 implements all or part of the steps in the training data generation method shown in the above-mentioned embodiments by executing the at least one instruction, at least one program, code set or instruction set.

[0084] Figure 8 The following is a block diagram of a computer device 800 according to an exemplary embodiment of the present application. The computer device 800 may be implemented as the above-mentioned terminal device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, etc. The computer device 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, etc.

[0085] Typically, the computer device 800 includes a processor 801 and a memory 802 .

[0086] In some embodiments, the computer device 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807 and a power supply 808.

[0087] In some embodiments, the computer device 800 further includes one or more sensors 809 , including but not limited to: an acceleration sensor 810 , a gyroscope sensor 811 , a pressure sensor 812 , an optical sensor 813 , and a proximity sensor 814 .

[0088] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the computer device 800, and the computer device 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0089] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one computer program is stored, and the computer program is loaded and executed by a processor to implement the above-mentioned training data generation method, and / or all or part of the steps in the training data generation method. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0090] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer is executed to implement the above Figure 1 or Figure 2 All or part of the steps of the training data generation method shown in the embodiment.

[0091] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0092] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating training data, characterized in that: The method comprises: Obtaining a target text segment; the target text segment is any text segment obtained based on the original document; Extracting question-answer pairs from the target text segment through multiple large language models to obtain multiple question-answer pairs of the target text segment; different prompting modes are set between the large language models; In the case where the multiple question-answer pairs include at least two target question-answer pairs with the same question text, a reorganized question-answer pair is generated based on the at least two target question-answer pairs; the reorganized question-answer pair includes a question text and a reorganized answer text; the reorganized answer text is generated based on the answer texts in the at least two target question-answer pairs; Training data is generated based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

2. The method according to claim 1, characterized in that The generating a recombined question-answer pair based on at least two of the target question-answer pairs comprises: Based on at least two of the target question-answer pairs, generating a list of answer texts corresponding to the question texts; De-duplication processing is performed on at least two answer texts in the answer text list corresponding to the question text to obtain the reorganized question-answer pair consisting of the de-duplicated answer text and the question text.

3. The method according to claim 2, characterized in that The step of performing deduplication processing on at least two answer texts in the answer text list to obtain the reorganized question-answer pair consisting of the deduplicated answer text and the question text includes: Calculating the similarity of each answer text in the answer text list to obtain the similarity between each answer text; At least two question-answer texts are deduplicated based on the similarity between the answer texts to obtain the reorganized question-answer pair consisting of the deduplicated answer text and the question text.

4. The method according to claim 3, characterized in that The deduplication processing of at least two question and answer texts based on the similarity between the answer texts includes: The top N pairs of answer texts with the smallest similarity among at least two question-answer texts are determined as the answer texts after deduplication; N is a positive integer.

5. The method according to claim 3, characterized in that: The deduplication processing of at least two question and answer texts based on the similarity between the answer texts includes: M pairs of answer texts whose relative degree among at least two answer texts is less than a similarity threshold are determined as answer texts after deduplication; M is a positive integer.

6. The method according to claim 1, characterized in that The method further comprises: Initialize the template of each large language model based on the prompt method corresponding to each large language model; The multiple large language models are randomly extracted from the initialized large language models.

7. The method according to claim 1, characterized in that The step of obtaining the target text segment includes: Performing text extraction on the original document to obtain the original text corresponding to the original document; Segment the original text using a segmentation algorithm to obtain a text segment list corresponding to the original text; Any text segment in the text segment list is obtained as the target text segment.

8. The method according to claim 7, characterized in that The extracting text from the original document to obtain the original text corresponding to the original document includes: In the case where the original document includes image information, performing image recognition on the image information to obtain text content corresponding to the image information; In the case that the original document includes audio information, speech recognition is performed on the audio information to obtain text content corresponding to the audio information.

9. The method according to claim 7, characterized in that: The segmentation algorithm includes at least one of the following: Segmenting the original text based on the target delimiter; Segmenting the original text based on text semantics; Segmenting the original text based on the number of words in the text; The original text is segmented based on the specified keywords.

10. The method according to claim 1, characterized in that The generating of training data based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs, including: The reorganized question-answer pairs and the non-target question-answer pairs are formatted based on the training data format of the target training model to obtain training data corresponding to the target training model.

11. A training data generating device, characterized in that: The device comprises: A text segment acquisition module is used to acquire a target text segment; the target text segment is any text segment obtained based on the original document; A question-answer pair extraction module, used to extract question-answer pairs from the target text segment through multiple large language models to obtain multiple question-answer pairs of the target text segment; the prompt modes of the various large language models are set differently; A question-answer pair generation module, configured to generate a reorganized question-answer pair based on at least two target question-answer pairs having the same question text, when the multiple question-answer pairs include at least two target question-answer pairs having the same question text; the reorganized question-answer pair includes a question text and a reorganized answer text; the reorganized answer text is generated based on the answer texts in at least two target question-answer pairs; A data generation module is used to generate training data based on the reorganized question-answer pairs and non-target question-answer pairs; the non-target question-answer pairs are other question-answer pairs among the multiple question-answer pairs except the target question-answer pairs.

12. The device according to claim 11, characterized in that The question-answer pair generating module comprises: A list generation submodule, used to generate an answer text list corresponding to the question text based on at least two of the target question-answer pairs; The deduplication submodule is used to deduplicate at least two answer texts in the answer text list corresponding to the question text to obtain the reorganized question-answer pair consisting of the deduplicated answer text and the question text.

13. The device according to claim 12, characterized in that The deduplication submodule comprises: A similarity calculation unit, used for performing similarity calculation on each answer text in the answer text list to obtain the similarity between each answer text; The deduplication unit is used to perform deduplication processing on at least two question and answer texts based on the similarity between the answer texts to obtain the reorganized question and answer pair consisting of the deduplicated answer text and the question text.

14. The device according to claim 13, characterized in that The deduplication unit is used to determine the first N pairs of answer texts with the smallest similarity in at least two question and answer texts as the deduplicated answer texts; N is a positive integer.

15. The device according to claim 13, characterized in that The deduplication unit is used to determine M pairs of answer texts whose relative degrees are less than a similarity threshold among at least two answer texts as deduplicated answer texts; M is a positive integer.

16. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the training data generation method according to any one of claims 1 to 10.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the computer program is loaded and executed by a processor to implement the training data generating method according to any one of claims 1 to 10.

18. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device is caused to execute to implement the training data generation method as described in any one of claims 1 to 10.

Images