Corpus enhancement method and device and electronic equipment

Abstract

The invention is suitable for the technical field of artificial intelligence, and provides a corpus enhancement method and device and electronic equipment. The corpus enhancement method comprises the steps of inputting a problem rewriting prompt instruction into a first large language model for problem rewriting processing, obtaining rewriting problem information, carrying out similarity evaluation processing through a first similarity evaluation model, obtaining a first similarity evaluation result of the rewriting problem information output by the first similarity evaluation model and seed corpus problem information, and obtaining a second similarity evaluation result of the rewriting problem information and the seed corpus problem information; if the problem information indicating rewriting is not similar to the seed corpus problem information, inputting a problem disassembling prompt instruction into a second large language model to perform problem disassembling processing, obtaining multiple pieces of sub-problem information, performing similarity evaluation processing through a second similarity evaluation model, and obtaining the sub-problem information; and obtaining a second similarity evaluation result of the sub-question information output by the second similarity evaluation model and the seed corpus question information, and if it is indicated that the sub-question information is not similar to the seed corpus question information, reserving the sub-question information. According to the method, the quantity, magnitude and quality of the text corpora are improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to corpus enhancement methods, devices and electronic devices. Background Technology

[0002] With the rapid development of artificial intelligence and natural language processing (NLP) technologies, text data is increasingly being used in various fields such as machine translation, text classification, semantic analysis, and dialogue systems. Among these, the emergence of Large Language Models (LLMs) has brought revolutionary progress to these applications. LLMs perform exceptionally well on multiple tasks, but their training heavily relies on large and diverse corpora. From the perspective of model understanding of natural language, rich, accurate, and diverse training corpora directly impact the model's performance and robustness.

[0003] However, existing corpus augmentation techniques have several shortcomings. There are two main types of existing corpus augmentation techniques. The first type generates new corpus by modifying existing corpus, such as synonym replacement, sentence shuffling, and word order inversion. Synonym replacement involves replacing certain words in the original text with their synonyms to generate a different but semantically similar sentence. This method is relatively simple and effectively increases the size and diversity of the dataset. For example, the original sentence "He is a smart student" becomes "He is a wise student." Sentence shuffling involves rearranging the order of phrases or clauses in a sentence to generate a new sentence. This method can improve data diversity. For example, the original sentence "The cat sat on the cushion, and the dog barked at it" becomes "The dog barked at it, and the cat sat on the cushion." Word order inversion involves reversing the order of words in a sentence to generate a new sentence. This is often used to handle inversion phenomena in specific tasks, such as machine translation or sentence alignment. This method may result in sentences that do not conform to grammatical rules or, in some cases, change the meaning of the original sentence. Examples of word order reversal include the sentence "I read this book every day," which becomes "I read this book every day." While such alterations to existing corpora, such as synonym replacement, sentence shuffling, and word order reversal, enrich the content and improve the quality of the corpus to some extent, they also have the following three drawbacks: 1. Overuse of this method may lead to semantic differences between the generated corpus and the original corpus, even changing the meaning of the original sentence. 2. The generated sentences may not conform to grammatical rules or be ambiguous in meaning. 3. It involves significant manual costs. The second method is to use Generative Adversarial Networks (GANs) to enhance the corpus. GANs are increasingly widely used in the field of text corpus enhancement. The basic idea of ​​GANs is to design two neural networks: a generator network and a discriminator network. The generator network generates samples from random noise, while the discriminator network attempts to distinguish between generated and real samples. The two compete against each other, iterating and optimizing continuously, aiming to make the generator network generate increasingly realistic fake samples and the discriminator network increasingly unable to distinguish between real and fake samples. However, the technical solution of using Generative Adversarial Networks (GANs) to enhance corpora also has the following four shortcomings: 1. Training difficulty. The training process of GANs requires coordinating the relationship between the generator network and the discriminator network, which is a difficult process to achieve a good balance. An overly powerful generator network may cause the model to converge prematurely and produce mode collapse, that is, the generated text lacks diversity; conversely, an overly powerful discriminator network will make it difficult for the generator network to find a direction for improvement. 2. Discreteness problem.Unlike image generation, text generation involves discrete sequences of symbols. This makes it impossible for GANs to directly use gradient descent for text generation tasks, as this method requires the objective function to be continuously differentiable. 3. Evaluation Difficulty: Due to the lack of clear evaluation criteria, measuring the quality and diversity of generated text is very difficult. Although some evaluation metrics exist, such as BLEU and ROUGE, they often only partially reflect the quality of the generated text. 4. Quality of Generated Text: Although generative adversarial networks can generate novel text, sometimes the generated text may have problems with grammatical or logical consistency, or deviate from the intended topic. Summary of the Invention

[0004] This application provides a corpus enhancement method, apparatus, and electronic device, which can solve the problems of training difficulties and high labor costs in existing corpus enhancement technologies.

[0005] In a first aspect, embodiments of this application provide a corpus enhancement method, including:

[0006] Information on obtaining seed corpus questions;

[0007] The problem rewriting prompt instruction is input into the first large language model for problem rewriting processing, and the rewritten problem information output by the first large language model is obtained, wherein the problem rewriting prompt instruction includes the problem information of the seed corpus;

[0008] The rewriting problem information and the seed corpus problem information are input into a first similarity evaluation model for similarity evaluation processing, and the first similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the first similarity evaluation model is obtained.

[0009] If the first similarity evaluation result indicates that the rewritten problem information is not similar to the problem information in the seed corpus, the problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained, wherein the problem decomposition prompt instruction includes the rewritten problem information;

[0010] The sub-question information and the seed corpus question information are input into the second similarity evaluation model for similarity evaluation processing, and the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model is obtained;

[0011] If the second similarity assessment result indicates that the sub-question information is not similar to the seed corpus question information, the sub-question information is retained.

[0012] In one possible implementation of the first aspect, the first similarity evaluation model includes multiple similarity evaluation sub-models. The step of inputting the rewritten problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtaining the first similarity evaluation result of the rewritten problem information and the seed corpus problem information output by the first similarity evaluation model, includes:

[0013] The rewriting problem information and the seed corpus problem information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the at least one similarity evaluation sub-model is obtained;

[0014] Based on the at least one model similarity evaluation result, a first similarity evaluation result is determined between the rewritten problem information and the seed corpus problem information.

[0015] In one possible implementation of the first aspect, the second similarity evaluation model includes multiple similarity evaluation sub-models. The step of inputting the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtaining the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model, includes:

[0016] The sub-question information and the seed corpus question information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the sub-question information and the seed corpus question information output by the at least one similarity evaluation sub-model is obtained;

[0017] Based on the at least one model similarity evaluation result, a second similarity evaluation result is determined between the sub-question information and the seed corpus question information.

[0018] In one possible implementation of the first aspect, the similarity evaluation sub-model includes any of the following: a contrastive learning model based on text embeddings, or a large language model for similarity problem evaluation.

[0019] In one possible implementation of the first aspect, the method further includes:

[0020] If the first similarity assessment result indicates that the rewritten question information is similar to the question information in the seed corpus, the step of inputting the question rewriting prompt instruction into the first large language model for question rewriting processing is performed again until the first similarity assessment result indicates that the rewritten question information is not similar to the question information in the seed corpus.

[0021] In one possible implementation of the first aspect, the problem rewriting suggestion instruction further includes problem rewriting suggestion words, a problem rewriting database table, and problem rewriting sample examples. The step of inputting the problem rewriting suggestion instruction into a first large language model for problem rewriting processing and obtaining the rewriting problem information output by the first large language model includes:

[0022] The question rewriting prompt instruction is input into the first large language model for question rewriting processing, and the rewritten question information output by the first large language model is obtained. The question rewriting prompt instruction includes the seed corpus question information, the question rewriting prompt words, the question rewriting database table, and the question rewriting sample examples.

[0023] In one possible implementation of the first aspect, the problem decomposition hint instruction further includes problem decomposition hint words, a problem decomposition database table, and problem decomposition sample examples. The step of inputting the problem decomposition hint instruction into a second language model for problem decomposition processing, and obtaining multiple sub-problem information output by the second language model, includes:

[0024] The problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained. The problem decomposition prompt instruction includes the rewritten problem information, the problem decomposition prompt words, the problem decomposition database table, and the problem decomposition sample example.

[0025] In one possible implementation of the first aspect, the method further includes:

[0026] The sub-problem information is used as the seed corpus problem information, and the corpus enhancement method is applied again.

[0027] Secondly, embodiments of this application provide a corpus enhancement device, including:

[0028] The problem information acquisition module is used to acquire problem information from the seed corpus.

[0029] The problem rewriting processing module is used to input the problem rewriting prompt instruction into the first large language model for problem rewriting processing, and obtain the rewriting problem information output by the first large language model, wherein the problem rewriting prompt instruction includes the problem information of the seed corpus;

[0030] The first similarity evaluation module is used to input the rewriting problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtain the first similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the first similarity evaluation model;

[0031] The problem decomposition processing module is used to input a problem decomposition prompt instruction into the second large language model for problem decomposition processing if the first similarity evaluation result indicates that the rewritten problem information is not similar to the problem information in the seed corpus, and to obtain multiple sub-problem information output by the second large language model, wherein the problem decomposition prompt instruction includes the rewritten problem information;

[0032] The second similarity evaluation module is used to input the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtain the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model;

[0033] The sub-question information retention module is used to retain the sub-question information if the second similarity evaluation result indicates that the sub-question information is not similar to the seed corpus question information.

[0034] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the corpus enhancement method as described in the first aspect above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the corpus enhancement method as described in the first aspect above.

[0036] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the corpus enhancement method described in the first aspect.

[0037] The beneficial effects of the embodiments in this application compared with the prior art are:

[0038] This application obtains seed corpus question information, inputs a question rewriting prompt instruction into a first large language model for question rewriting processing, obtains rewritten question information output by the first large language model, wherein the question rewriting prompt instruction includes the seed corpus question information, inputs the rewritten question information and the seed corpus question information into a first similarity evaluation model for similarity evaluation processing, obtains a first similarity evaluation result between the rewritten question information and the seed corpus question information output by the first similarity evaluation model, and if the first similarity evaluation result indicates that the rewritten question information and the seed corpus question information... If the information is dissimilar, the problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, obtaining multiple sub-problem information output by the second large language model. The problem decomposition prompt instruction includes rewriting the problem information. The sub-problem information and the seed corpus problem information are input into the second similarity evaluation model for similarity evaluation processing, obtaining the second similarity evaluation result of the sub-problem information and the seed corpus problem information output by the second similarity evaluation model. If the second similarity evaluation result indicates that the sub-problem information and the seed corpus problem information are dissimilar, the sub-problem information is retained. The corpus enhancement scheme provided in this application can generate text corpus such as sub-problem information by using the first large language model for problem rewriting processing and the second large language model for problem decomposition processing. Furthermore, the text corpus such as sub-problem information can be generated in batches multiple times through iteration. Moreover, in this application's scheme, the similarity evaluation of information through the first and second similarity evaluation models allows for multiple settings and adjustments to the quality of the generated corpus, ensuring that the quality of the generated sub-problem information corpus meets the standards. By retaining sub-question information that is dissimilar to the seed corpus information, the independence of sub-question information from the seed corpus information is ensured. Specifically, sub-question information similar to the original seed corpus information is excluded to avoid duplication and redundancy. Sub-question information dissimilar to the original seed corpus information is retained to enrich and expand the diversity of questions. This application's solution significantly improves the quantity and quality of text corpora, greatly reduces the cost of manually generated corpora, and provides a high-quality corpus guarantee for subsequent language model training and optimization, as well as the construction and expansion of knowledge base corpora.

[0039] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic flowchart of a corpus enhancement method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the structure of a corpus enhancement device provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0045] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0046] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0047] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0048] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0049] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0050] The corpus enhancement method provided in this application can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0051] For example, the electronic device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a set-top box (STB), customer premises equipment (CPE), and / or other devices for communication over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks or mobile terminals in future evolved Public Land Mobile Network (PLMN) networks.

[0052] Figure 1 This is a schematic flowchart of a corpus enhancement method provided in an embodiment of this application.

[0053] S11, Obtain seed corpus information.

[0054] Here, pre-configured seed corpus question information can be obtained. Alternatively, manually input seed corpus question information can also be obtained. Those skilled in the art should understand that all methods of obtaining seed corpus question information fall within the scope of this application.

[0055] Multiple seed corpus questions can form a seed corpus. The design requirements for seed corpus questions include the following two points: 1. Diverse Question Types: To enhance the diversity of the seed corpus, it needs to include various types of questions. Ensuring diversity helps the large language model iteratively generate more different types of questions. 2. Content Validation: Each seed corpus question undergoes a rigorous screening and validation process before being included in the seed corpus. This includes verifying the accuracy of the corpus content to ensure it aligns with facts and database knowledge in the relevant domain. Simultaneously, the relevance of the corpus is evaluated to ensure it is closely related to the intended application scenario, such as a specific vertical domain application scenario.

[0056] S12, input the problem rewriting prompt instruction into the first large language model for problem rewriting processing, and obtain the rewriting problem information output by the first large language model, wherein the problem rewriting prompt instruction includes the problem information of the seed corpus.

[0057] Here, the primary language model (e.g., GPT-4, which OpenAI released for the chatbot ChatGPT; GPT stands for Generative Pre-trained Transformer) is used for question rewriting. Upon receiving a question rewriting prompt including seed corpus information, the primary language model outputs the rewritten question information. The rewritten question information is the question information output by the primary language model based on the seed corpus information. For example, the seed corpus information could be "What is the amount on each invoice?". The rewritten question information output by the primary language model could be "Query the total quantity and total value of the goods included in each invoice".

[0058] In the first major language model, the temperature parameter typically needs to be increased; it can be set to 1. The temperature parameter is a hyperparameter used to adjust the creativity and diversity of the text generated by the model. It is a value greater than 0, usually between 0 and 1. The temperature parameter primarily controls the randomness and diversity of the text generated by the model. A higher temperature parameter leads to more diverse and random text, while a lower temperature parameter results in more deterministic and consistent text.

[0059] In one embodiment, the question rewriting suggestion instruction further includes question rewriting suggestion words, a question rewriting database table, and question rewriting sample examples. The step of inputting the question rewriting suggestion instruction into a first large language model for question rewriting processing and obtaining the rewriting question information output by the first large language model includes:

[0060] The question rewriting prompt instruction is input into the first large language model for question rewriting processing, and the rewritten question information output by the first large language model is obtained. The question rewriting prompt instruction includes the seed corpus question information, question rewriting prompt words, question rewriting database table, and question rewriting sample examples.

[0061] Specifically, the question rewriting prompts can guide the First Language Model to generate complex and diverse rewritten question information, while also limiting the format and word count of the generated rewritten question information. Furthermore, the prompts can specify which question rewriting database tables the First Language Model should refer to when generating rewritten question information. Additionally, the prompts can ensure that the basic structure of the rewritten question information meets user requirements.

[0062] Here is an example of a question rewriting prompt:

[0063] "I need you to play the role of a data generation expert. First, you need to understand the following #Prompt#, which consists of two parts: a table DDL (i.e., the question rewriting database table) and user questions (i.e., seed corpus question information). The executable SQL statements corresponding to the user questions can correctly query results in the table created by the table DDL. Your goal is to rewrite the user questions in #Prompt#. Specific requirements are as follows: 1. Try not to make the rewritten content too long; it can be increased by a maximum of about 10 words. 2. The rewritten user questions should be more complex, diverse, and reasonable, and should be understandable and answerable by humans. 3. The rewritten questions must completely reference the table DDL, ensuring that the rewritten questions (i.e., rewritten question information) can be correctly queried in the table. You can expand upon the information appropriately, but it cannot deviate from the table DDL. 4. When encountering dates or other numeric data, you can change the dates; apply this principle to other situations. 5. Only generate rewritten questions; do not add other descriptive information."

[0064] The problem rewriting database tables are described using Database Definition Language (DDL). Rewriting the database tables allows the first language model to understand the problem background and related content. Fields in the problem rewriting database tables should be as concise as possible (containing only field names, field types, and comments). Non-standard field names need to be adjusted to ensure they can be understood by the first language model after tokenization.

[0065] The following is an example of how to rewrite a database table:

[0066]

[0067] The following is an explanation of the example of rewriting the database table to address the above issues:

[0068] "This code creates a table named 'invoice_items' to store detailed information about invoice items. The table contains five fields: InvoiceLineId, InvoiceId, TrackId, UnitPrice, and Quantity. InvoiceLineId is the primary key, while TrackId and InvoiceId are foreign keys, referencing the corresponding fields in the tracks and invoices tables, respectively. Indexes are created on the TrackId and InvoiceId fields to improve query performance. This table is typically designed to record detailed information for each invoice item, including the associated record, invoice, unit price, and quantity."

[0069] Question rewriting sample examples can be in the form of few-shot examples. For example, three specific examples can be provided, each containing an original question text and its corresponding rewritten question text. These examples are used to demonstrate to the first major language model how the original question can be transformed into a more complex and diverse question.

[0070] Here is an example of a problem rewriting sample:

[0071] Original question: What is the total amount of each invoice?

[0072] Rewrite the problem: Query the total quantity and total amount of goods included in each invoice.

[0073] A question rewriting prompt is composed of seed corpus question information, question rewriting prompt words, a question rewriting database table, and question rewriting sample examples. This prompt is input into a first large language model for question rewriting processing, yielding rewritten question information output by the first large language model. The temperature parameter of the first large language model can be increased to enhance its randomness, ensuring that the generated rewritten question information exhibits greater complexity and diversity while maintaining the original meaning of the seed corpus question information. By inputting the question rewriting prompt, which includes the seed corpus question information, question rewriting prompt words, question rewriting database table, and question rewriting sample examples, into the first large language model for question rewriting processing, and obtaining the rewritten question information output by the first large language model, concise and easy-to-understand questions can be effectively generated, improving question quality, even with limited rewritten sample examples and intuitive expressions.

[0074] S13, input the rewriting problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtain the first similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the first similarity evaluation model.

[0075] Here, the first similarity assessment model refers to a model used to evaluate the similarity between two pieces of information. By inputting the rewritten question information and the seed corpus question information into the first similarity assessment model for similarity evaluation, the first similarity assessment result between the rewritten question information and the seed corpus question information can be obtained. The first similarity assessment result can be qualitative (similar, dissimilar) or quantitative (similarity value).

[0076] In one embodiment, the first similarity evaluation model includes multiple similarity evaluation sub-models. The step of inputting the rewritten problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtaining the first similarity evaluation result of the rewritten problem information and the seed corpus problem information output by the first similarity evaluation model, includes:

[0077] The rewriting problem information and the seed corpus problem information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the at least one similarity evaluation sub-model is obtained;

[0078] Based on the at least one model similarity evaluation result, a first similarity evaluation result is determined between the rewritten problem information and the seed corpus problem information.

[0079] Here, the first similarity evaluation model refers to a similarity evaluation system that includes multiple similarity evaluation sub-models that can be used to evaluate the similarity between two pieces of information. A similarity evaluation sub-model is a model that can be used to evaluate the similarity between two pieces of information. The model similarity evaluation result can be qualitative (similar or dissimilar) or quantitative (similarity value). Taking qualitative similarity or dissimilarity as an example, after obtaining multiple model similarity evaluation results for the rewritten problem information and the seed corpus problem information output by multiple similarity evaluation sub-models, the majority of these model similarity evaluation results can be determined as the first similarity evaluation result. For example, if two out of three model similarity evaluation results indicate that the rewritten problem information and the seed corpus problem information are dissimilar, then the first similarity evaluation result is dissimilar. Taking quantitative similarity value as an example, after obtaining multiple model similarity evaluation results for the rewritten problem information and the seed corpus problem information output by multiple similarity evaluation sub-models, the calculated result can be determined as the first similarity evaluation result by taking the average or a weighted average.

[0080] For example, when the first similarity evaluation model includes three similarity evaluation sub-models, the rewritten question information and the seed corpus question information can be input into the three similarity evaluation sub-models respectively, and the three model similarity evaluation results output by these three similarity evaluation sub-models can be obtained. Then, based on these three model similarity evaluation results, the first similarity evaluation result between the rewritten question information and the seed corpus question information is determined.

[0081] The first similarity evaluation model could be a first multi-model expert system. This first multi-model expert system uses the voting evaluation mechanism described above for similarity evaluation.

[0082] In one embodiment, the similarity evaluation sub-model includes any one of the following: a contrastive learning model based on text embeddings, or a large language model for similarity problem evaluation.

[0083] Among them, large language models (LLMs) for similarity problem evaluation include, but are not limited to, GPT-4 and Deepseek-V2-Chat.

[0084] A contrastive learning model based on text embeddings could be the SimCSE (Simple Contrastive Sentence Embedding) contrastive learning evaluation model. SimCSE is a contrastive learning model for learning sentence embeddings. Using the supervised version of SimCSE for evaluating rewritten question information primarily considers its direct use in vertical training, thus ensuring that the SimCSE model can better understand the semantic information of the vertical domain corpus during evaluation. SimCSE can evaluate the similarity between the rewritten question information and the seed question information during the inference phase. After training, the SimCSE model performs inference and calculates cosine similarity, setting a similarity threshold of 0.5. The specific criteria are as follows: if the absolute value of the cosine similarity is less than the similarity threshold, the rewritten question information and the seed question information are considered similar. If the absolute value of the cosine similarity is greater than the similarity threshold, the rewritten question information and the seed question information are considered dissimilar.

[0085] GPT-4 can evaluate similar questions (GPT-4 is a language model released by OpenAI for the chatbot ChatGPT; GPT stands for Generative Pre-trained Transformer). The general capabilities of closed-source models are far superior to small-sized models like SimCSE. The specific process involves first designing question rewriting prompts (i.e., language model prompts). These prompts include seed question information, rewriting prompt words, a question rewriting database table, and rewriting sample examples. The model similarity evaluation results (similarity scores) output by GPT-4 can range from [0,1]. Here, 0 represents the most similar, and 1 represents the least similar. A similarity threshold of 0.5 can be set for discrimination. If the model similarity evaluation result between the rewritten question information and the seed corpus question information is less than 0.5, the rewritten question information is considered similar to the seed corpus question information. If the model similarity evaluation result between the rewritten question information and the seed corpus question information is greater than 0.5, the rewritten question information is considered dissimilar to the seed corpus question information.

[0086] Deepseek-V2-Chat has strong understanding and reasoning capabilities in the Chinese domain, and this model can be selected for evaluation of similar problems.

[0087] In one embodiment, a contrastive learning model for generating text embeddings is trained based on the following steps:

[0088] In the supervised training phase, a training corpus is first constructed, consisting of three types of sentences:

[0089]

[0090] in, This refers to the seed corpus itself. Indicates a positive sample. Let represent a negative sample, where i∈[1,m]. You can choose to load the WWM (Whole Word Masking) version of RoBERTa (Robustly Optimized BERT Pretraining Approach, also known as an enhanced version of the bidirectional Transformer encoder) onto the SimCSE model. The seed corpus, positive samples, and negative samples are encoded into vectors by the pre-trained model:

[0091]

[0092] Next, the objective function l i First, calculate the embeddings y of the two positive sample sentences. i and The differences between them are used to measure their similarity. To further enhance the model's discriminative ability, a [missing information] is introduced. (Negative samples) These negative samples are semantically dissimilar to positive samples, but may be somewhat misleading. By introducing these negative samples, the objective function l i This encourages the model to focus more on the similarity between positive samples during contrastive learning, while reducing the interference of negative samples on the similarity of positive samples. Finally, the loss function l is calculated using the backpropagation algorithm. i The gradient of the model parameters is calculated, and the gradient descent method is used to update the model parameters in order to minimize the loss function.

[0093] During training, the model parameters were set to epoch=4, dropout_rate=0.1, batch_size=32, and the pooling layer (polling layer) was represented by cls vectors to ensure that the model maintained its generalization ability while avoiding overfitting. After testing, the model achieved an accuracy of 98.5%, demonstrating extremely high performance.

[0094] S14, if the first similarity evaluation result indicates that the rewritten problem information is not similar to the seed corpus problem information, the problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained, wherein the problem decomposition prompt instruction includes the rewritten problem information.

[0095] The first similarity assessment result can be qualitative (similar or dissimilar) or quantitative (similarity value). When the first similarity assessment result is quantitative, for example, if the first similarity assessment result is less than the first similarity threshold (e.g., 0.5), the first similarity assessment result can be determined as similar. If the first similarity assessment result is greater than the first similarity threshold (e.g., 0.5), the first similarity assessment result can be determined as dissimilar.

[0096] Here, the second language model is used for problem decomposition. Upon receiving a problem decomposition prompt including rewriting problem information, the second language model outputs multiple sub-problem information. Sub-problem information is the information derived from the rewritten problem information after being decomposed by the second language model based on the rewritten problem information. For example, the rewritten problem information could be "Query the total quantity and total value of goods included in each invoice." The multiple sub-problem information output by the second language model could be, for example, "Total quantity included in each invoice," "Unit price of goods in each invoice," and "Total value of goods in each invoice."

[0097] Rewritten problem information that is dissimilar to the problem information in the seed corpus is retained. This rewritten problem information is usually more complex, and some descriptions are verbose. Therefore, in S14, the rewritten problem information is further broken down and refined using the second major language model.

[0098] In one embodiment, the problem decomposition prompt instruction further includes problem decomposition prompt words, a problem decomposition database table, and problem decomposition sample examples. The step of inputting the problem decomposition prompt instruction into a second language model for problem decomposition processing, and obtaining multiple sub-problem information output by the second language model, includes:

[0099] The problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained. The problem decomposition prompt instruction includes the rewritten problem information, the problem decomposition prompt words, the problem decomposition database table, and the problem decomposition sample example.

[0100] The problem decomposition prompts are designed to provide clear guidance and context for the second-largest language model, enabling more accurate understanding, processing, and decomposition of problems. The decomposed sub-problem information must satisfy sentence semantic integrity.

[0101] Among them, the problem decomposition prompts are used to indicate the problem decomposition goals that the second major language model needs to achieve.

[0102] Here are some examples of problem breakdown hints:

[0103] "You are a problem decomposition expert. Your goal is to break down the original problem (i.e., rewritten problem information) into several sub-problems (i.e., sub-problem information), based on DDL tables (i.e., problem decomposition database tables). The requirements are as follows:"

[0104] 1. Decompose the problem based on the original problem, simplifying it as much as possible, and ensuring that it is not closely related to the original problem;

[0105] 2. Referring to the field information in the DDL, the split sub-problems must be able to yield executable SQL statements in this table;

[0106] 3. Use divergent thinking to break down sub-problems, requiring the ability to apply knowledge to new situations, but avoid using vague terms such as "a certain" or "a specific one" in the generated questions;

[0107] 4. Each question should be expressed in a different way, sub-questions should be independent, and no more than 5 questions should be generated;

[0108] 5. List only the sub-problems, do not generate other descriptive information, and do not include serial numbers before the sub-problems.

[0109] ----------------------------------”

[0110] The problem is broken down into database tables that are described using Database Definition Language (DDL).

[0111] Here is an example of breaking down a problem into database tables:

[0112] ------------------------------

[0113] --invoice_items table

[0114] ------------------------------

[0115] CREATE TABLE "invoice_items"(

[0116] "InvoiceLineId"INTEGER NOT NULL,

[0117] "InvoiceId"INTEGER NOT NULL,

[0118] "TrackId"INTEGER NOT NULL,

[0119] "UnitPrice" REAL NOT NULL,

[0120] "Quantity" INTEGER NOT NULL,

[0121] PRIMARY KEY("InvoiceLineId"),

[0122] CONSTRAINT"0"FOREIGN KEY("TrackId")REFERENCES"tracks"("TrackId")ONUPDATE NO ACTION ON DELETE NO ACTION,

[0123] CONSTRAINT"1"FOREIGN KEY("InvoiceId")REFERENCES"invoices"("InvoiceId")ON UPDATE NO ACTION ON DELETE NO ACTION ) ;

[0126] CREATE INDEX"IFK_InvoiceLineTrackId"ON"invoice_items"("TrackId");

[0127] CREATE INDEX"IFK_InvoiceLineInvoiceId"ON"invoice_items"("InvoiceId")

[0128] The following is an explanation of the example of breaking down database tables to address the above issues:

[0129] "This code creates a table named 'invoice_items' to store detailed information about invoice items. The table contains five fields: InvoiceLineId, InvoiceId, TrackId, UnitPrice, and Quantity. InvoiceLineId is the primary key, while TrackId and InvoiceId are foreign keys, referencing the corresponding fields in the tracks and invoices tables, respectively. Indexes are created on the TrackId and InvoiceId fields to improve query performance. This table is typically designed to record detailed information for each invoice item, including the associated record, invoice, unit price, and quantity."

[0130] Problem decomposition sample examples can be in the form of few-shot examples. For example, three specific examples can be provided, each containing a rewritten problem statement and its corresponding decomposed sub-problem statements. These examples are used to demonstrate to the second-largest language model how to transform the original problem into sub-problem statements.

[0131] Here is an example of a problem breakdown sample:

[0132] "Original question (i.e., rewritten question information): Query the total quantity and total value of goods included in each invoice;

[0133] Sub-problems (i.e., sub-problem information):

[0134] The total number of items contained in each invoice;

[0135] The unit price of the goods on each invoice;

[0136] The total value of the goods on each invoice.

[0137] By inputting problem decomposition prompts into a second-largest language model for problem decomposition, and obtaining multiple sub-problem information output by the second-largest language model, the rewrite problem information is effectively broken down into multiple more specific and easier-to-process sub-problem information. This not only improves the manageability of the problem but also enhances the accuracy and efficiency of subsequent processing.

[0138] In one embodiment, the corpus enhancement method further includes:

[0139] If the first similarity assessment result indicates that the rewritten question information is similar to the question information in the seed corpus, the step of inputting the question rewriting prompt instruction into the first large language model for question rewriting processing is performed again until the first similarity assessment result indicates that the rewritten question information is not similar to the question information in the seed corpus.

[0140] If the first similarity assessment result indicates that the rewritten question information is similar to the seed corpus question information, then the rewritten question information similar to the seed corpus question information will be output. The step of inputting the question rewriting prompt instruction into the first large language model for question rewriting processing is repeated to continue generating new rewritten question information until the new rewritten question information is no longer similar to the original seed corpus question information. That is, the question rewriting prompt instruction is input into the first large language model for question rewriting processing again to obtain the rewritten question information output by the first large language model, wherein the question rewriting prompt instruction includes the seed corpus question information. Subsequently, the rewritten question information and the seed corpus question information are input into the first similarity assessment model for similarity assessment processing to obtain the first similarity assessment result output by the first similarity assessment model of the rewritten question information and the seed corpus question information, until the first similarity assessment result indicates that the rewritten question information is not similar to the seed corpus question information.

[0141] S15, input the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtain the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model.

[0142] Here, the second similarity evaluation model refers to a model used to evaluate the similarity between two pieces of information. By inputting the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, the second similarity evaluation result between the sub-question information and the seed corpus question information, output by the second similarity evaluation model, can be obtained. The second similarity evaluation result can be qualitative (similar, dissimilar) or quantitative (similarity value).

[0143] In one embodiment, the second similarity evaluation model comprises multiple similarity evaluation sub-models. The step of inputting the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtaining the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model, includes:

[0144] The sub-question information and the seed corpus question information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the sub-question information and the seed corpus question information output by the at least one similarity evaluation sub-model is obtained;

[0145] Based on the at least one model similarity evaluation result, a second similarity evaluation result is determined between the sub-question information and the seed corpus question information.

[0146] Here, the second similarity evaluation model refers to a similarity evaluation system that includes multiple similarity evaluation sub-models that can be used to evaluate the similarity between two pieces of information. A similarity evaluation sub-model is a model that can be used to evaluate the similarity between two pieces of information. The model similarity evaluation result can be qualitative (similar or dissimilar) or quantitative (similarity value). Taking qualitative similarity or dissimilarity as an example, after obtaining multiple model similarity evaluation results for the sub-question information and the seed corpus question information output by multiple similarity evaluation sub-models, the majority of these model similarity evaluation results can be determined as the first similarity evaluation result. For example, if two out of three model similarity evaluation results indicate that the sub-question information and the seed corpus question information are dissimilar, then the second similarity evaluation result is dissimilar. Taking quantitative similarity value as an example, after obtaining multiple model similarity evaluation results for the sub-question information and the seed corpus question information output by multiple similarity evaluation sub-models, the calculated result can be determined as the second similarity evaluation result by taking the average or a weighted average.

[0147] For example, when the second similarity assessment model includes three similarity assessment sub-models, the sub-question information and the seed corpus question information can be input into the three similarity assessment sub-models respectively, and the three model similarity assessment results output by these three similarity assessment sub-models can be obtained. Then, based on these three model similarity assessment results, the second similarity assessment result between the sub-question information and the seed corpus question information is determined.

[0148] The second similarity evaluation model could be a second multi-model expert system. This second multi-model expert system uses the voting evaluation mechanism described above for similarity evaluation.

[0149] In one embodiment, the similarity evaluation sub-model includes any one of the following: a contrastive learning model based on text embeddings, or a large language model for similarity problem evaluation.

[0150] Among them, large language models (LLMs) for similarity problem evaluation include, but are not limited to, GPT-4 and Deepseek-V2-Chat.

[0151] A contrastive learning model based on text embeddings could be the SimCSE (Simple Contrastive Sentence Embedding) contrastive learning evaluation model. SimCSE is a contrastive learning model for learning sentence embeddings. Using the supervised version of SimCSE for evaluating rewritten question information primarily considers its direct use in vertical training, thus ensuring that the SimCSE model is more easily understood in relation to the semantic information of the vertical domain corpus during evaluation. During the inference phase, SimCSE can evaluate the similarity between sub-question information and seed corpus question information. After training, the SimCSE model performs inference and calculates cosine similarity, setting a similarity threshold of 0.5. The specific criteria are as follows: if the absolute value of the cosine similarity is less than the similarity threshold, the sub-question information and the seed corpus question information are considered similar. If the absolute value of the cosine similarity is greater than the similarity threshold, the sub-question information and the seed corpus question information are considered dissimilar.

[0152] GPT-4 can evaluate similar questions (GPT-4 is a language model released by OpenAI for the chatbot ChatGPT; GPT stands for Generative Pre-trained Transformer). The general capabilities of closed-source models are far superior to small-sized models like SimCSE. The specific process involves first designing question decomposition prompts (i.e., language model prompts). These prompts include rewriting the question information, providing question decomposition prompts, creating a question decomposition database table, and providing question decomposition sample examples. The model similarity evaluation results (similarity scores) output by GPT-4 can range from [0,1]. Here, 0 represents the most similar, and 1 represents the least similar. A similarity threshold of 0.5 can be set for discrimination. If the model similarity evaluation result between the sub-question information and the seed corpus question information is less than 0.5, the sub-question information is considered similar to the seed corpus question information. If the model similarity evaluation result between the sub-question information and the seed corpus question information is greater than 0.5, the sub-question information is considered dissimilar to the seed corpus question information.

[0153] Deepseek-V2-Chat has strong understanding and reasoning capabilities in the Chinese domain, and this model can be selected for evaluation of similar problems.

[0154] In one embodiment, the steps for training the contrastive learning model that generates text embeddings are the same as or similar to those described in S13, and will not be repeated here.

[0155] S16, if the second similarity evaluation result indicates that the sub-question information is not similar to the seed corpus question information, the sub-question information is retained.

[0156] The second similarity assessment result can be qualitative (similar or dissimilar) or quantitative (similarity value). When the second similarity assessment result is quantitative, for example, if the second similarity assessment result is less than the second similarity threshold, the second similarity assessment result can be determined as similar. If the second similarity assessment result is greater than the second similarity threshold, the second similarity assessment result can be determined as dissimilar.

[0157] If the second similarity assessment result indicates that the sub-question information is not similar to the seed corpus question information, then the sub-question information is retained.

[0158] This application obtains seed corpus question information, inputs a question rewriting prompt instruction into a first large language model for question rewriting processing, obtains rewritten question information output by the first large language model, wherein the question rewriting prompt instruction includes the seed corpus question information, inputs the rewritten question information and the seed corpus question information into a first similarity evaluation model for similarity evaluation processing, obtains a first similarity evaluation result between the rewritten question information and the seed corpus question information output by the first similarity evaluation model, and if the first similarity evaluation result indicates that the rewritten question information and the seed corpus question information... If the information is dissimilar, the problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, obtaining multiple sub-problem information output by the second large language model. The problem decomposition prompt instruction includes rewriting the problem information. The sub-problem information and the seed corpus problem information are input into the second similarity evaluation model for similarity evaluation processing, obtaining the second similarity evaluation result of the sub-problem information and the seed corpus problem information output by the second similarity evaluation model. If the second similarity evaluation result indicates that the sub-problem information and the seed corpus problem information are dissimilar, the sub-problem information is retained. The corpus enhancement scheme provided in this application can generate text corpus such as sub-problem information by using the first large language model for problem rewriting processing and the second large language model for problem decomposition processing. Furthermore, the text corpus such as sub-problem information can be generated in batches multiple times through iteration. Moreover, in this application's scheme, the similarity evaluation of information through the first and second similarity evaluation models allows for multiple settings and adjustments to the quality of the generated corpus, ensuring that the quality of the generated sub-problem information corpus meets the standards. By retaining sub-question information that is dissimilar to the seed corpus information, the independence of sub-question information from the seed corpus information is ensured. Specifically, sub-question information similar to the original seed corpus information is excluded to avoid duplication and redundancy. Sub-question information dissimilar to the original seed corpus information is retained to enrich and expand the diversity of questions. This application's solution significantly improves the quantity and quality of text corpora, greatly reduces the cost of manually generated corpora, and provides a high-quality corpus guarantee for subsequent language model training and optimization, as well as the construction and expansion of knowledge base corpora.

[0159] The corpus augmentation scheme provided in this application can be applied to the training of large-scale language models. It can be widely used in natural language processing tasks such as text classification, entity recognition, relation extraction, and question answering systems. Furthermore, the corpus augmentation scheme provided in this application can also be applied to the field of knowledge base construction, where it can collect, organize, and store domain knowledge to support various knowledge-based question answering and document-based question answering applications.

[0160] In the above application areas, the corpus enhancement solution provided in this application can achieve the following functions.

[0161] 1. Improve the quality and diversity of domain knowledge corpora: By generating richer and more diverse domain knowledge corpora, support the performance improvement of models in various application scenarios.

[0162] 2. Reduce the cost of manually building corpora: Automatically generating high-quality corpora can effectively reduce the cost and time of manually building corpora, thereby reducing labor costs.

[0163] 3. Improve model training performance: Increasing the amount of training data and improving data quality can help improve the performance of large language models on various tasks.

[0164] In one embodiment, the corpus enhancement method further includes:

[0165] The sub-problem information is used as the seed corpus problem information, and the corpus enhancement method is applied again.

[0166] Here, the retained sub-problem information can be used again for the next iteration through corpus enhancement methods. That is, the sub-problem information can be used as seed corpus problem information to repeat steps S11 to S16.

[0167] Here, the number of iterations for the retained sub-problem information to be further processed using corpus augmentation methods is not fixed, but flexibly adjusted based on the quality and quantity of the generated sub-problem information data. The iteration process is dynamic and adaptive. This iteration control mechanism based on data quality and quantity ensures the high quality and diversity of the final output sub-problem information data, while also optimizing the efficiency and effectiveness of the entire processing flow.

[0168] Figure 2 This is a schematic diagram of the structure of a corpus enhancement device provided in an embodiment of this application.

[0169] like Figure 2 As shown, the corpus enhancement device 2 includes:

[0170] Problem information acquisition module 21 is used to acquire problem information from seed corpus;

[0171] The problem rewriting processing module 22 is used to input the problem rewriting prompt instruction into the first large language model for problem rewriting processing, and obtain the rewriting problem information output by the first large language model, wherein the problem rewriting prompt instruction includes the seed corpus problem information;

[0172] The first similarity evaluation module 23 is used to input the rewriting problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtain the first similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the first similarity evaluation model;

[0173] The problem decomposition processing module 24 is used to input a problem decomposition prompt instruction into the second large language model for problem decomposition processing if the first similarity evaluation result indicates that the rewritten problem information is not similar to the seed corpus problem information, and to obtain multiple sub-problem information output by the second large language model, wherein the problem decomposition prompt instruction includes the rewritten problem information;

[0174] The second similarity evaluation module 25 is used to input the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtain the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model;

[0175] The sub-question information retention module 26 is used to retain the sub-question information if the second similarity evaluation result indicates that the sub-question information is not similar to the seed corpus question information.

[0176] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the first similarity evaluation model includes multiple similarity evaluation sub-models, and the first similarity evaluation module 23 is used for:

[0177] The rewriting problem information and the seed corpus problem information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the at least one similarity evaluation sub-model is obtained;

[0178] Based on the at least one model similarity evaluation result, a first similarity evaluation result is determined between the rewritten problem information and the seed corpus problem information.

[0179] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the second similarity evaluation model includes multiple similarity evaluation sub-models, and the second similarity evaluation module 25 is used for:

[0180] The sub-question information and the seed corpus question information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the sub-question information and the seed corpus question information output by the at least one similarity evaluation sub-model is obtained;

[0181] Based on the at least one model similarity evaluation result, a second similarity evaluation result is determined between the sub-question information and the seed corpus question information.

[0182] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiments, the similarity evaluation sub-model includes any one of the following: a contrastive learning model based on text embedding, or a large language model for similarity problem evaluation.

[0183] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the corpus enhancement device 2 further includes:

[0184] The question rewriting module is used to perform the step of inputting the question rewriting prompt instruction into the first language model for question rewriting processing again if the first similarity evaluation result indicates that the rewritten question information is similar to the question information in the seed corpus, until the first similarity evaluation result indicates that the rewritten question information is not similar to the question information in the seed corpus.

[0185] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the problem rewriting prompt instruction further includes problem rewriting prompt words, a problem rewriting database table, and a problem rewriting sample example. The problem rewriting processing module 22 is used for:

[0186] The question rewriting prompt instruction is input into the first large language model for question rewriting processing, and the rewritten question information output by the first large language model is obtained. The question rewriting prompt instruction includes the seed corpus question information, the question rewriting prompt words, the question rewriting database table, and the question rewriting sample examples.

[0187] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the problem breakdown prompt instruction further includes problem breakdown prompt words, a problem breakdown database table, and a problem breakdown sample example. The problem breakdown processing module 24 is used for:

[0188] The problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained. The problem decomposition prompt instruction includes the rewritten problem information, the problem decomposition prompt words, the problem decomposition database table, and the problem decomposition sample example.

[0189] Another embodiment of the present invention discloses a corpus enhancement device 2. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the corpus enhancement device 2 further includes:

[0190] The re-iteration module is used to take the sub-problem information as the seed corpus problem information and perform the corpus enhancement method again.

[0191] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0192] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0193] This application also provides an electronic device, such as... Figure 3 As shown, the electronic device 3 includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 executes the computer program 32 to implement the steps in any of the above-described method embodiments.

[0194] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0195] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0196] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0197] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0198] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0199] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0200] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0201] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A corpus enhancement method, characterized in that, include: Information on obtaining seed corpus; The problem rewriting prompt instruction is input into the first large language model for problem rewriting processing, and the rewritten problem information output by the first large language model is obtained, wherein the problem rewriting prompt instruction includes the problem information of the seed corpus; The rewriting problem information and the seed corpus problem information are input into a first similarity evaluation model for similarity evaluation processing, and the first similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the first similarity evaluation model is obtained. If the first similarity evaluation result indicates that the rewritten problem information is not similar to the problem information in the seed corpus, the problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained, wherein the problem decomposition prompt instruction includes the rewritten problem information; The sub-question information and the seed corpus question information are input into the second similarity evaluation model for similarity evaluation processing, and the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model is obtained; If the second similarity assessment result indicates that the sub-question information is not similar to the seed corpus question information, the sub-question information is retained.

2. The corpus enhancement method as described in claim 1, characterized in that, The first similarity evaluation model includes multiple similarity evaluation sub-models. The step of inputting the rewritten problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtaining the first similarity evaluation result of the rewritten problem information and the seed corpus problem information output by the first similarity evaluation model, includes: The rewriting problem information and the seed corpus problem information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the at least one similarity evaluation sub-model is obtained; Based on the at least one model similarity evaluation result, a first similarity evaluation result is determined between the rewritten problem information and the seed corpus problem information.

3. The corpus enhancement method as described in claim 1, characterized in that, The second similarity evaluation model includes multiple similarity evaluation sub-models. The step of inputting the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtaining the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model, includes: The sub-question information and the seed corpus question information are respectively input into at least one similarity evaluation sub-model, and at least one model similarity evaluation result of the sub-question information and the seed corpus question information output by the at least one similarity evaluation sub-model is obtained; Based on the at least one model similarity evaluation result, a second similarity evaluation result is determined between the sub-question information and the seed corpus question information.

4. The corpus enhancement method as described in claim 2 or 3, characterized in that, The similarity evaluation sub-model includes any one of the following: a contrastive learning model based on text embedding, or a large language model for similarity problem evaluation.

5. The corpus enhancement method as described in claim 1, characterized in that, The method further includes: If the first similarity assessment result indicates that the rewritten question information is similar to the question information in the seed corpus, the step of inputting the question rewriting prompt instruction into the first large language model for question rewriting processing is performed again until the first similarity assessment result indicates that the rewritten question information is not similar to the question information in the seed corpus.

6. The corpus enhancement method as described in claim 1, characterized in that, The question rewriting prompt instruction also includes question rewriting prompt words, a question rewriting database table, and question rewriting sample examples. The step of inputting the question rewriting prompt instruction into the first large language model for question rewriting processing and obtaining the rewriting question information output by the first large language model includes: The question rewriting prompt instruction is input into the first large language model for question rewriting processing, and the rewritten question information output by the first large language model is obtained. The question rewriting prompt instruction includes the seed corpus question information, the question rewriting prompt words, the question rewriting database table, and the question rewriting sample examples.

7. The corpus enhancement method as described in claim 1 or 6, characterized in that, The problem decomposition prompt instruction also includes problem decomposition prompt words, a problem decomposition database table, and problem decomposition sample examples. The step of inputting the problem decomposition prompt instruction into the second large language model for problem decomposition processing, and obtaining multiple sub-problem information output by the second large language model, including: The problem decomposition prompt instruction is input into the second large language model for problem decomposition processing, and multiple sub-problem information output by the second large language model is obtained. The problem decomposition prompt instruction includes the rewritten problem information, the problem decomposition prompt words, the problem decomposition database table, and the problem decomposition sample example.

8. The corpus enhancement method as described in claim 1, characterized in that, The method further includes: The sub-problem information is used as the seed corpus problem information, and the corpus enhancement method is applied again.

9. A corpus enhancement device, characterized in that, include: The problem information acquisition module is used to acquire problem information from the seed corpus. The problem rewriting processing module is used to input the problem rewriting prompt instruction into the first large language model for problem rewriting processing, and obtain the rewriting problem information output by the first large language model, wherein the problem rewriting prompt instruction includes the problem information of the seed corpus; The first similarity evaluation module is used to input the rewriting problem information and the seed corpus problem information into the first similarity evaluation model for similarity evaluation processing, and obtain the first similarity evaluation result of the rewriting problem information and the seed corpus problem information output by the first similarity evaluation model; The problem decomposition processing module is used to input a problem decomposition prompt instruction into the second large language model for problem decomposition processing if the first similarity evaluation result indicates that the rewritten problem information is not similar to the problem information in the seed corpus, and to obtain multiple sub-problem information output by the second large language model, wherein the problem decomposition prompt instruction includes the rewritten problem information; The second similarity evaluation module is used to input the sub-question information and the seed corpus question information into the second similarity evaluation model for similarity evaluation processing, and obtain the second similarity evaluation result of the sub-question information and the seed corpus question information output by the second similarity evaluation model; The sub-question information retention module is used to retain the sub-question information if the second similarity evaluation result indicates that the sub-question information is not similar to the seed corpus question information.

10. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1-8.

Images