File generation method, device, electronic device, storage medium and program

Abstract

To provide a file generation method, a device, an electronic device, a storage medium, and a program.SOLUTION: A method comprises the steps of: respectively inputting M1 first-type files into a first model, and respectively outputting second-type files corresponding to the first-type files from the first model; determining a plurality of file pairs according to the output result, wherein each file pair includes one first-type file and one second-type file corresponding to the first-type file; adjusting a second model by using the plurality of file pairs; and respectively inputting M2 first-type files into the adjusted second model, and respectively outputting second-type files corresponding to the first-type files from the adjusted second model, where M1 and M2 are positive integers. The present disclosure can save cost and improve the number and diversity of the generated files.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence, neural network models, large-scale language models, and the like. [Background technology]

[0002] Neural network models often need to be trained with large-scale, high-quality training samples before they can be applied, and the quality and scale of the training samples are very important to the representation and application effect of the neural network model. Summary of the Invention [Problem to be solved by the invention]

[0003] The present disclosure provides a file generation method, an apparatus, an electronic device, a storage medium, and a program. [Means for solving the problem]

[0004] In one aspect of the present disclosure, there is provided a file generation method, the method comprising: inputting M first type files into a first model, respectively, and outputting a second type file corresponding to each first type file from the first model; determining a plurality of file pairs based on the output result, each file pair including a first type file and a corresponding second type file of the first type file; using the plurality of file pairs to refine the second model; inputting M2 first type files into a respective adjusted second model, and outputting a corresponding second type file for each first type file from the adjusted second model, where M1 and M2 are positive integers.

[0005] In another aspect of the present disclosure, there is provided a file generation device, the device comprising: a first input module for inputting each of the M first type files into a first model and outputting a corresponding second type file from the first model for each first type file; A determination module for determining a plurality of file pairs based on the output result, each file pair including a first type file and a corresponding second type file of the first type file; a tuning module for tuning the second model using the plurality of file pairs; a second input module for inputting M2 first type files into the adjusted second model, respectively, and outputting a corresponding second type file for each first type file from the adjusted second model, where M1, M2 are positive integers.

[0006] In another aspect of the present disclosure, an electronic device is provided, the device comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0007] Another aspect of the present disclosure provides a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform any one of the methods in the embodiments of the present disclosure.

[0008] Another aspect of the present disclosure provides a program that, when executed by a processor, performs any of the methods of the embodiments of the present disclosure.

[0009] The present disclosure proposes a file generation method, and the files generated by the method can be used as training samples for neural network models. Specifically, a first model is used to generate a plurality of file pairs, each of which includes a first type file and a second type file corresponding to the first type file, and these file pairs are used to adjust a second model, and then the adjusted second model is used to generate a plurality of corresponding second type files based on the plurality of first type files, and the second type files can be used as training samples for other neural network models. In this way, the training samples generated by the first model can be used to train a second model, and the training samples for other neural network models can be generated from the trained second model. In this way, a new training sample generation scheme is proposed, which can avoid manual labeling and save costs. In addition, by utilizing the generation ability of the first model, the generated files can cover various fields and scenarios, and the number and diversity of generated files can be increased.

[0010] It should be understood that the contents described herein are not intended to describe key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure, other features of the present disclosure are facilitated by the following specification.

[0011] The accompanying drawings are for a better understanding of the solutions of the present disclosure, but are not intended to limit the present disclosure. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram illustrating an application scenario of an embodiment of the present disclosure. [Diagram 2] 1 is a flowchart illustrating an implementation of a file generation method according to an embodiment of the present disclosure. [Diagram 3]FIG. 2 is a schematic diagram illustrating a first stage of a file generation method according to an embodiment of the present disclosure. [Figure 4] FIG. 2 is a schematic diagram illustrating a second stage of a file generation method according to an embodiment of the present disclosure. [Diagram 5] FIG. 11 is a schematic diagram illustrating a third stage of a file generation method according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic diagram showing a configuration of a file generation device 600 according to an embodiment of the present disclosure. [Figure 7] 7 is a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. These drawings include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely illustrative. Therefore, it should be understood that those skilled in the art can make various changes and modifications to the embodiments described herein without departing from the scope of the present disclosure. Similarly, descriptions of well-known features and structures are omitted in the following description for clarity and conciseness.

[0014] The term "and / or" in the present disclosure indicates that three types of relationships may exist, for example, A and / or B are three types of situations: A exists alone, A and B exist simultaneously, and B exists alone. The term "at least one" in the present disclosure indicates any combination of at least two of any one or more of a plurality, for example, at least one of A, B, and C may select any one element or multiple elements from the set consisting of A, B, and C. The terms "first" and "second" in the present disclosure are used to refer to and distinguish multiple similar technical terms, and are not meant to limit the order or to limit only two, for example, the first feature and the second feature mean the existence of two types / two features, and the first feature may be one or more, and the second feature may be one or more.

[0015] Neural network models often need to be trained with large-scale, high-quality training samples before they can be applied, and the quality and scale of the training samples are very important to the representation and application effect of neural network models. Take large language models (LLMs) as an example. When large language models are applied to vertical fields, they often need to be fine-tuned supervisedly by large-scale, high-quality dialogue corpora (or dialogue corpora, dialogue-level corpora, etc.). The quality and scale of the corpus set the upper limit for the application of large language models to vertical fields. The quality of the dialogue corpus refers to the accuracy, consistency, and rationality of the dialogue content in the dialogue corpus, which is very important to the representation and application effect of the model.

[0016] Accuracy means that dialogue content must be verified and reviewed against reliable sources to ensure the authenticity and reliability of the information, and also includes understanding and correctly applying specific subject and terminology to avoid misleading or sending the wrong message.

[0017] Consistency means that the model maintains consistency in its answers to the same questions or scenarios across different dialogues. To ensure the consistency and reliability of the model's output, the creators or labelers of the dialogue corpus need to have a consistent understanding and standards.

[0018] The larger the corpus, the richer the model's knowledge base and response capabilities in a particular domain. Leveraging a large corpus can help the model better understand and answer questions in a particular domain, providing more accurate and comprehensive information.

[0019] Thus, the quality and size of the dialogue corpus are crucial for the application of large-scale language models in vertical fields. Only by fine-tuning the model using a large dialogue corpus can the model better adapt to the demands of a specific field and provide high-quality and accurate answers and solutions. Therefore, building and maintaining a high-quality, large-scale dialogue corpus library is crucial to increasing the applicability of large-scale language models in vertical fields.

[0020] There are various implementations in the prior art to obtain files that can be used as model training samples. Below are some common implementations:

[0021] (1) Manual labeling: This is currently one of the most common methods, in which files are labeled by professional labelers to ensure accuracy and consistency with the training samples. This method is costly and requires a lot of manpower and time.

[0022] (2) Involvement of vertical experts: Invite experts in vertical fields to participate in file labeling. To ensure the accuracy and rationality of files as training samples in vertical fields, these experts can provide their domain expertise. Although this method can improve the quality of the corpus, it requires close cooperation with experts and the rational arrangement of their time and resources, which may limit the speed and scale of corpus generation and increase the cost of communication and coordination with experts.

[0023] (3) In-field data collection: Use data resources within a vertical field, such as internal conversation records and chat records. These data can be converted into high-quality training samples by organizing and processing them. This method can fully utilize knowledge and resources within a field, but it has the problem of limited data scale. Especially in emerging fields or fields that handle confidential information, it may be difficult to access files in specific fields, which may limit the size and diversity of the training sample library.

[0024] (4) Data alignment and transfer learning: Using training sample libraries from traditional general-purpose fields, we transform them into training samples suitable for vertical fields through data alignment and transfer learning techniques. This method can reduce the reliance on corpora in new fields, but still requires human involvement and domain knowledge.

[0025] As described above, existing methods for generating training samples have problems such as the need for manual labeling, high cost, limited scale and diversity, etc. Although an example of generating training samples for training a large-scale language model has been described above, in the conventional technology, when generating training samples for training neural network models of other functions such as image recognition models and classification models, manual labeling is also required, which has problems such as high cost, limited scale and diversity, etc.

[0026] In order to solve the above problems, the embodiment of the present disclosure proposes a file generation method for generating a file that can be used as a training sample of a neural network model. FIG. 1 is a schematic diagram showing an application scenario of the embodiment of the present disclosure. As shown in FIG. 1, the schematic diagram of the application scenario of the embodiment of the present disclosure may include, but is not limited to, a file generation device 110 and a model training device 120, and the file generation device 110 and the model training device 120 can communicate with each other via any type of wired or wireless network. Specifically, the file generation device 110 can generate a file that can be used as a training sample of a neural network model, and can transmit the file to the model training device 120. The model training device 120 receives the file and uses the file to train or fine-tune the neural network model. Here, the file generation device 110 and the model training device 120 proposed in the embodiment of the present disclosure may include an electronic device or a server. Further, embodiments of the present disclosure are not specifically limited to the number of file generation devices 110 or model training devices 120, and for example, a schematic diagram of an application scenario of an embodiment of the present disclosure may include one or more file generation devices 110 and / or model training devices 120.

[0027] FIG. 2 is an implementation flowchart of a file generation method according to an embodiment of the present disclosure, which includes:

[0028] In S210, M1 first type files are input into a first model, and a second type file corresponding to each of the first type files is output from the first model.

[0029] At S220, a plurality of file pairs are determined based on the output result, where each file pair includes a first type file and a corresponding second type file of the first type file.

[0030] At S230, the second model is trained using the plurality of file pairs.

[0031] In S240, M2 first type files are input to a respective adjusted second model, and a corresponding second type file of each first type file is output from the adjusted second model.

[0032] Here, M1 and M2 are positive integers.

[0033] In some embodiments, the first model can include an existing public large-scale language model, where the first model is large in scale and can serve a variety of different users, and the second model can include an efficient, low-cost language model, where the second model is a proprietary model that is small in scale and low in cost and can be used for a particular vertical.

[0034] The embodiment of the present disclosure combines a relatively large language model with an efficient and low-cost language model to generate files, which can be used as training samples or corpora for other models. The present solution can avoid manual labeling and save costs. The present solution can also utilize the generation capabilities of the large language model, generate files that can cover a variety of fields and scenarios, and increase the number and diversity of generated files.

[0035] When using an existing large-scale language model to generate files to be used as training samples or corpora, one option is to use the large-scale language model to generate all training samples or corpora. However, since the cost of using a large-scale language model is high, a large number of large-scale language models must be used to generate a huge number of training samples or corpora, and this method is therefore expensive. In addition, data must be input into the large-scale language model during the generation process, but the large-scale language model provides services to the public, so the security of this data cannot be guaranteed.

[0036] The file generation method proposed in the embodiment of the present disclosure uses an existing large-scale language model and an efficient and low-cost unique language model, inputting some first-type files (M1 first-type files in the above solution) into the large-scale language model respectively, and outputting M1 corresponding second-type files from the large-scale language model respectively, where one first-type file and one corresponding second-type file form one file pair, resulting in a total of M1 file pairs (for convenience of explanation, the data cleansing process is not considered here. If the cleansing process is considered, the number of file pairs will be less than M1). Here, the first-type files may be files of a specific vertical field, and in this way, the generated M1 file pairs also correspond to the specific vertical field.

[0037] The M1 file pairs can be used to train a second model, for example, the second model is a pre-trained language model, and the second model is a unique model. The embodiment of the present disclosure trains the second model using the file pairs generated by the first model (for example, performing supervised fine-tuning), and this training process can improve the capability of the second model in the specific vertical field. Then, a large number of first type files (for example, M2 first type files in the above solution) are input into the trained second model, respectively, and the corresponding M2 second type files are output from the large-scale language model. The M2 second type files are used as training samples or corpora for other models to train the capability of other models in the specific vertical field.

[0038] In some embodiments, M2 is greater than M1. For example, M2 is much greater than M1 (M2>>M1). In this way, a small amount of first type files can be input into a first model (for example, an existing large-scale language model), and a file pair is constructed using the output result of the existing large-scale language model, and the file pair is used to adjust a unique language model, and then a large amount of first type files is input into the adjusted unique language model, and a large amount of second type files is output from the adjusted unique language model as a training sample or corpus for other models. In this way, the use of existing large-scale language models can be reduced as much as possible, and costs can be reduced.

[0039] In some embodiments, the M2 first type files input to the adjusted unique language model (i.e., the second model) may include files that require security protection, and the M1 first type files input to the existing large-scale language model (i.e., the first model) may be files with low security requirements. Therefore, according to the present solution, the files that require security protection can be converted locally using the unique language model without being leaked to the outside, so that data security can be effectively protected.

[0040] In this way, the file generation method proposed in the embodiments of the present disclosure can simultaneously achieve the effects of cost reduction and data security protection by combining existing large-scale language models with unique language models.

[0041] The first type file and the second type file proposed in the embodiment of the present disclosure may be in various formats, for example, the first type file is a text corpus and the second type file is a dialogue corpus. Alternatively, the first type file is a dialogue corpus and the second type file is a text corpus. Alternatively, the first type file is text (e.g., text or dialogue) and the second type file is an image, video, animation, etc. Alternatively, the first type file is an image, video, animation, etc. and the second type file is text (e.g., text or dialogue). Alternatively, the first type file is text (e.g., text) and the second type file is a summary of the text corresponding to the text. Alternatively, the first type file is a summary of the text and the second type file is text (e.g., text) corresponding to the summary. In this way, the types may be various. In the embodiments of the present disclosure, the specific types of the first type file and the second type file are not limited, and the first model and the second model in the embodiments of the present disclosure may be other types of models, such as a Multimodal Large Language Model (MLLM), which can generate and understand richer natural language by combining information of various modes such as text, images, and videos.

[0042] In the following, we will take as an example that the first type of file is a text corpus and the second type of file is a dialogue corpus. A text corpus can include content such as books, articles, blogs, and research reports. A text is a fully-structured form of linguistic expression, and is a complete piece of writing or work that is organically formed by assembling various sentences and paragraphs through a unified theme and logical relationships. A dialogue corpus includes conversational content between two or more people, and can include multiple dialogue questions and answers to each of the dialogue questions.

[0043] The file generation method proposed in the embodiment of the present disclosure includes the following steps.

[0044] In the first stage, a text corpus is collected and converted into a dialogue corpus.

[0045] FIG. 3 is a schematic diagram illustrating a first stage of a file generation method according to an embodiment of the present disclosure. In the first stage, a small amount of text corpora (denoted as P in this embodiment) of a specific vertical field is collected. These text corpora include books, blogs, research reports, etc. Using an existing large-scale language model, a transformation is performed on the text corpus P in combination with a specific prompt to obtain a corresponding dialogue corpus (denoted as D in this embodiment). Here, the prompt can include the content of the text corpus P. Through this step, a number of file pairs (or input / output pairs) can be obtained, each file pair including one text corpus and its corresponding dialogue corpus, denoted as (text corpus P, dialogue corpus D). The text corpus and dialogue corpus in this stage can correspond to a specific vertical field. For example, a text corpus of a specific vertical field is collected, and accordingly, the dialogue corpus generated by the existing large-scale language model based on the text corpus can also correspond to a specific vertical field.

[0046] The second stage involves fine-tuning the unique language model.

[0047] FIG. 4 is a schematic diagram showing a second stage of a file generation method according to an embodiment of the present disclosure. In this stage, the M1 file pairs (or input / output pairs) collected in the first stage are used to perform supervised fine-tuning on the unique language model. For example, the unique language model is fine-tuned using a backpropagation algorithm. Specifically, the sentence corpus in the file pair is taken as a training sample, and the corresponding dialogue corpus is taken as the label of the training sample. The training sample is input into the unique language model, and the corresponding dialogue corpus is output from the unique language model, and the output dialogue corpus is compared with the label of the training sample, and the parameters of the unique language model are adjusted according to the comparison result. In this way, supervised fine-tuning of the unique language model can be realized. The fine-tuning process can improve the dialogue generation ability of the unique language model in a specific vertical field, which is the corresponding field of the file pair. The language model in this stage is an efficient, low-cost, small-scale model.

[0048] The third step is to generate a large-scale, high-quality dialogue corpus.

[0049] 5 is a schematic diagram showing the third stage of the file generation method according to an embodiment of the present disclosure. In this stage, a larger text corpus, for example a text corpus with a number M2 (M2 is much larger than M1), is used to input the fine-tuned efficient and low-cost small-scale model. In this way, a large-scale and high-quality dialogue corpus can be output from the efficient and low-cost small-scale model to meet the demands of a specific vertical field. In this stage, the text corpus employed can be collected through routes such as books, research reports, blogs, etc.

[0050] To realize a customized reply style of a dialogue corpus, an embodiment of the present disclosure can input a small amount of sentence corpus with a specific expression style (e.g., a specific manner of speaking, tone, expression method, etc.) to an existing large-scale language model in the first stage described above. By introducing data with these specific expression styles, a dialogue corpus that imitates the specific expression style can be generated in the existing large-scale language model.

[0051] In some embodiments, an example of a prompt that may be input to an existing large-scale language model in the first stage above is:

[0052] "A dialogue corpus is generated based on the following text corpus, where both parties in the dialogue are a teacher and a student.

[0053] Example 1 input: "Dream of the Red Chamber" is one of the four great classical novels of China. It has a total of 120 chapters, the first 80 of which were written by the Qing Dynasty writer Cao Xueqin, and the last 40 were written anonymously by the author, and are generally believed to have been compiled by Cheng Weiyuan and Gao E. The novel is set against the backdrop of the rise and fall of the four great families of Jia, Shi, Wang, and Xue, and is a historical and poetic masterpiece that depicts the various aspects of life, with the tragic love and marriage of Jia Baoyu, Lin Daiyu, and Xue Baochai as its main theme.

[0054] output: Student A: Teacher, I am reading "Dream of the Red Chamber", but the content is so rich that I don't know where to start. Teacher: Dream of the Red Chamber is certainly rich in content, but it's best to start reading with its main theme, the tragic love and marriage story between Jia Baoyu, Lin Daiyu, and Xue Baochai. Student A: I see. So who is the author of this book? Teacher: It is generally believed that the first 80 chapters of Dream of the Red Chamber were written by Cao Xueqin, a famous writer in the Qing Dynasty, and the last 40 chapters were written anonymously and later compiled and published by Cheng Weiyuan and Gao Ye. Student A: So what's the background to this book? Teacher: "Dream of the Red Chamber" is set against the backdrop of the rise and fall of the four great families: Jia, Shi, Wang, and Xue.

[0055] Here, the text corpus that needs to be converted is:

[0056] input: "Romance of the Three Kingdoms" is a full-length historical novel written by the author Luo Guanzhong, who lived in the late Yuan and early Ming dynasties, and is one of the four great classical novels of China. "Romance of the Three Kingdoms" depicts the historical storm of nearly a hundred years from the end of the Eastern Han Dynasty to the beginning of the Western Jin Dynasty, focusing on war and the chaotic battles between the many warlords at the end of the Eastern Han Dynasty, and the political and military struggles between the three kingdoms of Wei, Shu, and Wu.

[0057] Based on the above prompt, an existing large-scale language model can generate a dialogue corpus that corresponds to the "text corpus that needs conversion" contained in the prompt.

[0058] By inputting a small amount of the above type of prompt, the existing large-scale language model can generate multiple dialogue corpora, which can generate multiple file pairs, each file pair including a sentence corpus and its corresponding dialogue corpus, and the multiple file pairs are used to fine-tune the unique language model in the second stage. In order to improve the quality of the file pairs generated in the first stage and thereby improve the effect of fine-tuning the unique language model, the embodiments of the present disclosure may adopt at least the following methods.

[0059] Method 1 In one example, a text corpus is input to a first model (e.g., an existing large-scale language model), and a plurality of dialogue questions related to the text corpus are output from the first model; A corresponding dialogue corpus is output from the first model based on the corpus and the plurality of dialogue questions, where the dialogue corpus includes the plurality of dialogue questions and answers to each of the dialogue questions.

[0060] In this way, the first model (e.g., an existing large-scale language model) can be guided to generate the dialogue corpus step by step, i.e., the first step generates dialogue questions in the dialogue corpus, and the second step generates answers to each dialogue question, thereby forming a complete dialogue corpus. In this manner, the large-scale language model can be guided in a more detailed manner to complete the work of each step, and the work to be completed in each step is relatively simple, so that the effect of the large-scale language model can be improved, thereby improving the quality of the file pair, and thus improving the effect of fine-tuning the second model (e.g., an original language model).

[0061] For example, one typical prompt content to enter into the first model is:

[0062] “Ask several dialogue questions for the following text corpus. ..."

[0063] The prompt is input to a large-scale language model, and multiple dialogue questions for this text corpus are output from the large-scale language model. After that, the following prompt is input to the large-scale language model:

[0064] "Please generate a dialogue corpus for the above text corpus and the above dialogue questions."

[0065] In the above example, two prompts are input to the first model in succession. Here, the first prompt includes the content of a text corpus, and requests the first model to output a number of dialogue questions corresponding to the text corpus. The second prompt requests the first model to output the aforementioned output and then output a dialogue corpus corresponding to the text corpus. By outputting two prompts in succession, the guidance to the first model is subdivided, improving the effectiveness of the dialogue corpus generation by the first model.

[0066] Method 2 A prompt is generated for a text corpus, the prompt having the content of the text corpus and interlocutor identity features, and the interlocutor identity features are used to cause a first model (e.g., an existing large-scale language model) to output a dialogue corpus that satisfies the interlocutor identity features.

[0067] For example, in the above example prompt, "both parties in the dialogue are a teacher and a student" is an identity feature of the interlocutors, and including the identity feature of the interlocutors in the prompt allows the large-scale language model to output a dialogue corpus that matches the identity feature. Using the file pairs generated by the dialogue corpus that matches such identity features, the unique language model can be fine-tuned in the second stage to generate a dialogue corpus that matches the identity features of the interlocutors in the unique language model.

[0068] Method 3 Using a prompt optimization method, the prompt is optimized, and the optimized prompt is input to a first model (e.g., an existing large-scale language model).

[0069] An embodiment of the present disclosure can enhance the effectiveness of the first model in the first stage and optimize the output results of the first stage by using Chain-of-Thought (CoT), In-Context Learning (ICL), a self-optimization method, and an optimization prompt.

[0070] Thought chaining is a technique for designing prompts, i.e., prompts that contain intermediate steps of reasoning (intermediate thoughts) in addition to task-bearing inputs and outputs. Thought chaining can significantly improve the capabilities of LLMs.

[0071] ICL is a method for training a large-scale language model on a specific task using a small number of labeling samples. The core idea of ​​this method is to form a prompt template by designing task-related instructions, and then guide the model to generate prediction results on new test data by using a small number of labeling samples as prompts.

[0072] The core idea of ​​self-optimization is that the large-scale language model optimizes prompts by itself. Specifically, the large-scale language model records past repetition history, optimizes goals, and compiles rules by itself to gradually repeat prompts.

[0073] Some of the above techniques can realize prompt optimization, and by using the optimized prompt, the performance of the first model can be improved, i.e., the effectiveness of dialogue corpus generation by the first model can be improved.

[0074] Method 4 The text corpus input to the first model (e.g., an existing large-scale language model) has a preset expression style, and the prompt input to the first model includes the identity features of the interlocutor. After the first model outputs the corresponding dialogue corpus, the dialogue corpus that does not match the preset expression style and the dialogue corpus that does not match the identity features are removed from the output result of the first model to obtain the remaining output result; Determine multiple file pairs based on the remaining output results.

[0075] In this process, the output of the first model can be cleansed to remove low-quality file pairs.

[0076] In some embodiments, the first model can be used to determine the expression style and / or identity features of the generated dialogue corpus. For example, the dialogue corpus generated by the first model can be re-input into the first model, and the expression style and / or identity features of the dialogue corpus can be determined by the first model. Alternatively, another neural network model can be used to determine the expression style and / or identity features of the dialogue corpus output by the first model.

[0077] By performing cleansing on the output results of the first model in the first stage, high-quality file pairs can be obtained. In the second stage, these high-quality file pairs are used to train the second model, thereby improving the performance of the second model.

[0078] The above multiple methods can improve the quality of the file pairs generated by the first model in the first stage, and in the subsequent process, the second model can be trained using higher quality file pairs to improve the performance of the trained second model, thereby improving the quality of the files generated by the trained second model.

[0079] As can be seen from the above, the file generation method proposed in the embodiment of the present disclosure can use a small amount of sample data and an existing large-scale language model to generate a small amount of training samples for a specific vertical field, and the training samples can be used to fine-tune a unique language model to improve the ability of the unique language model in the specific vertical field; and a large amount of text corpus is input into the adjusted language model to generate a large amount of dialogue corpus from the adjusted language model. The generation of large-scale and high-quality dialogue corpora can better meet the needs of a specific vertical field. These dialogue corpora can cover expertise, problem solving, common scenarios, etc. in a specific field, making the application of related products in the field more comprehensive and practical.

[0080] In addition, the file generation method proposed in the embodiments of the present disclosure can effectively protect data privacy by converting the sensitive text corpus in a local unique language model and fine-tuning it with an existing large-scale language model. Compared with the direct use of an existing large-scale language model, the solution of the present disclosure can save costs and reduce the expenses of using a large-scale language model.

[0081] The dialogue corpus generated by the file generation method proposed in the embodiments of the present disclosure can be used to build a unique large-scale language model, which can then be fine-tuned and optimized in specific vertical fields, thereby providing the unique large-scale language model with unique technical advantages and differentiating capabilities.

[0082] In an embodiment of the present disclosure, a file generating device is proposed. FIG. 6 is a schematic diagram showing a configuration of a file generating device 600 according to an embodiment of the present disclosure, which includes: a first input module 601 for inputting M1 first type files into a first model, respectively, and outputting a second type file corresponding to each first type file from the first model; A determination module 602 for determining a plurality of file pairs based on the output result, each file pair including a first type file and a corresponding second type file of the first type file; an adjustment module 603 for adjusting the second model using the plurality of file pairs; and a second input module 604 for inputting M2 first type files into the adjusted second model, respectively, and outputting a corresponding second type file for each first type file from the adjusted second model, where M1, M2 are positive integers.

[0083] In some embodiments, M2 is greater than M1.

[0084] In some embodiments, the M2 files of the first type include files that require security protection.

[0085] In some embodiments, the first type of file comprises a text corpus and the second type of file comprises a dialogue corpus.

[0086] In some embodiments, the first input module 601 includes: inputting a text corpus into a first model and outputting a plurality of dialogue questions related to the text corpus from the first model; The method is used to output a corresponding dialogue corpus from the first model based on the sentence corpus and a plurality of dialogue questions, the dialogue corpus including a plurality of dialogue questions and answers to each of the dialogue questions.

[0087] In some embodiments, the first input module 601 includes: A prompt is generated for each sentence corpus, and the prompt is input to the first model, where the prompt has a content of the sentence corpus and an identity characteristic of a conversational party, and the identity characteristic of the conversational party is used to cause the first model to output a conversational corpus that satisfies the identity characteristic of the conversational party.

[0088] In some embodiments, the first input module 601 includes: optimizing a prompt using a prompt optimization method; The optimized prompts are input to a first model.

[0089] In some embodiments, the text corpus has a predefined presentation style.

[0090] In some embodiments, the determination module 602 may: removing a dialogue corpus that does not match a preset expression style and a dialogue corpus that does not match an identity feature of the interlocutor from the output result of the first model to obtain a remaining output result; and determining a number of file pairs based on the remaining output results.

[0091] The specific functions and exemplary descriptions of each module and sub-module of the apparatus according to the embodiments of the present disclosure may refer to the relevant descriptions of the corresponding steps in the above-mentioned method embodiments, and will not be repeated here.

[0092] In the technical solution disclosed herein, the acquisition, storage, and application of users' personal information complies with the provisions of relevant laws and regulations and does not violate public order and morals.

[0093] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a non-transitory computer-readable storage medium, and a program product.

[0094] 7 is a block diagram of an electronic device 700 for implementing an embodiment of the present disclosure. The electronic device refers to any type of digital computer, such as a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device also refers to any type of mobile device, such as a personal digital assistant, a cellular phone, an intelligent phone, a wearable device, and other similar computing devices. The components, their connections, and functions described in this disclosure are merely exemplary and are not intended to limit the implementation of what is described and specified in this disclosure.

[0095] 7, device 700 includes a computing unit 701 that can perform various appropriate operations and processes based on computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 can further store various programs and data required for the operation of device 700. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0096] The components of the device 700 are connected to an I / O interface 705, which includes an input unit 706 such as a keyboard and a mouse, an output unit 707 such as various displays and speakers, a storage unit 708 such as a magnetic disk or an optical disk, and a communication unit 709 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various carrier networks.

[0097] The computing unit 701 may be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes each of the methods and processes described above, such as the file generation method. For example, in some embodiments, the file generation method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, some or all of the computer program can be loaded and / or installed in the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, it can perform one or more steps of the file generation method described above. Additionally, in other embodiments, the computing unit 701 can be configured to execute the file generation method in any other suitable manner (e.g., firmware).

[0098] Various embodiments of the systems or techniques described in this disclosure may be implemented as digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard packages (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. Each of these embodiments may include implementation by one or more computer programs executed and / or interpreted by a programmable system including at least one programmable processor, which may be a special purpose or general purpose programmable processor capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] The program code for carrying out the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programming data processing apparatus, such that when the program code is executed by the processor or controller, it can perform the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a separate soft encapsulation, or entirely on a remote machine or server.

[0100] In the description of this disclosure, a machine-readable medium may be a tangible medium, which contains or stores a program for use by or in conjunction with an instruction execution system, device or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Further examples of machine-readable storage media include one or more wired electrical connections, a portable computer disk cartridge, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any combination of the foregoing.

[0101] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer that can include a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, etc.) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball, etc.) for the user to provide input to the computer. Other types of devices can also be used to provide for interaction with a user, for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, haptic feedback, etc.) and input from the user can be received in any form (e.g., acoustic input, voice input, tactile input, etc.).

[0102] The systems and techniques described herein may be implemented in a computing system that includes a background component (e.g., as a data server), or a middleware component (e.g., an application server), or a front-end component (e.g., a user computer having a graphical user interface or a network browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such background, middleware, or front-end components. The components of the system may be connected to each other via any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0103] The computer system may include a client and a server. Typically, the client and the server are remote from each other and generally interact with each other via a communication network. The client-server relationship is created by a computer program that runs on a corresponding computer and has the client-server relationship. The server may be a cloud server, a server of a distributed system, a server incorporating a blockchain, or the like.

[0104] It should be understood that steps can be newly ranked, added, or deleted using the various aspects of the flow shown above. For example, each step described in this disclosure can be performed in parallel, sequentially, or in a different order. As long as the technical solutions disclosed in this disclosure can achieve the desired results, this disclosure is not limited thereto.

[0105] The above specific examples do not constitute limitations on the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions are possible depending on design matters and other factors. Any modifications, equivalent replacements, improvements, etc. within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. 1. A method for generating a file, comprising: inputting M first type files into a first model, respectively, and outputting a second type file corresponding to each first type file from the first model; determining a plurality of file pairs based on the output results, each file pair including a file of the first type and a corresponding file of the second type of the first type; adjusting a second model using the plurality of file pairs; and inputting M2 first type files into a respective adjusted second model, and outputting a corresponding second type file for each first type file from the adjusted second model, where M1 and M2 are positive integers; How the file is generated.

2. The M2 is greater than the M1. The file generation method according to claim 1 .

3. The M2 first type files include files that require security protection; The file generation method according to claim 1 .

4. the first type of files includes a text corpus and the second type of files includes a dialogue corpus; The file generation method according to claim 1 .

5. inputting each of the M1 first type files into a first model and outputting a corresponding second type file from the first model for each of the first type files, inputting the text corpus into the first model and outputting a plurality of dialogue questions related to the text corpus from the first model; and outputting a corresponding dialogue corpus from the first model based on the sentence corpus and the plurality of dialogue questions, the dialogue corpus including the plurality of dialogue questions and answers to each of the dialogue questions. The file generating method according to claim 4.

6. inputting each of the M first type files into a first model, generating a prompt for each of the sentence corpora and inputting the prompt into the first model; Wherein, the prompt has the content of the sentence corpus and an identity feature of a conversational party, and the identity feature of the conversational party is used to cause the first model to output a dialogue corpus that satisfies the identity feature of the conversational party. The file generating method according to claim 4.

7. Inputting the prompt into the first model includes: optimizing the prompt using a prompt optimization method; inputting the optimized prompts into the first model. The file generating method according to claim 6.

8. The text corpus has a preset expression style. The file generating method according to claim 6.

9. determining a plurality of file pairs based on the output result, removing the dialogue corpus that does not match the predetermined expression style and the dialogue corpus that does not match the identity characteristics of the interlocutor from the output result of the first model to obtain a remaining output result; determining a number of file pairs based on the remaining output results; The file generation method according to claim 8.

10. 1. A file generation device, comprising: a first input module for inputting M first type files into a first model, respectively, and outputting a second type file corresponding to each first type file from said first model; a determination module for determining a plurality of file pairs based on the output result, each file pair including one of the first type files and a corresponding one of the first type files and the second type file; an adjustment module for adjusting a second model using the plurality of file pairs; a second input module for inputting M2 first type files into the adjusted second model, respectively, and outputting a corresponding second type file for each first type file from the adjusted second model, where M1 and M2 are positive integers; File generator.

11. The M2 is greater than the M1. The file generating device according to claim 1 .

12. The M2 first type files include files that require security protection; 12. The file generating device according to claim 10 or 11.

13. the first type of files includes a text corpus and the second type of files includes a dialogue corpus; 12. The file generating device according to claim 10 or 11.

14. The first input module is inputting the text corpus into the first model and outputting a plurality of dialogue questions related to the text corpus from the first model; and outputting a corresponding dialogue corpus from the first model based on the sentence corpus and the plurality of dialogue questions, the dialogue corpus including the plurality of dialogue questions and answers to each of the dialogue questions. The file generation device according to claim 13.

15. The first input module is generating prompts for each of the sentence corpora and inputting the prompts into the first model; Wherein, the prompt has the content of the sentence corpus and an identity feature of a conversational party, and the identity feature of the conversational party is used to cause the first model to output a dialogue corpus that satisfies the identity feature of the conversational party. The file generation device according to claim 13.

16. The first input module is optimizing the prompt using a prompt optimization method; inputting the optimized prompts into the first model; and 16. The file generation device according to claim 15.

17. The text corpus has a preset expression style.

16. The file generation device according to claim 15.

18. The decision module: removing the dialogue corpus that does not match the predetermined expression style and the dialogue corpus that does not match the identity characteristics of the interlocutor from the output result of the first model to obtain a remaining output result; determining a plurality of file pairs based on the remaining output results; 18. The file generation device according to claim 17.

19. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 9. Electronic devices.

20. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1 to 9.

21. A program for implementing the method according to any one of claims 1 to 9 when executed by a processor in a computer.

Images