Instruction fine tuning data generation method and device, electronic equipment and storage medium

Abstract

The invention provides an instruction fine tuning data generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining domain background data, inputting the domain background data and a first instruction into a preset large language model, and enabling the preset large language model to output problem data based on the domain background data and the first instruction; and inputting the question data, the domain background data and the second instruction into a preset big language model, so that the preset big language model answers the question data based on the domain background data and the second instruction, and outputting answer data. The field background knowledge to which the instruction fine-tuning data belongs is added as one-dimensional input, so that the large language model can generate the instruction fine-tuning data based on the related field background knowledge, the field correlation of the instruction fine-tuning data is ensured, and the accuracy of the instruction fine-tuning data is improved. And in addition, field background knowledge can be referred to in the question and answer generation stage, the correctness of answer data is improved, and the accuracy of instruction fine adjustment data is further improved.

Description

Technical Field

[0001] The present invention relates to the field of data collection technology, and in particular to a method, device, electronic device and storage medium for generating instruction fine-tuning data. Background Art

[0002] With the rapid development of AI (Artificial Intelligence) technology, Large Language Model (LLM) has been widely used. Large Language Model is a deep learning model trained based on massive text data. It uses neural network technology, usually containing billions of parameters, and can generate and understand natural language text by learning from large-scale corpora. Large language model can handle a variety of natural language tasks, such as text classification, question and answer, dialogue, translation, text summarization, etc.

[0003] Instruction Tuning is an important method to improve the controllability of large language models. The core idea of ​​this method is to adjust the behavior of the model through a set of prompts and expected outputs written by humans to make it more consistent with human intentions. Specifically, the process of instruction tuning is to perform supervised learning on a dataset containing human-written prompts and expected outputs. In this way, the model can learn how to generate corresponding outputs based on given instructions, thereby improving its task generalization ability and ability to understand instructions.

[0004] The above-mentioned prompts and expectations written by humans are usually called instructions. In related technologies, instructions are usually constructed by manual annotation or self-instruct methods. The self-instruct method is to manually write less seed data and call the existing large model to generate instruction data based on the seed data. However, this method that relies on manual labor is very time-consuming and labor-intensive, resulting in high labor costs. The self-instruct method relies too much on the capabilities of the large model itself, and may cause inaccurate instruction data due to the large model hallucination problem (the model may produce some seemingly reasonable but actually wrong outputs when processing inputs. This may be because the model overfits the training data during training, resulting in excessive sensitivity to noise or specific samples). Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method, device, electronic device and storage medium for generating instruction fine-tuning data to improve the generation accuracy of instruction data.

[0006] According to one aspect of the present invention, a method for generating instruction fine-tuning data is provided, the method comprising:

[0007] Acquire domain background data, wherein the domain background data is determined based on the task of the model to be fine-tuned;

[0008] Inputting the domain background data and the first instruction into a preset large language model, so that the preset large language model outputs question data based on the domain background data and the first instruction;

[0009] The question data, the domain background data and the second instruction are input into the preset large language model, so that the preset large language model answers the question data based on the domain background data and the second instruction, and outputs answer data.

[0010] In a possible embodiment, the acquiring of domain background data includes:

[0011] Determine the target domain keywords based on the task of the model to be fine-tuned;

[0012] Based on the target domain keywords, search from web pages and / or books to obtain domain background data.

[0013] In a possible embodiment, the domain background data includes structured data and unstructured data, wherein the structured data includes tables and graphs, and the unstructured data includes web pages and books; the method further includes:

[0014] Extracting target field information from the structured data according to a preset conversion template, wherein the preset conversion template includes a target field identifier;

[0015] Based on the preset conversion template and the target field information, unstructured data corresponding to the structured data is obtained.

[0016] In a possible embodiment, the method further includes:

[0017] Based on a preset data condition, the instruction fine-tuning data is detected, wherein the preset data condition is in a regular matching form, and the instruction fine-tuning data includes the question data and answer data corresponding to the question data;

[0018] The instruction fine-tuning data that does not meet the preset data condition is pruned to make the instruction fine-tuning data meet the preset data condition.

[0019] According to another aspect of the present invention, there is provided a device for generating instruction fine-tuning data, the device comprising:

[0020] An acquisition module, used for acquiring domain background data, wherein the domain background data is determined based on the task of the model to be fine-tuned;

[0021] An input module, used for inputting the domain background data and the first instruction into a preset large language model, so that the preset large language model outputs question data based on the domain background data and the first instruction;

[0022] The answering module is used to input the question data, the domain background data and the second instruction into the preset large language model, so that the preset large language model answers the question data based on the domain background data and the second instruction, and outputs answer data.

[0023] In a possible embodiment, the acquiring of domain background data includes:

[0024] Determine the target domain keywords based on the task of the model to be fine-tuned;

[0025] Based on the target domain keywords, search from web pages and / or books to obtain domain background data.

[0026] In a possible embodiment, the domain background data includes structured data and unstructured data, wherein the structured data includes tables and graphs, and the unstructured data includes web pages and books; the device further includes:

[0027] A conversion module, used to extract target field information from the structured data according to a preset conversion template, wherein the preset conversion template includes a target field identifier;

[0028] Based on the preset conversion template and the target field information, unstructured data corresponding to the structured data is obtained.

[0029] In a possible embodiment, the device further includes:

[0030] A detection module, configured to detect the instruction fine-tuning data based on a preset data condition, wherein the preset data condition is in a regular matching form, and the instruction fine-tuning data includes the question data and answer data corresponding to the question data;

[0031] The instruction fine-tuning data that does not meet the preset data condition is pruned to make the instruction fine-tuning data meet the preset data condition.

[0032] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0033] Processor; and

[0034] Memory for storing programs,

[0035] The program includes instructions, which, when executed by the processor, enable the processor to execute any of the above-mentioned instruction fine-tuning data generation methods.

[0036] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute any of the above-mentioned instruction fine-tuning data generation methods.

[0037] One or more technical solutions provided in the embodiments of the present invention, by adding the domain background knowledge to which the instruction fine-tuning data belongs as a one-dimensional input in the process of using the large model to generate instruction fine-tuning data, enable the large model to generate instruction fine-tuning data based on the relevant domain background knowledge, ensure the relevance of the instruction fine-tuning data to the domain, and thus improve the accuracy of the generation of the instruction fine-tuning data. In addition, in the process of generating the instruction fine-tuning data, the domain background knowledge is referenced in the stages of generating questions and answers, which improves the correctness of the answer data and further improves the accuracy of the instruction fine-tuning data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Further details, features and advantages of the invention are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0039] Figure 1 An example diagram of fine-tuning data for generated instructions;

[0040] Figure 2 A schematic diagram of a flow chart of a method for generating instruction fine-tuning data provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a process of converting structured data in the instruction fine-tuning data generation method provided in an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of another process of converting structured data in the instruction fine-tuning data generation method provided in an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of another flow chart of a method for generating instruction fine-tuning data provided by an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of the structure of a device for generating instruction fine-tuning data provided by an embodiment of the present invention;

[0045] Figure 7 A block diagram of an exemplary electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0046] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0047] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0048] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0049] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0050] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0051] As described in the background technology, the related technology usually constructs instruction data through manual annotation or self-instruct method. Manual annotation means that the required instruction data is written manually, and self-instruct usually uses the generation capability of the GPT model to generate a large amount of supervised instruction fine-tuning data. Exemplarily, the related technology usually generates instruction data through the following steps:

[0052] Instruction generation: Manually write seed instruction examples to build a task pool, and then randomly extract a portion of instructions from the task pool as learning samples for the large model each time. By using prompt templates, these instructions are combined and input into the model, allowing the existing large model to generate new task instructions. Specifically, the following can be generated: Figure 1The instruction data shown is generally composed of instructions, inputs and outputs.

[0053] Classification task discrimination: Since the next stage handles classification tasks and non-classification tasks differently, it is necessary to discriminate whether the instruction belongs to a classification task. Here, we can directly construct a few-shot prompt template to let the large model make classification judgments.

[0054] Instance generation: Use the existing large model to generate supervised fine-tuning instances containing instructions, inputs, and outputs based on the type of task and the existing data in the task pool, using the input-first or output-first approach. Classification tasks often use the output-first approach, that is, the model first generates the output category and then generates the input content, thereby avoiding the generated content being biased towards a certain category label.

[0055] Filtering: Filter out low-quality examples through some heuristic rules, such as comparing the ROUGE-L similarity of the generated new instructions with the existing instructions, and filtering out those with greater similarity; filtering out invalid instruction instances that are too short, cannot be parsed, involve multi-modal correlation, etc.

[0056] It can be seen that in the related technology, when using a large model to generate instruction data, it is completely dependent on the generation ability of the large model itself, and it is very easy to have instruction data generation errors caused by model hallucination problems.

[0057] In order to improve the accuracy of instruction data generation, the embodiments of the present invention provide an instruction fine-tuning data generation method, device, electronic device and storage medium. The instruction fine-tuning data generation method provided by the embodiments of the present invention can be applied to any electronic device with an instruction data generation function, which can be a server, a computer or a mobile terminal. The following describes the solution of the present invention with reference to the accompanying drawings:

[0058] Figure 2 A flow chart of a method for generating instruction fine-tuning data provided by an embodiment of the present invention may include the following steps:

[0059] S201, acquiring domain background data, wherein the domain background data is determined based on the task of the model to be fine-tuned;

[0060] S202, inputting the domain background data and the first instruction into a preset large language model, so that the preset large language model outputs question data based on the domain background data and the first instruction;

[0061] S203: input the question data, the domain background data and the second instruction into the preset large language model, so that the preset large language model answers the question data based on the domain background data and the second instruction, and outputs answer data.

[0062] By applying the embodiment of the present invention, in the process of using the large model to generate instruction fine-tuning data, the domain background knowledge to which the instruction fine-tuning data belongs is added as a one-dimensional input, so that the large model can generate instruction fine-tuning data based on the relevant domain background knowledge, thereby ensuring the relevance of the instruction fine-tuning data to the domain, and thus improving the accuracy of the generation of the instruction fine-tuning data. In addition, in the process of generating instruction fine-tuning data, the domain background knowledge is referenced in the stages of generating questions and answers, thereby improving the correctness of the answer data and further improving the accuracy of the instruction fine-tuning data.

[0063] S201-S203 are exemplarily described below:

[0064] In S201, the domain background data refers to the background knowledge related to the domain to which the instruction data to be generated belongs. The background knowledge may include all the contents retrieved from the network using the domain keyword as the search term. The domain to which the above instruction data belongs may be determined based on a specific model task. For example, if the instruction data to be generated is instruction fine-tuning data for fine-tuning a large financial model, the domain to which the instruction data belongs is the financial domain. The above domain keywords may be pre-set for different fields, and the present invention does not specifically limit this. For example, based on the above example, for the financial field, the domain keywords may include "finance", "stocks", "investment" and the like.

[0065] In a possible embodiment, the above-mentioned domain keywords can be determined for specific model tasks. For example, for a large financial model used for market forecasting, domain keywords can be set to include "financial market", "stock", "bond", "foreign exchange" and "trend", etc. For a large financial model used for risk assessment, domain keywords can be set to include "financial institution" and "financial product", etc. In this way, by setting domain keywords based on specific model tasks, it is possible to filter the domain background knowledge, reduce the amount of subsequent data processing, and thus improve the efficiency of instruction data generation.

[0066] In a possible embodiment, target domain keywords may be determined based on the task of the model to be fine-tuned; and based on the target domain keywords, a search may be performed from web pages and / or books to obtain domain background data.

[0067] In a possible embodiment, domain background data related to the domain keywords in the network can be obtained by using crawler software based on the above-mentioned domain keywords. The above-mentioned domain background data can be any form of data retrieved from the network or paper documents, such as unstructured data, such as books, web pages, etc., or structured data, such as tables or graphs, etc., wherein the graph includes nodes and edges between nodes, the nodes are used to represent objects, and the objects may include people, nationalities, things, etc., and the edges are used to represent the relationship between nodes, and specifically, the edges may be directed edges.

[0068] In a possible embodiment, since a large language model generally has a high understanding ability of natural language text, in order to improve the large language model's understanding of domain background data and thereby improve the accuracy of the generated instruction data, the above structured data may be converted into unstructured data.

[0069] In a possible embodiment, the structured data may be converted into unstructured data according to the following steps:

[0070] S11, extracting target field information from the structured data according to a preset conversion template, wherein the preset conversion template includes a target field identifier;

[0071] S12: Based on the preset conversion template and the target field information, obtain unstructured data corresponding to the structured data.

[0072] like Figure 3 As shown, Figure 3 The process diagram of converting a table into unstructured data in an embodiment of the present invention is as follows: For a table with company, date, founder, and address information, a conversion template can be pre-set, such as the fields to be included in the conversion template and the direct connectives of the fields. Specifically, the conversion template can be set to "[Company] was established on [Date], and its headquarters is located at [Address]. The founder of the company is [Founder]", and the target fields included in the conversion template are [Company], [Date], [Address], and [Founder].

[0073] Tables are usually stored in a key-value format, where the key is the representation of each piece of data, and the value is the specific value of the field contained in each data identifier. Therefore, as a possible implementation, the corresponding value can be obtained based on the target field in the conversion template to obtain the corresponding information in the table and fill it into the matching format. For example, for the first piece of data, the unstructured data "Microsoft was founded in 1975 and its headquarters is located in Redmond, Washington. The company's founder is Bill Gates." can be obtained.

[0074] like Figure 4 As shown, Figure 4 Schematic diagram of a process of converting a graph into unstructured data in an embodiment of the present invention. Specifically, the starting node of a directed edge can be used as a subject, the relationship represented by the edge can be used as an attributive, and the ending node of the directed edge can be used as an object. Specifically, the conversion template can be set to [starting node][edge][ending node]. Figure 4 As shown in the figure, we can get the unstructured data "Yoshua Bengio's birthday is March 5, 1964", "Yoshua Bengio's nationality is Canadian", "Yoshua Bengio's brother is Samy Bengio".

[0075] After obtaining the domain background data, the domain background data and instructions can be input into the preset large model, wherein the instructions may include requirements for the question data to be generated, such as the number, format, content requirements and prohibited words of the question data to be generated, etc. The instructions can be flexibly set according to the specific model task, and the present invention does not make specific limitations on this. Exemplarily, the instruction may be "Please generate ten questions or task instructions that are as diverse as possible based on the domain background. These questions can be questions about facts, or they can be understanding and evaluation of related content. These instructions and questions may involve the exploration, extraction, and summarization of factual knowledge. Please assume that there are no corresponding articles for reference when asking questions, so do not use demonstrative pronouns "this" or "these" in the questions. Please generate questions in the following format". The above-mentioned large model can also be flexibly selected according to the actual application scenario, such as chatGPT, Kimi, etc.

[0076] The preset large model can generate the question data in the instruction fine-tuning data based on the domain background data and the first instruction, and then the question data, the domain background data and the second instruction data can be input into the above-mentioned preset large model, wherein the second instruction data is the relevant instruction requiring the large model to answer the above-mentioned question data, and the specific content can be flexibly set according to the actual application scenario, and can include answer conditions and answer formats, etc., wherein the answer conditions may include prohibited words and the like. Exemplarily, the second instruction data may be "Please answer the following question based on the domain background: [question data], please answer this question as detailed as possible, but do not change the key information in the original text, and do not include expressions such as "based on the above article" in the answer. Please generate the corresponding answer in the following format [question, answer]".

[0077] Through the above steps, multiple instruction fine-tuning data generated by the preset large model can be obtained, that is, [question, answer] pairs. In actual applications, even if the instructions require the model not to assume prior knowledge of external documents in the generated questions, and prohibit the use of "this" as a pronoun to generate questions, and the generated answers do not contain reference phrases such as "according to the above content", due to the performance of the large model, the generated instruction fine-tuning data may still not conform to the instructions. Therefore, in a possible embodiment, after generating the instruction fine-tuning data, the instruction fine-tuning data can be trimmed and filtered to further improve the accuracy of the generated instruction fine-tuning data.

[0078] As a possible implementation method, data conditions can be preset, that is, the instruction fine-tuning data format can be preset, and a regular matching formula can be set based on the format. If the instruction fine-tuning data successfully matches the regular matching formula, it can be determined that the format of the instruction fine-tuning data meets the requirements. If the match fails, the instruction fine-tuning data can be adjusted. Specifically, the part of the instruction fine-tuning data that has not been matched successfully can be determined and deleted, so as to obtain instruction fine-tuning data that meets the requirements.

[0079] As a possible implementation, the regular matching formula may be stored in a filter, and the filter may filter the instruction fine-tuning data based on the regular matching formula. The filter may be implemented by coding.

[0080] In a possible embodiment, Figure 2 As shown, the above steps may also include:

[0081] S204: whether the question data and the answer data meet the preset data conditions; if so, execute S205; if not, return to S202 to regenerate the instruction fine-tuning data;

[0082] S205: Use the question data and answer data as instruction fine-tuning data for large model training.

[0083] In a possible embodiment, instructions and domain background data may also be added to the question data so that the model can learn customized domain tasks, such as reading comprehension and information extraction.

[0084] Figure 5Another flow chart of the method for generating instruction fine-tuning data provided by the embodiment of the present invention may include the following steps: inputting unsupervised data into a large language model, the unsupervised data may be searched from a medium such as a web page or data, and specifically may include structured data and unstructured data, wherein structured data may include knowledge graphs and tables, etc., and unstructured data may include books and web pages, etc. The large language model may generate question data based on the unsupervised data and the input instructions, and then the unsupervised data, instructions and question data may be input into the large language model, and the large language model performs natural language processing on the unsupervised data, instructions and question data, thereby obtaining answer data corresponding to each question data, thereby obtaining instruction fine-tuning data.

[0085] By using the embodiments of the present invention, in the absence of instruction data, a high-quality large model can be used to generate instruction data from unsupervised documents, and the model's ability to follow instructions can be improved through supervised fine-tuning. This method provides an effective solution for supervised fine-tuning data generation in vertical fields.

[0086] Based on the same inventive concept, the embodiment of the present invention also provides a command data generating device, such as Figure 6 As shown, the apparatus 600 may include:

[0087] An acquisition module 601 is used to acquire domain background data, wherein the domain background data is determined based on the task of the model to be fine-tuned;

[0088] An input module 602, configured to input the domain background data and the first instruction into a preset large language model, so that the preset large language model outputs question data based on the domain background data and the first instruction;

[0089] The answering module 603 is used to input the question data, the domain background data and the second instruction into the preset large language model, so that the preset large language model answers the question data based on the domain background data and the second instruction, and outputs answer data.

[0090] In a possible embodiment, the acquiring of domain background data includes:

[0091] Determine the target domain keywords based on the task of the model to be fine-tuned;

[0092] Based on the target domain keywords, search from web pages and / or books to obtain domain background data.

[0093] In a possible embodiment, the domain background data includes structured data and unstructured data, wherein the structured data includes tables and graphs, and the unstructured data includes web pages and books; the device further includes:

[0094] A conversion module, used to extract target field information from the structured data according to a preset conversion template, wherein the preset conversion template includes a target field identifier;

[0095] Based on the preset conversion template and the target field information, unstructured data corresponding to the structured data is obtained.

[0096] In a possible embodiment, the device further includes:

[0097] A detection module, configured to detect the instruction fine-tuning data based on a preset data condition, wherein the preset data condition is in a regular matching form, and the instruction fine-tuning data includes the question data and answer data corresponding to the question data;

[0098] The instruction fine-tuning data that does not meet the preset data condition is pruned to make the instruction fine-tuning data meet the preset data condition.

[0099] Among them, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0100] The exemplary embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform a method according to an embodiment of the present invention when executed by the at least one processor.

[0101] Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0102] An exemplary embodiment of the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform a method according to an embodiment of the present invention.

[0103] refer to Figure 7, a block diagram of an electronic device 700 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0104] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0105] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 may be any type of device capable of inputting information to the electronic device 700, and the input unit 706 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 707 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 708 may include but is not limited to a disk, an optical disk. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0106] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned instruction fine-tuning data generation method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 may be configured to execute the above-mentioned instruction fine-tuning data generation method by any other appropriate means (e.g., by means of firmware).

[0107] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0108] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0109] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0112] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

Claims

1. A method for generating instruction fine-tuning data, characterized in that: The method comprises: Acquire domain background data, wherein the domain background data is determined based on the task of the model to be fine-tuned; Inputting the domain background data and the first instruction into a preset large language model, so that the preset large language model outputs question data based on the domain background data and the first instruction; The question data, the domain background data and the second instruction are input into the preset large language model, so that the preset large language model answers the question data based on the domain background data and the second instruction, and outputs answer data.

2. The method according to claim 1, characterized in that The obtaining of domain background data includes: Determine the target domain keywords based on the task of the model to be fine-tuned; Based on the target domain keywords, search from web pages and / or books to obtain domain background data.

3. The method according to claim 2, characterized in that The domain background data includes structured data and unstructured data, wherein the structured data includes tables and graphs, and the unstructured data includes web pages and books; the method further includes: Extracting target field information from the structured data according to a preset conversion template, wherein the preset conversion template includes a target field identifier; Based on the preset conversion template and the target field information, unstructured data corresponding to the structured data is obtained.

4. The method according to claim 1, characterized in that: The method further comprises: Based on a preset data condition, the instruction fine-tuning data is detected, wherein the preset data condition is in a regular matching form, and the instruction fine-tuning data includes the question data and answer data corresponding to the question data; The instruction fine-tuning data that does not meet the preset data condition is pruned to make the instruction fine-tuning data meet the preset data condition.

5. A device for generating instruction fine-tuning data, characterized in that: The device comprises: An acquisition module, used for acquiring domain background data, wherein the domain background data is determined based on the task of the model to be fine-tuned; An input module, used for inputting the domain background data and the first instruction into a preset large language model, so that the preset large language model outputs question data based on the domain background data and the first instruction; The answering module is used to input the question data, the domain background data and the second instruction into the preset large language model, so that the preset large language model answers the question data based on the domain background data and the second instruction, and outputs answer data.

6. The device according to claim 5, characterized in that The obtaining of domain background data includes: Determine the target domain keywords based on the task of the model to be fine-tuned; Based on the target domain keywords, search from web pages and / or books to obtain domain background data.

7. The device according to claim 6, characterized in that The domain background data includes structured data and unstructured data, wherein the structured data includes tables and graphs, and the unstructured data includes web pages and books; the device also includes: A conversion module, used to extract target field information from the structured data according to a preset conversion template, wherein the preset conversion template includes a target field identifier; Based on the preset conversion template and the target field information, unstructured data corresponding to the structured data is obtained.

8. The device according to claim 5, characterized in that The device also includes: A detection module, configured to detect instruction fine-tuning data based on preset data conditions, wherein the preset data conditions are in the form of regular matching, and the instruction fine-tuning data includes the question data and answer data corresponding to the question data; The instruction fine-tuning data that does not meet the preset data condition is pruned to make the instruction fine-tuning data meet the preset data condition.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-4.

Images