Question and answer data generation method and device oriented to large language model and computing equipment

Abstract

The invention discloses a large language model-oriented question and answer data generation method, which comprises the following steps of: obtaining a plurality of original technical documents in various document formats, and converting each original technical document into a structured document in a uniform format; extracting a multi-level document structure and field features from the structured document, and cleaning the structured document according to the multi-level document structure and field features to obtain a structured cleaned document; building a domain context for the structured cleaning document based on the domain knowledge base, and performing data enhancement on the structured cleaning document according to the domain context to obtain cleaning enhanced data; and performing data expansion based on the cleaning enhanced data by using a large language model, and generating question and answer data. On the basis, high-quality question and answer pairs with domain pertinence can be generated.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a question-and-answer data generation method, a question-and-answer data generation device, and a computing device for large language models. Background Technology

[0002] In the research and development process of specialized technical fields, a large number of technical documents need to be processed, including academic research papers, engineering design documents, experimental reports, patent documents, etc. These documents are usually characterized by multi-source heterogeneity, complex structure, high levels of noisy data, and strong domain specialization.

[0003] Currently, the main problems with document cleaning and question-answer pair generation technologies are as follows: 1) Limitations of general document cleaning methods. Existing document cleaning tools are mainly designed for general documents, simply removing formatting information such as headers, footers, and watermarks. They cannot identify and process special content structures in professional fields (such as professional data tables, parameter charts, formula derivation processes, etc.). Moreover, they lack semantic understanding for cross-disciplinary content, which can easily lead to the accidental deletion of valuable technical information or the retention of irrelevant content.

[0004] 2) Lack of domain-specific adaptation in data augmentation. General LLM data augmentation methods cannot accurately understand domain-specific concepts, and the generated augmented data may contain factual errors.

[0005] 3) The quality of the question-and-answer pairs is not high. The generated questions lack domain specificity, fail to cover core knowledge points, and have a limited range of question types, making it difficult to guarantee the accuracy of the answers, especially when quantitative parameters and technical details are involved.

[0006] 4) Lack of quality assessment and feedback mechanisms. The system cannot automatically assess the accuracy and practicality of generated question-and-answer pairs within the professional field, lacks a verification and error-correction mechanism based on a domain expert knowledge base, and makes it difficult to conduct iterative optimization and continuous improvement.

[0007] Therefore, a question-answering data generation method for large language models is needed to solve the problems existing in the above technical solutions. Summary of the Invention

[0008] To address this, the present invention provides a question-and-answer data generation method and apparatus for large language models, in order to solve or at least alleviate the problems mentioned above.

[0009] According to one aspect of the present invention, a method for generating question-and-answer data for a large language model is provided, executed in a computing device, comprising: acquiring multiple original technical documents in various document formats, converting each of the original technical documents into a structured document in a uniform format; extracting multi-level document structure and domain features from the structured documents, cleaning the structured documents according to the multi-level document structure and the domain features to obtain a cleaned structured document; constructing a domain context for the cleaned structured document based on a domain knowledge base, and performing data augmentation on the cleaned structured document according to the domain context to obtain cleaned and augmented data; and using a large language model, expanding the data based on the cleaned and augmented data, and generating question-and-answer data, wherein the question-and-answer data includes multiple question-and-answer pairs, each containing a question and a corresponding answer.

[0010] Optionally, in the question-answering data generation method for a large language model according to the present invention, cleaning the structured document according to the multi-level document structure and the domain features includes: generating prompt word instructions according to the multi-level document structure and the domain features; and cleaning the structured document according to the prompt word instructions using a large language model.

[0011] Optionally, in the question-answering data generation method for large language models according to the present invention, extracting multi-level document structure and domain features from the structured document includes: performing document structure recognition on the structured document to obtain a multi-level document structure, wherein the multi-level document structure includes a format layer, a structure layer, and a content layer, wherein the format layer includes headers and footers, watermarks, and page numbers, the structure layer includes a table of contents, chapters, abstracts, and references, and the content layer includes tables, formulas, and technical parameters; and extracting domain features from the structured document, wherein the domain features include domain classification information, professional term tags, technical parameters, and knowledge type tags.

[0012] Optionally, in the question-answering data generation method for a large language model according to the present invention, the large language model is used to expand the data based on the cleaned and enhanced data and generate question-answering data, including: using the large language model to extract multiple words from the cleaned and enhanced data, and generating corresponding synonyms or near-synonyms for each word, wherein the words include keywords and / or phrases; generating multiple semantically similar text variants based on each word and its corresponding synonyms or near-synonyms; and generating multiple question-answer pairs based on the multiple semantically similar text variants.

[0013] Optionally, in the question-answering data generation method for large language models according to the present invention, the domain knowledge base includes a domain classification system, a professional terminology system, a technical parameter system, and a knowledge relationship graph; the domain context includes domain classification information, key concepts, technical parameters, relevant literature, and domain constraints; data augmentation of the structured cleaned document based on the domain context includes: rewriting enhancement, expansion enhancement, and multi-angle generation of the structured cleaned document based on the domain context; wherein, the rewriting enhancement includes synonym rewriting, perspective switching, and detail adjustment; the expansion enhancement includes principle explanation, application scenarios, comparative analysis, and parameter derivation; and the multi-angle generation includes design angle generation, analysis angle generation, evaluation angle generation, and application angle generation.

[0014] Optionally, the question-answering data generation method for large language models according to the present invention further includes: performing quality assessment on each question-answer pair according to multi-dimensional evaluation indicators to obtain corresponding quality assessment results, wherein the multi-dimensional evaluation indicators include accuracy evaluation indicators, completeness evaluation indicators, diversity evaluation indicators, and usability evaluation indicators; updating the domain knowledge base and fine-tuning the large language model according to the quality assessment results.

[0015] Optionally, the question-and-answer data generation method for large language models according to the present invention further includes: generating export files in multiple export formats based on the question-and-answer data to export the question-and-answer data, wherein the multiple export formats include CSV format, OpenAI JSONL format, and JSON format.

[0016] Optionally, in the question-answering data generation method for large language models according to the present invention, multiple document formats include DOCX format, PDF format, and TXT format; the unified format is JSON format.

[0017] According to one aspect of the present invention, a question-and-answer data generation apparatus is provided, deployed in a computing device, the apparatus comprising: The acquisition module is suitable for acquiring multiple original technical documents in various document formats and converting each of the original technical documents into a structured document in a unified format; The intelligent cleaning module is adapted to extract multi-level document structure and domain features from the structured document, and clean the structured document according to the multi-level document structure and the domain features to obtain a structured cleaned document. The data augmentation module is adapted to construct a domain context for the structured cleaned document based on a domain knowledge base, and perform data augmentation on the structured cleaned document according to the domain context to obtain cleaned and augmented data. The domain knowledge base includes a domain classification system, a professional terminology system, a technical parameter system, and a knowledge relationship graph. The question-and-answer generation module is suitable for using a large language model to expand the data based on the cleaned and enhanced data, and to generate question-and-answer data, which includes multiple question-and-answer pairs, each containing a question and its corresponding answer.

[0018] According to one aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the question-answering data generation method for large language models as described above.

[0019] According to one aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method as described above.

[0020] According to one aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the question-answer data generation method for large language models as described above.

[0021] According to the technical solution of this invention, a method for generating question-and-answer data for large language models is provided. This method involves converting each original technical document into a structured document in a unified format, and then cleaning the document based on its multi-level structure and domain characteristics to obtain a cleaned structured document. This process effectively removes noise while accurately preserving key technical information, improving cleaning accuracy. Furthermore, a domain context is constructed for the cleaned structured document based on a domain knowledge base, and data augmentation is performed on the cleaned document according to this domain context. This allows for data augmentation based on an accurate understanding of professional domain concepts, thereby improving the quality of data augmentation, avoiding factual errors in the generated augmented data, and ultimately generating high-quality question-and-answer pairs with domain specificity.

[0022] Furthermore, according to the technical solution of the present invention, each question-answer pair is evaluated for quality based on multi-dimensional evaluation indicators, and optimization operations such as updating the domain knowledge base and fine-tuning the large language model are performed based on the quality evaluation results. In this way, not only can the accuracy and practicality of the generated question-answer pairs in the professional field be automatically evaluated, but a closed-loop iterative optimization mechanism can also be formed to continuously optimize the quality of the generated question-answer pairs.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout the invention, the same reference numerals generally refer to the same parts or elements.

[0025] Figure 1 A schematic diagram of a computing device 100 provided according to an embodiment of the present invention is shown; Figure 2 A flowchart illustrating a question-answering data generation method 200 for large language models provided according to an embodiment of the present invention is shown. Figure 3 A schematic diagram of a question-and-answer data generation apparatus 300 provided according to an embodiment of the present invention is shown. Detailed Implementation

[0026] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0027] To address some of the problems existing in current question-answer pair generation technologies, this invention proposes a question-answer data generation method for large language models. This method can perform data augmentation based on an accurate understanding of domain-specific concepts, avoiding factual errors in the generated augmented data, and thus generating high-quality question-answer pairs with domain relevance.

[0028] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown. Figure 1As shown, in a basic configuration, computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. System memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 104 includes an operating system 105.

[0030] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1 The basic configuration is illustrated by the components within the dashed lines. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The middle part is shown by removable storage device 109 and non-removable storage device 110.

[0031] As stated above, according to one aspect, program module 103 is stored in system memory 104. According to one aspect, program module 103 may include one or more applications. The present invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0032] According to one aspect, program module 103 may include a plurality of program instructions adapted to execute the question-and-answer data generation method 200 for large language models of the present invention, such that computing device 100 is configured to execute the question-and-answer data generation method 200 for large language models of the present invention.

[0033] According to one aspect, program module 103 may include a question-and-answer data generation device 300, which may be configured to perform the question-and-answer data generation method 200 for large language models of the present invention.

[0034] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0035] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0036] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program module 103). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0037] According to one aspect, the communication medium is implemented by computer-readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, the communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0038] In an embodiment of the invention, computing device 100 is configured to execute the question-answering data generation method 200 for large language models of the invention. Computing device 100 includes one or more processors and one or more readable storage media storing program instructions that, when executed by the one or more processors, cause the computing device to execute the question-answering data generation method 200 for large language models of the invention.

[0039] Figure 2 A flowchart illustrating a question-answering data generation method 200 for large language models according to an embodiment of the present invention is shown. The question-answering data generation method 200 for large language models can be executed in a computing device (e.g., the aforementioned computing device 100).

[0040] In embodiments of the present invention, the computing device 100 for executing the question-answering data generation method 200 for large language models can be a terminal or a server. The question-answering data generation method 200 for large language models of the present invention is applicable to knowledge management and intelligent question-answering system construction scenarios in professional fields.

[0041] like Figure 2 As shown, the question-answering data generation method 200 for large language models includes the following steps 210-240.

[0042] Step 210: The computing device 100 can acquire multiple original technical documents in various document formats and convert each original technical document into a structured document in a unified format.

[0043] In this embodiment of the invention, the computing device 100 can perform format recognition, document parsing, and preliminary structuring processing on each original technical document to convert each original technical document into a structured document with a unified format.

[0044] It should be noted that the technical documents in the embodiments of this invention may include academic research papers, engineering design documents, experimental reports, patent documents, etc. These technical documents are typically characterized by multi-source heterogeneity, complex structure, high levels of noisy data, and strong domain specialization.

[0045] In some embodiments, the multiple document formats include DOCX, PDF, and TXT formats, and the unified format can be JSON. That is, in step 210, the computing device 100 can acquire multiple original technical documents in various formats, including DOCX, PDF, and TXT, and convert each original technical document into a structured document in JSON format.

[0046] It should be noted that structured documents contain structured information, which may include one or more of the following: paragraphs, tables, and formulas.

[0047] Step 220: For each original technical document corresponding to a structured document, the computing device 100 can extract multi-level document structure and domain features from the structured document. Then, it can clean the structured document based on the multi-level document structure and domain features to obtain a cleaned structured document (i.e., the cleaned structured document).

[0048] It should be noted that cleaning structured documents based on their multi-level document structure and domain characteristics can accurately retain key technical information such as professional data tables, technical parameters, and mathematical formulas while removing noise such as headers, footers, and watermarks. This avoids accidentally deleting valuable technical information or retaining irrelevant content, thus improving the accuracy of the cleaning process.

[0049] In some embodiments, in step 220, the computing device 100 can perform document structure recognition (including format layer recognition, structure layer recognition, and content layer recognition) on the structured document to obtain a multi-level document structure. Here, the multi-level document structure includes a format layer, a structure layer, and a content layer. The format layer includes headers and footers, watermarks, page numbers, etc.; the structure layer includes a table of contents, chapters, abstracts, references, etc.; and the content layer includes tables (professional data tables), formulas, technical parameters, etc. Simultaneously, the computing device 100 can extract domain features from the structured document. These domain features include domain classification information, professional terminology tags, technical parameters, and knowledge type markers (e.g., including principle knowledge, design knowledge, experimental knowledge, and applied knowledge).

[0050] For example, after converting original technical documents related to electrical engineering into structured documents, the domain features extracted from these structured documents are as follows: the domain classification information is "Electrical Engineering", the professional term tags are "PLC Controller" and "Inverter", the technical parameters are, for example, "380V" and "50Hz", and the knowledge type is marked as "Principle-based + Application-based Knowledge".

[0051] Furthermore, structured documents can be cleaned based on multi-level document structure and domain characteristics. For example, format noise, content noise, and redundant information in structured documents can be identified and cleaned based on multi-level document structure and domain characteristics. At the same time, key information such as professional terminology descriptions, technical parameters, tables, formulas and mathematical models, and experimental data in structured documents can be retained to obtain cleaned structured documents.

[0052] In some embodiments, in step 220, the computing device 100 can generate prompt words based on the multi-level document structure and domain features obtained from the structured document. Subsequently, the prompt words are input into a large language model, which can then be used to clean the structured document according to the prompt words, resulting in a cleaned structured document. This can be achieved using the semantic understanding capabilities and regular expression substitution of the large language model.

[0053] For example, prompts generated based on multi-level document structure and domain characteristics include: "When a regular table structure is detected, retain the complete row and column relationships and parameter correspondences, and only clean the headers and footers around the table; when a formula is identified, retain the complete formula and variable descriptions, and delete the margin comments."

[0054] Step 230: The computing device 100 can construct a domain context for the structured cleaned document based on the domain knowledge base, and perform data augmentation on the structured cleaned document according to the domain context to obtain cleaned and augmented data.

[0055] In some embodiments, the domain knowledge base may include a domain classification system, a terminology system, a technical parameter system (and its reasonable range), and a knowledge relationship graph. The knowledge relationship graph may include mapping relationships between concepts, parameter constraint relationships, and performance causal relationships.

[0056] Domain context may include, for example, domain classification information, key concepts, technical parameters, relevant literature, domain constraints, etc. Here, this invention does not limit the specific information in the domain context.

[0057] In some embodiments, in step 230, after constructing a domain context for the structured cleaned document based on the domain knowledge base, the structured cleaned document can be augmented using various strategies based on the domain context, such as rewriting enhancement (e.g., including synonym rewriting, perspective shifting, and detail adjustment), expansion enhancement (e.g., including principle explanation, application scenarios, comparative analysis, and parameter derivation), and multi-angle generation. Rewriting enhancement may include, for example, synonym rewriting, perspective shifting, and detail adjustment; expansion enhancement may include, for example, principle explanation, application scenarios, comparative analysis, and parameter derivation; multi-angle generation may include, for example, design angle generation, analysis angle generation, evaluation angle generation, and application angle generation.

[0058] It should be understood that by constructing a domain context for structured cleaned documents based on a domain knowledge base, and then performing data augmentation on the structured cleaned documents based on the domain context, data augmentation can be performed on the basis of an accurate understanding of professional domain concepts, thereby avoiding factual errors in the generated augmented data.

[0059] Step 240: The computing device 100 can utilize a large language model to augment the data based on the cleaned and enhanced data, and generate question-and-answer data. Here, the question-and-answer data includes multiple question-and-answer pairs, each containing a question and its corresponding answer.

[0060] Specifically, in step 240, the computing device 100 can first input the cleaned and enhanced data into a large language model. Then, it can use the large language model to extract multiple words (including keywords and / or phrases) from the cleaned and enhanced data, and generate corresponding synonyms or near-synonyms for each word. Furthermore, based on each word and its corresponding synonyms or near-synonyms, it generates multiple semantically similar text variants, and generates multiple question-answer pairs based on the multiple semantically similar text variants.

[0061] Based on this, by expanding the data, the types of questions generated become richer and more diverse.

[0062] In some embodiments, in step 240, the computing device 100 may, for each generated question, obtain the answer corresponding to the question from the corresponding original technical document, and / or generate the answer corresponding to the question using a large language model. Furthermore, the answer corresponding to the question may be subject to factual verification, logical verification, completeness verification, and accuracy scoring to ensure the accuracy of the generated answer.

[0063] In some embodiments, after generating question-and-answer data (multiple question-and-answer pairs), the computing device 100 can also perform quality assessment on each question-and-answer pair based on multi-dimensional evaluation metrics to obtain corresponding quality assessment results. Furthermore, optimization operations such as updating the domain knowledge base and fine-tuning the large language model can be performed based on the quality assessment results, thereby forming a closed-loop optimization mechanism to continuously optimize the quality of the generated question-and-answer pairs.

[0064] Here, multi-dimensional evaluation metrics can include accuracy evaluation metrics, completeness evaluation metrics, diversity evaluation metrics, and usability evaluation metrics. Accuracy evaluation metrics may include, for example, factual accuracy, parameter correctness, and logical consistency; completeness evaluation metrics may include, for example, knowledge coverage, answer completeness, and type distribution; diversity evaluation metrics may include, for example, question diversity, difficulty distribution, and type distribution; usability evaluation metrics may include, for example, question clarity, answer readability, and practicality. However, it should be noted that this invention is not limited to the evaluation metrics listed above.

[0065] In some embodiments, after generating question-and-answer data (and performing quality assessment), the computing device 100 can also generate export files in various export formats based on the question-and-answer data, so that the question-and-answer data can be exported based on the export files in various export formats for users to download and use. The various export formats may include CSV format (containing fields such as question, answer, difficulty, type, domain_category, knowledge_type, etc.), OpenAI JSONL format (messages array format, used for model training), and JSON format (a structured format containing metadata and a list of question-and-answer pairs), but the present invention is not limited to the above export formats.

[0066] By generating export files in multiple export formats, complete metadata information (difficulty, type, domain classification, knowledge type, etc.) can be provided for subsequent applications in scenarios such as large language model training, knowledge base construction, and intelligent question answering systems, facilitating data filtering and evaluation.

[0067] According to the question-answering data generation method 200 for large language models in this embodiment of the invention, each original technical document is converted into a structured document in a unified format. Then, the document is cleaned based on its multi-level document structure and domain characteristics to obtain a cleaned structured document. This process removes noise while accurately preserving key technical information, improving the cleaning accuracy. Furthermore, a domain context is constructed for the cleaned structured document based on a domain knowledge base, and data augmentation is performed on the cleaned structured document according to the domain context. This allows for data augmentation based on an accurate understanding of professional domain concepts, thereby improving the quality of data augmentation, avoiding factual errors in the generated augmented data, and ultimately generating high-quality question-answer pairs with domain specificity.

[0068] Furthermore, according to the technical solution of the present invention, each question-answer pair is evaluated for quality based on multi-dimensional evaluation indicators, and optimization operations such as updating the domain knowledge base and fine-tuning the large language model are performed based on the quality evaluation results. In this way, not only can the accuracy and practicality of the generated question-answer pairs in the professional field be automatically evaluated, but a closed-loop iterative optimization mechanism can also be formed to continuously optimize the quality of the generated question-answer pairs.

[0069] Figure 3 A schematic diagram of a question-and-answer data generation apparatus 300 according to an embodiment of the present invention is shown. The question-and-answer data generation apparatus 300 can be deployed in a computing device 100, and the question-and-answer data generation apparatus 300 is configured to execute the question-and-answer data generation method 200 for large language models of the present invention.

[0070] like Figure 3As shown, in an embodiment of the present invention, the question-and-answer data generation device 300 includes an acquisition module 310, an intelligent cleaning module 320, a data enhancement module 330, and a question-and-answer generation module 340 that are connected in series.

[0071] The acquisition module 310 can acquire multiple original technical documents in various document formats and convert each original technical document into a structured document in a unified format.

[0072] The intelligent cleaning module 320 can extract multi-level document structure and domain features from structured documents, and clean the structured documents according to the multi-level document structure and the domain features to obtain structured cleaned documents.

[0073] The data augmentation module 330 can construct a domain context for the structured cleaned document based on the domain knowledge base, and perform data augmentation on the structured cleaned document according to the domain context to obtain cleaned and augmented data.

[0074] The question-and-answer generation module 340 can utilize a large language model to augment data based on cleaned and enhanced data, and generate question-and-answer data. The question-and-answer data includes multiple question-and-answer pairs, each containing a question and its corresponding answer.

[0075] It should be noted that the acquisition module 310, the intelligent cleaning module 320, the data augmentation module 330, and the question-and-answer generation module 340 are respectively used to execute the aforementioned steps 210 to 240. Here, the specific execution logic of each unit can be found in the description of steps 210 to 240 in method 200 above, and will not be repeated here.

[0076] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0077] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the question-answering data generation method for large language models of the present invention according to instructions in the program code stored in the memory.

[0078] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0079] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0080] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0081] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof.

[0082] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0083] Unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

Claims

1. A method for generating question-answering data for large language models, executed on a computing device, comprising: Obtain multiple original technical documents in various document formats, and convert each of the original technical documents into a structured document in a unified format; Extract multi-level document structure and domain features from the structured document, and clean the structured document according to the multi-level document structure and the domain features to obtain a cleaned structured document; A domain context is constructed for the structured cleaned document based on a domain knowledge base, and data augmentation is performed on the structured cleaned document based on the domain context to obtain cleaned and augmented data. Using a large language model, data augmentation is performed based on the cleaned and enhanced data, and question-and-answer data is generated. The question-and-answer data includes multiple question-and-answer pairs, each containing a question and its corresponding answer.

2. The method as described in claim 1, wherein, Cleaning the structured document based on the multi-level document structure and the domain characteristics includes: Based on the multi-level document structure and the domain characteristics, prompt words are generated; The structured document is cleaned using a large language model based on the prompt word instructions.

3. The method as described in claim 1 or 2, wherein, Extracting multi-level document structure and domain features from the structured document includes: The structured document is subjected to document structure recognition to obtain a multi-level document structure, wherein the multi-level document structure includes a format layer, a structure layer, and a content layer. The format layer includes headers and footers, watermarks, and page numbers. The structure layer includes a table of contents, chapters, abstracts, and references. The content layer includes tables, formulas, and technical parameters. Extract domain features from the structured document, wherein the domain features include domain classification information, technical term tags, technical parameters, and knowledge type tags.

4. The method according to any one of claims 1-3, wherein, Using a large language model, data augmentation is performed based on the cleaned and enhanced data, and question-and-answer data is generated, including: Using a large language model, multiple words are extracted from the cleaned and enhanced data, and corresponding synonyms or near-synonyms are generated for each word. The words include keywords and / or phrases. Based on each of the words and their corresponding synonyms or near-synonyms, generate multiple semantically similar text variants; Multiple question-answer pairs are generated based on the multiple semantically similar text variants.

5. The method according to any one of claims 1-4, wherein, The domain knowledge base includes a domain classification system, a terminology system, a technical parameter system, and a knowledge relationship graph. The domain context includes domain classification information, key concepts, technical parameters, relevant literature, and domain constraints. Data augmentation of the structured cleaned document based on the domain context includes: The structured cleaned document is rewritten, enhanced, expanded, and generated from multiple perspectives based on the domain context. The rewriting enhancement includes synonym rewriting, perspective switching, and detail adjustment. The expansion enhancement includes principle explanation, application scenarios, comparative analysis, and parameter derivation. The multi-angle generation includes design perspective generation, analysis perspective generation, evaluation perspective generation, and application perspective generation.

6. The method according to any one of claims 1-5, wherein, Also includes: Based on multi-dimensional evaluation indicators, a quality assessment is performed on each question-answer pair to obtain the corresponding quality assessment results. The multi-dimensional evaluation indicators include accuracy evaluation indicators, completeness evaluation indicators, diversity evaluation indicators, and usability evaluation indicators. Based on the quality assessment results, the domain knowledge base is updated, and the large language model is fine-tuned.

7. The method according to any one of claims 1-6, wherein, Also includes: Based on the question-and-answer data, export files in various export formats are generated to export the question-and-answer data. The various export formats include CSV format, OpenAI JSONL format, and JSON format.

8. The method according to any one of claims 1-7, wherein, Multiple document formats including DOCX, PDF, and TXT; The unified format is JSON.

9. A question-and-answer data generation apparatus, deployed in a computing device, the apparatus comprising: The acquisition module is suitable for acquiring multiple original technical documents in various document formats and converting each of the original technical documents into a structured document in a unified format; The intelligent cleaning module is adapted to extract multi-level document structure and domain features from the structured document, and clean the structured document according to the multi-level document structure and the domain features to obtain a structured cleaned document. The data augmentation module is adapted to construct a domain context for the structured cleaned document based on a domain knowledge base, and perform data augmentation on the structured cleaned document according to the domain context to obtain cleaned and augmented data. The domain knowledge base includes a domain classification system, a professional terminology system, a technical parameter system, and a knowledge relationship graph. The question-and-answer generation module is suitable for using a large language model to expand the data based on the cleaned and enhanced data, and to generate question-and-answer data, which includes multiple question-and-answer pairs, each containing a question and its corresponding answer.

10. A computing device, comprising: At least one processor; and A memory storing program instructions, wherein the program instructions are configured to be processed by the at least one processor, the program instructions including instructions for processing the method as claimed in any one of claims 1-8.

11. A computer program product comprising computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method as described in any one of claims 1-8.

Images