Railway construction engineering QA question and answer pair generation method and system based on large model

Abstract

The invention discloses a railway construction engineering QA question and answer pair generation method and system based on a large model. The method comprises the following steps: collecting and processing professional data text data in the field of railway construction engineering; writing prompt words for question input according to a preset prompt word writing rule; the written prompt words and the processed professional data text data are input into an intelligent question and answer system based on a large model, and QA question and answer pairs are generated; and establishing a matching relationship between the generated QA question and answer pair and the corresponding professional data text data, and inputting the matching relationship into a pre-established QA table template. According to the method, unstructured professional data text data of railway construction engineering is automatically transferred into structured QA pairs by using a large model technology; the problems that an existing rule-based traditional QA question and answer pair generation method seriously depends on manual writing of specific question and answer rules, the question coverage is narrow, answer matching is difficult, and manpower and time cost consumption is large are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of railway construction engineering, and in particular to a method and system for generating QA question-answer pairs for railway construction engineering based on a large model. Background Art

[0002] In the field of railway construction engineering, the generation method of QA question-answer pairs plays a vital role. It not only helps to build an efficient knowledge management system, but also provides a solid foundation for the successful implementation of the project.

[0003] Currently, traditional rule-based Q&A pair generation methods rely heavily on manually written, specific question-answering rules. This approach not only fails to fully cover all possible questions, but also, particularly given the vast and complex scope of railway construction engineering knowledge, many questions lack matching answers. Because each rule requires detailed research and validation to ensure its accuracy and applicability, writing and maintaining these rules consumes significant manpower and time. Furthermore, with the rapid advancement of industries and technologies, existing rules quickly become outdated, necessitating constant updates and revisions, further increasing the workload and complexity of maintenance. Furthermore, writing effective rules often requires a deep professional background, limiting the pool of qualified personnel and leading to higher costs. Therefore, while rule-based approaches still have value in certain specific scenarios, their limitations are becoming increasingly apparent when dealing with complex and ever-changing knowledge domains, making them difficult to meet the demands of efficient and flexible knowledge management. Summary of the Invention

[0004] This application provides a large-scale model-based QA question-answer pair generation method and system for railway construction projects to solve the problems of existing traditional rule-based QA question-answer pair generation methods, which heavily rely on manual compilation of specific question-answer rules, have narrow question coverage, are difficult to match answers, and consume a lot of manpower and time costs.

[0005] According to the first aspect, an embodiment provides a method for generating QA question-answer pairs for railway construction projects based on a large model, the method comprising:

[0006] Collect and process professional text data in the field of railway construction engineering;

[0007] Write prompt words for question input according to the preset prompt word writing rules;

[0008] Input the written prompt words and processed professional material text data into the intelligent question-answering system based on the big model to generate QA question-answer pairs;

[0009] A matching relationship is established between the generated QA question-answer pairs and the corresponding professional material text data, and they are entered into the pre-created QA form template.

[0010] Furthermore, we collected and processed professional text data in the field of railway construction engineering, including:

[0011] The professional materials collected and uploaded include unstructured texts of railway construction engineering professional technical standards, specifications, question banks, and audit manuals.

[0012] Furthermore, we collected and processed professional text data in the field of railway construction engineering, including:

[0013] Generate a corresponding URL for each uploaded professional document, take the URL of the uploaded professional document as input, and use the text reading plug-in API to output the text content of the professional document in the form of string data.

[0014] Furthermore, the prompt words for question input are written according to the preset prompt word writing rules, specifically including:

[0015] The prompt words include three parts: role positioning, skill requirements and restrictions;

[0016] Role positioning, including:

[0017] Experienced Q&A extraction experts in the railway construction engineering field: This requires extensive knowledge and experience in the railway construction engineering field, the ability to accurately understand professional text content, pinpoint problem scenarios, and efficiently generate question-answer pairs that meet actual needs;

[0018] Skill requirements include extracting problem scenarios and generating question-answer pairs;

[0019] Extract the problem scenario:

[0020] Comprehensive and meticulous: This requires a comprehensive and in-depth exploration of all possible problem scenarios in the text, without missing any potential problems, ensuring the comprehensiveness of the problem scenarios;

[0021] Segmented analysis and processing: When faced with lengthy and complex professional texts, they should be broken down into several paragraphs or sections and analyzed one by one to more accurately grasp the problem scenarios in each section and avoid inaccurate problem scenario extraction due to the complexity of the overall content;

[0022] Generate question-answer pairs:

[0023] Concise and clear: For each question scenario, the answer should be concise and clear, directly hitting the core of the question, avoiding redundant information, and ensuring that users can quickly understand and obtain the required information;

[0024] Clear format segmented output: Questions and answers should be presented in segments according to a certain format, with a clear structure and distinct levels, making it easier for users to read and find relevant information;

[0025] Restrictions include:

[0026] Content limitation: Question scenario extraction and question-answer pair generation must be based solely on the text of the provided materials. Other content unrelated to the given materials must not be involved to ensure the pertinence and relevance of the question-answer pairs.

[0027] Format requirements: The output must strictly follow the given format and must not be changed at will to ensure the standardization and consistency of the question and answer pairs, and facilitate subsequent organization, storage and use.

[0028] Furthermore, the written prompt words and processed professional material text data are input into the intelligent question-answering system based on the big model to generate QA question-answer pairs, including:

[0029] Set the prompt words of the intelligent question answering system to use the json format converter to convert the input text into json format output;

[0030] The QA question-answer pairs generated by the large model are output in JSON format as a string array.

[0031] Furthermore, a matching relationship is established between the generated QA question-answer pairs and the corresponding professional information text data, and the data is entered into the pre-created QA form template, specifically including:

[0032] The QA table template is based on an online multidimensional table plug-in and is created by defining a dictionary through an asynchronous function.

[0033] Furthermore, a matching relationship is established between the generated QA question-answer pairs and the corresponding professional information text data, and the data is entered into the pre-created QA form template, specifically including:

[0034] The QA form template includes three index columns: "Question", "Answer" and "Original File Data"; and has a unique URL.

[0035] According to a second aspect, an embodiment provides a large-scale model-based railway construction project QA question-answer pair generation system, the system comprising:

[0036] Data collection and processing module, used to collect and process professional text data in the field of railway construction engineering;

[0037] A prompt word writing module, used to write prompt words for question input according to preset prompt word writing rules;

[0038] The big model processing module is used to input the written prompt words and processed professional material text data into the big model-based intelligent question-answering system to generate QA question-answer pairs;

[0039] The result entry module is used to establish a matching relationship between the generated QA question-answer pairs and the corresponding professional information text data, and enter them into the pre-created QA form template.

[0040] According to a third aspect, an embodiment provides an electronic device, the device comprising: a processor and a memory;

[0041] The memory is used to store one or more program instructions;

[0042] The processor is used to run one or more program instructions to execute the steps of the method for generating QA question and answer pairs for railway construction projects based on a large model as described in any of the above items.

[0043] According to the fourth aspect, an embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for generating QA question-answer pairs for railway construction projects based on a large model as described in any one of the above items are implemented.

[0044] This application provides a method and system for generating QA pairs for railway construction projects based on a large model, which has the following beneficial effects:

[0045] (1) Based on the professional data of railway construction engineering, it can quickly and conveniently sort out and generate questions and answers of the corresponding content, forming a structured knowledge system, and improving the existing traditional rule-based QA question-answer pair generation method, which is heavily dependent on manual writing of specific question-answer rules, has narrow question coverage, is difficult to match answers, and consumes a lot of manpower and time costs.

[0046] (2) The unstructured professional text data of railway construction projects is automatically transcribed into structured QA pairs using large model technology, which significantly improves the accuracy of knowledge base recall. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of a method for generating QA question-answer pairs for railway construction projects based on a large model, provided by one embodiment of the present invention;

[0048] Figure 2 A flowchart of a specific implementation of a method for generating QA question-answer pairs for railway construction projects based on a large model, provided by one embodiment of the present invention;

[0049] Figure 3An example of a QA table template in a method for generating QA questions and answers for railway construction projects based on a large model, provided by one embodiment of the present invention;

[0050] Figure 4 An example of a QA question-answer pair input result in a method for generating QA question-answer pairs for a railway construction project based on a large model provided by one embodiment of the present invention;

[0051] Figure 5 This is a diagram showing the overall architecture of a large-scale model-based railway construction project QA question-answer pair generation system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0053] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0054] The first embodiment of the present invention provides a method for generating QA pairs for railway construction projects based on a large model. Figure 1 and Figure 2 Provide detailed explanation.

[0055] like Figure 1 As shown, in step S100, professional text data in the field of railway construction engineering is collected and processed.

[0056] Collection and preparation of professional materials: In this embodiment, professional materials include unstructured texts such as professional technical standards, specifications, question banks, and audit manuals for railway construction projects. The text formats include but are not limited to html, xml, doc, docx, txt, pdf, csv, and xlsx. After uploading professional materials from the system web, a URL (Uniform Resource Locator) corresponding to each professional material is generated.

[0057] Text data processing: In this embodiment, the URL of the uploaded data is obtained as an input item, and the system "file reading" plug-in API is called to output the text content of the data in the form of a string.

[0058] like Figure 1 As shown, in step S200, prompt words for question input are written according to preset prompt word writing rules.

[0059] Prompt word writing: In this embodiment, the system "text processing" plug-in API is called to write scene prompt words and output them in the form of string data.

[0060] The scene prompts include three parts: "role positioning", "skill requirements" and "restrictions".

[0061] The rules are written as follows:

[0062] 1) Role positioning

[0063] Experienced Q&A extraction experts in the railway construction engineering field: They must possess extensive knowledge and experience in the railway engineering field, be able to accurately understand professional text content, accurately identify problem scenarios, and efficiently generate question-and-answer pairs that meet actual needs.

[0064] 2) Skill requirements

[0065] Includes "extracting problem scenarios" and "generating question-answer pairs".

[0066] Extract the problem scenario:

[0067] Comprehensive and detailed: It is necessary to comprehensively and deeply explore all possible problem scenarios in the text, without missing any potential problems, and ensure the comprehensiveness of the problem scenarios.

[0068] Segmented analysis and processing: When faced with lengthy and complex professional texts, they should be split into several paragraphs or parts and analyzed one by one in order to more accurately grasp the problem scenarios of each part and avoid inaccurate extraction of problem scenarios due to the complexity of the overall content.

[0069] Generate question-answer pairs:

[0070] Concise and clear: For each problem scenario, the answer should be concise and clear, directly hitting the core of the problem, avoiding redundant information, and ensuring that users can quickly understand and obtain the required information.

[0071] Clear format segmented output: Questions and answers are presented in segments according to a certain format, making the structure clear and the levels distinct, making it easier for users to read and find relevant information.

[0072] 3) Restrictions

[0073] Content limitation: Question scenarios and question-answer pair generation are performed based solely on the text of the provided materials. Other content unrelated to the given materials must not be involved to ensure the pertinence and relevance of the question-answer pairs.

[0074] Format requirements: The output must strictly follow the given format and must not be changed at will to ensure the standardization and consistency of the question and answer pairs, and facilitate subsequent organization, storage and use.

[0075] Here is an example:

[0076] 1. Role

[0077] You are a senior expert in railway engineering QA question-answer pair extraction, skilled at accurately extracting problem scenarios from professional texts and generating concise and easy-to-understand QA question-answer pairs.

[0078] 2. Skills

[0079] Skill 1: Extracting Problem Scenarios

[0080] 1. Read the text carefully and fully explore all possible problem scenarios without missing any details.

[0081] 2. If the text is complex, you can analyze it step by step by paragraph to ensure the accuracy of problem scenario extraction.

[0082] Skill 2: Generate QA Question-Answer Pairs

[0083] 1. For each question scenario, give a concise answer of no more than XX words. The answer should be close to reality and consistent with daily conversation habits.

[0084] 2. Output QA question and answer pairs in a clear format for easy reading and understanding.

[0085] III. Restrictions

[0086] It only extracts question scenarios and generates QA question-answer pairs based on the given text content, without involving other irrelevant content.

[0087] The output must be in the given format and cannot be changed at will.

[0088] like Figure 1As shown, in step S300, the written prompt words and the processed professional material text data are input into the intelligent question-answering system based on the large model to generate QA question-answer pairs.

[0089] Large model processing: In this embodiment, the system "large model" component API is called, the output items of step S100 and step S200 are used as the input items of the component, and the large model system prompt words are set to "json format converter, good at converting input text into json format output"; the QA questions and answers (questions and answers) generated by the large model are output in the form of a string array in json format.

[0090] The large model used in this embodiment is a publicly open large language model, which is called through the open API interface of the large language model. The model architecture generally includes an input processing layer, a feature extraction layer, a knowledge retrieval layer, an answer generation layer, an answer sorting layer, an output processing layer, and the like.

[0091] like Figure 1 As shown, in step S400, a matching relationship is established between the generated QA question-answer pairs and the corresponding professional material text data, and the data is entered into a pre-created QA form template.

[0092] Pre-QA table template creation: In this embodiment, the QA table template is based on the system online multi-dimensional table plug-in and is created by defining a dictionary through an asynchronous function; Figure 3 As shown, it contains three index columns: "Question", "Answer", and "Original File Data"; and has a unique URL (Uniform Resource Locator).

[0093] Data conversion and entry: In this embodiment, a list is initialized through program development, and the output of step S400 (questions and answers) is paired with the output of step S100 (text content). These pairs are traversed to obtain the URL of the created QA form template, and all the generated data of the QA form are entered one by one, such as Figure 4 shown.

[0094] Corresponding to the above-disclosed method for generating QA question-answer pairs for railway construction projects based on a large model, the embodiment of the present invention further discloses a system for generating QA question-answer pairs for railway construction projects based on a large model, such as Figure 5 As shown, it specifically includes:

[0095] Data collection and processing module, used to collect and process professional text data in the field of railway construction engineering;

[0096] A prompt word writing module, used to write prompt words for question input according to preset prompt word writing rules;

[0097] The big model processing module is used to input the written prompt words and processed professional material text data into the big model-based intelligent question-answering system to generate QA question-answer pairs;

[0098] The result entry module is used to establish a matching relationship between the generated QA question-answer pairs and the corresponding professional information text data, and enter them into the pre-created QA form template.

[0099] It should be noted that for the detailed description of the large-scale model-based railway construction project QA question-answer pair generation system provided in an embodiment of the present invention, reference can be made to the relevant description of the large-scale model-based railway construction project QA question-answer pair generation method provided in an embodiment of the present application, which will not be repeated here.

[0100] In addition, an embodiment of the present invention also provides an electronic device, comprising: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of a method for generating QA question and answer pairs for railway construction projects based on a large model as described in any of the above items.

[0101] It should be noted that for the detailed description of an electronic device provided in an embodiment of the present invention, reference can be made to the relevant description of a method for generating QA question and answer pairs for railway construction projects based on a large model provided in an embodiment of the present application, which will not be repeated here.

[0102] In addition, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for generating QA question-and-answer pairs for railway construction projects based on a large model as described in any of the above items are implemented.

[0103] It should be noted that for a detailed description of a computer-readable storage medium provided in an embodiment of the present invention, reference can be made to the relevant description of a method for generating QA question and answer pairs for railway construction projects based on a large model provided in an embodiment of the present application, which will not be repeated here.

[0104] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer program. When all or part of the functions in the above embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and saved in the memory of the local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be implemented.

[0105] The above examples are used to illustrate the present invention, which are only used to help understand the present invention and are not intended to limit the present invention. Those skilled in the art can make several simple deductions, modifications or substitutions based on the concept of the present invention.

Claims

1. A method for generating QA pairs for railway construction projects based on a large model, characterized by: The method comprises: Collect and process professional text data in the field of railway construction engineering; Write prompt words for question input according to the preset prompt word writing rules; Input the written prompt words and processed professional material text data into the intelligent question-answering system based on the big model to generate QA question-answer pairs; A matching relationship is established between the generated QA question-answer pairs and the corresponding professional material text data, and they are entered into the pre-created QA form template.

2. The method for generating QA question-answer pairs for railway construction projects based on a large model according to claim 1, characterized in that: Collect and process professional text data in the field of railway construction engineering, including: The professional materials collected and uploaded include unstructured texts of railway construction engineering professional technical standards, specifications, question banks, and audit manuals.

3. The method for generating QA pairs for railway construction projects based on a large model as claimed in claim 1, characterized in that: Collect and process professional text data in the field of railway construction engineering, including: Generate a corresponding URL for each uploaded professional document, take the URL of the uploaded professional document as input, and use the text reading plug-in API to output the text content of the professional document in the form of string data.

4. The method for generating QA pairs for railway construction projects based on a large model according to claim 1, characterized in that: Write prompt words for question input according to the preset prompt word writing rules, including: The prompt words include three parts: role positioning, skill requirements and restrictions; Role positioning, including: Experienced Q&A extraction experts in the railway construction engineering field: This requires extensive knowledge and experience in the railway construction engineering field, the ability to accurately understand professional text content, pinpoint problem scenarios, and efficiently generate question-answer pairs that meet actual needs; Skill requirements include extracting problem scenarios and generating question-answer pairs; Extract the problem scenario: Comprehensive and meticulous: This requires a comprehensive and in-depth exploration of all possible problem scenarios in the text, without missing any potential problems, ensuring the comprehensiveness of the problem scenarios; Segmented analysis and processing: When faced with lengthy and complex professional texts, they should be broken down into several paragraphs or sections and analyzed one by one to more accurately grasp the problem scenarios in each section and avoid inaccurate problem scenario extraction due to the complexity of the overall content; Generate question-answer pairs: Concise and clear: For each question scenario, the answer should be concise and clear, directly hitting the core of the question, avoiding redundant information, and ensuring that users can quickly understand and obtain the required information; Clear format segmented output: Questions and answers should be presented in segments according to a certain format, with a clear structure and distinct levels, making it easier for users to read and find relevant information; Restrictions include: Content limitation: Question scenario extraction and question-answer pair generation must be based solely on the text of the provided materials. Other content unrelated to the given materials must not be involved to ensure the pertinence and relevance of the question-answer pairs. Format requirements: The output must strictly follow the given format and must not be changed at will to ensure the standardization and consistency of the question and answer pairs, and facilitate subsequent organization, storage and use.

5. The method for generating QA question-answer pairs for railway construction projects based on a large model according to claim 1, characterized in that: The written prompt words and processed professional material text data are input into the intelligent question-answering system based on the big model to generate QA question-answer pairs, including: Set the prompt words of the intelligent question answering system to use the json format converter to convert the input text into json format output; The QA question-answer pairs generated by the large model are output in JSON format as a string array.

6. The method for generating QA pairs for railway construction projects based on a large model according to claim 1, characterized in that: Establish a matching relationship between the generated QA question-answer pairs and the corresponding professional information text data, and enter them into the pre-created QA form template, including: The QA table template is based on an online multi-dimensional table plug-in and is created by defining a dictionary through an asynchronous function.

7. The method for generating QA pairs for railway construction projects based on a large model as claimed in claim 1, characterized in that: Establish a matching relationship between the generated QA question-answer pairs and the corresponding professional information text data, and enter them into the pre-created QA form template, including: The QA form template includes three index columns: "Question", "Answer" and "Original File Data"; and has a unique URL.

8. A large-scale model-based railway construction project QA question-answer generation system, characterized by: The system comprises: Data collection and processing module, used to collect and process professional text data in the field of railway construction engineering; A prompt word writing module, used to write prompt words for question input according to preset prompt word writing rules; The big model processing module is used to input the written prompt words and processed professional material text data into the big model-based intelligent question-answering system to generate QA question-answer pairs; The result entry module is used to establish a matching relationship between the generated QA question-answer pairs and the corresponding professional information text data, and enter them into the pre-created QA form template.

9. An electronic device, characterized in that: The device includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is used to run one or more program instructions to execute the steps of a method for generating QA question-answer pairs for railway construction projects based on a large model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for generating QA question-answer pairs for railway construction projects based on a large model as described in any one of claims 1 to 7.

Images