Process industry text knowledge extraction data set automatic construction method and system

Abstract

The invention belongs to but not limited to the technical field of text extraction, and particularly relates to an automatic construction method and system for a process industry text knowledge extraction dataset. Firstly, based on a word segmentation tool and a TF-IDF algorithm, an unlabeled text in the process industry field is converted into a large language model fine tuning dataset, and a large language model is subjected to fine tuning; the system has specific knowledge of the field; then, generating an initial knowledge extraction sample set with confidence through multi-stage exploration-utilization ratio adjustment based on the fine-tuned large language model; and finally, through a confidence-based large-scale sample set error detection and correction method, obtaining a sample data set which is finally used for domain knowledge extraction model training. Through the low-cost and high-efficiency industrial text knowledge extraction sample set construction and application method, the input-output ratio of resources is increased to the maximum extent, and the construction and application efficiency of an industrial safety knowledge extraction algorithm is improved.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of text extraction technology, and in particular relates to a method and system for automatically constructing a process industry text knowledge extraction data set. Background Art

[0002] With the application of information technology in process industry production management, a large amount of digital text knowledge containing production process optimization, product quality improvement, etc. to improve corporate decision-making and improvement has shown explosive growth. At the same time, these text knowledge faces the characteristics of large volume but low quality density. Therefore, how to improve the level of automatic extraction of industrial text knowledge has become a new research hotspot. Among them, knowledge extraction methods based on machine learning, especially deep learning algorithms, have achieved amazing achievements in the public domain. They achieve the description of specific pattern features of text knowledge through iterative learning, so they have adaptability and accuracy in complex scenarios. However, in the specific field application of process industry, high-quality supervised learning methods are highly dependent on. In the specific field knowledge extraction application, due to the lack of professional knowledge annotation personnel, high-quality model training data sets are usually difficult to obtain. Therefore, a high-quality and efficient model training sample set construction method is urgently needed to solve the problem that the relevant algorithms are difficult to apply in the industrial field due to the lack of model training data sets, and to improve the development and application efficiency of industrial safety knowledge extraction models.

[0003] In view of the above analysis, the technical problems that urgently need to be solved in the existing technology are: in the application of knowledge extraction in specific fields of process industries, which is highly dependent on high-quality supervised learning methods, due to the lack of labeling personnel with professional knowledge, high-quality model training data sets are usually difficult to obtain. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a method and system for automatically constructing a process industry text knowledge extraction dataset. Based on the original industrial text data, a large language model industrial fine-tuning sample set is automatically constructed and the large language model is fine-tuned. By utilizing the complex text understanding and reasoning capabilities of the large language model, the fine-tuned large language model is applied to the construction process of the industrial text knowledge extraction dataset. Under the premise of reducing labor and resource consumption, the industrial knowledge extraction dataset is quickly and efficiently constructed, providing basic training data support for the research and application of localized industrial safety knowledge extraction models.

[0005] The present invention is implemented as follows: a method for automatically constructing a process industry text knowledge extraction data set, comprising:

[0006] S101: Select a suitable large language model according to the application scenario, segment the original unlabeled process industrial text into text blocks, generate a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tune the large language model;

[0007] S102, defining the entity and relationship types to be extracted, defining a large language model prompt word template, and filling the text blocks after the original unlabeled document is segmented into the template according to the prompt word template format according to the sentence granularity to form a process industry knowledge extraction problem to be input into the large language model;

[0008] S103, gradually adjusting the exploration-utilization ratio of the large language model in order from small to large, and performing frequency statistics on the corresponding entity and relationship instances identified by the large language model based on different exploration-utilization ratio parameters, and calculating the confidence of the corresponding entity and relationship according to the occurrence frequency to form an initial knowledge extraction sample set;

[0009] S104, according to application requirements, different confidence thresholds are set, entities and relationship items below the confidence level in the initial knowledge extraction sample set are screened out through rules, and they are manually verified and corrected to obtain the final sample data set;

[0010] S105, designing a knowledge extraction model for industrial field applications, and converting the generated final sample set into the format required by the model to train and test the model.

[0011] Furthermore, a small number of documents such as operating procedures, emergency manuals, operation cards, hazard and operability analysis (HAZOP) reports, and failure mode and effects analysis (FMEA) reports in the process industry are selected according to the category coverage principle, and special characters are removed from the text characters to construct a large model fine-tuning dataset;

[0012] The basic process of constructing the fine-tuning dataset is as follows: the text sequence blocks are divided into sentences or paragraphs, the original text sequence blocks are segmented using Chinese word segmentation tools, and high-importance phrases are selected as the answers of the large model fine-tuning dataset according to the word frequency-inverse document frequency (TF-IDF) method, and the high-importance phrases are replaced in the original text blocks using special characters, and the replaced text blocks are used as the question context of the large model fine-tuning dataset. The question and answer pairs are composed into a large model fine-tuning dataset according to the large model fine-tuning dataset format, and the large model is fine-tuned based on the fine-tuning dataset to obtain a large language model with process industry domain knowledge.

[0013] Furthermore, safety knowledge entities and relationship types are designed according to the application scenarios of industrial safety knowledge, where entity types include but are not limited to knowledge types related to system safety analysis such as equipment, function, accident, cause, status, etc., and the relationship types are the relationships between entities, including but not limited to monitoring, controlling, storing, generating, causing, running on, etc., which represent system connections and business-related relationships;

[0014] The prompt word template of the large language model is defined according to the defined entity and relationship types, which is a question template input in the form of a dialogue when the large model performs security knowledge extraction, and contains fixed prompt words to prompt the large model what type of knowledge extraction task it needs to complete and the original text sequence placeholder, so as to finally generate the input question for the large model to perform knowledge extraction;

[0015] Furthermore, the large language model is explored by gradually adjusting the temperature from low to high to perform multiple rounds of entity and relationship recognition in two stages, and the frequency of entity and relationship pseudo-labels in the recognition results is statistically calculated to obtain the confidence scores of entities and relationships.

[0016] The entity and relationship extraction is performed in two stages. First, an entity extraction question is generated according to a prompt word template, and the corresponding entity is parsed based on the generated text data in a specific format; then the identified head entity and tail entity are filled with the relationship prompt word template to identify a specific predefined relationship in the original text sequence, and if it exists, the head entity, relationship and tail entity triple information is output to form an initial knowledge extraction model training sample set;

[0017] Furthermore, the large model extraction results are verified based on rules or manual methods, and entities or relationship instances below a specific confidence level are selected for correction according to a confidence threshold, thereby greatly improving the efficiency of data set generation; first, based on the rules, it is checked whether the entity is in the original text sequence, and the entities that do not exist in the original sequence are filtered; then, the filtered entities and corresponding relationship instances are checked according to the confidence threshold, and the sample data below the confidence threshold will be manually verified to see whether the relationship is correct and whether the entity boundary is accurate, and the confidence after verification will be modified. Optionally, the verification threshold can be gradually increased until the data set quality meets the requirements; the calibrated entity and relationship data are integrated with the original text sequence data to form the final entity and relationship extraction data set.

[0018] Furthermore, the pipeline or joint mode can be selected according to the actual knowledge extraction needs. There is no restriction on the selected model. The final knowledge extraction data set format generated by the tool is converted according to the input requirements of different models.

[0019] Another object of the present invention is to provide a system for automatically constructing a process industry text knowledge extraction data set to implement the process industry text knowledge extraction data set automatic construction method, comprising:

[0020] The large language model fine-tuning module selects a suitable large language model according to the application scenario, divides the original unlabeled process industrial text into text blocks, generates a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tunes the large language model;

[0021] The extraction question generation module defines the entity and relationship types to be extracted, defines the large language model prompt word template, and fills the text blocks after the original unlabeled document is segmented into the template according to the sentence granularity according to the prompt word template format to form the process industry knowledge extraction question to be input into the large language model;

[0022] The knowledge extraction sample set acquisition module gradually adjusts the exploration-utilization ratio of the large language model in order from small to large, and counts the frequency of the corresponding entity and relationship instances identified by the large language model based on different exploration-utilization ratio parameters, and calculates the confidence of the corresponding entity and relationship according to the frequency of occurrence to form an initial knowledge extraction sample set;

[0023] The final sample data set acquisition module sets different confidence thresholds according to application requirements, filters out entities and relationship items below the confidence level in the initial knowledge extraction sample set through rules, and manually verifies and corrects them to obtain the final sample data set;

[0024] The knowledge extraction model acquisition module designs the knowledge extraction model for industrial field application and converts the generated final sample set into the format required by the model for model training and testing.

[0025] Another object of the present invention is to provide a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for automatically constructing a process industry text knowledge extraction data set.

[0026] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the method for automatically constructing a process industry text knowledge extraction data set.

[0027] Another object of the present invention is to provide an information data processing terminal, which includes the automatic construction system for process industry text knowledge extraction data set.

[0028] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0029] First, the present invention provides a method for automatically constructing a large model domain knowledge fine-tuning dataset based on TF-IDF. The unlabeled process industry text is segmented by a word segmentation tool, and phrases with noun parts of speech are screened out based on parts of speech. The high-value phrase information in a specific text sequence block is calculated by the TF-IDF method, and it is processed by a masking method to construct a knowledge extraction large language model fine-tuning dataset. The automated knowledge extraction fine-tuning dataset generation method provides a basis for the large language model to have domain-specific knowledge of the application scenario. Moreover, the construction of the fine-tuning dataset does not require human intervention, which improves the efficiency of the reasoning and analysis capabilities of the large language model under the background knowledge requirements of a specific domain.

[0030] The present invention provides a method for automatically constructing an industrial text knowledge extraction dataset based on a fine-tuned large language model. By dynamically adjusting the exploration-utilization ratio of the large language model, industrial knowledge is automatically extracted based on the fine-tuned large language model in a two-stage manner of entity extraction and relationship extraction, and the confidence of text knowledge extraction is measured according to the frequency of occurrence of specific results. Finally, low-confidence sample data is quickly screened based on the confidence to reduce the number of manual secondary verifications, thereby solving the problem of extremely high technical and time costs in the construction of domain knowledge extraction datasets and laying the foundation for the development of domain knowledge application models.

[0031] Second, the expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0032] The automatic construction method of industrial text knowledge extraction dataset based on fine-tuning large language models can greatly improve the utilization rate of domain knowledge while significantly reducing manpower and technical costs, and provide basic information extraction support for the intelligent upgrading and transformation of enterprises.

[0033] The technical solution of the present invention fills the technical gap in the industry at home and abroad: by automatically constructing a domain knowledge fine-tuning dataset for a large language model, a feasible solution is provided for the large language model to apply its powerful reasoning and analysis capabilities in specific fields. On this basis, by automatically generating a domain knowledge extraction dataset through a fine-tuning large language model with domain knowledge, basic data support is provided for the development of localized small models.

[0034] Third, the technical solution of the present invention solves many key problems in the prior art in industrial applications and has achieved significant technical progress in practical applications. Specifically, it is reflected in the following aspects:

[0035] 1. Problems with existing technologies: The extraction and organization of textual knowledge in process industries often relies on a large amount of manual annotation, which is not only inefficient but also prone to inconsistent annotations, resulting in low quality model training data. In addition, traditional rule- and template-based knowledge extraction methods have limited processing capabilities for complex sentence structures and cannot accurately extract the rich and diverse entities and relationships in industrial scenarios. Therefore, existing industrial knowledge extraction methods are difficult to efficiently and accurately obtain knowledge information in process industries in complex and changing scenarios.

[0036] 2. Solution: The present invention proposes a method for automatically constructing a data set of process industry text knowledge extraction based on a large language model, which can automatically complete the task of extracting entities and relationships from unlabeled text. By fine-tuning the large language model and combining it with an adaptive exploration-exploitation ratio strategy, the model can better adapt to the knowledge extraction needs in the process industry. By statistically analyzing the frequency and confidence, a high-quality initial knowledge sample set is formed, and the accuracy of the extracted data is further ensured through manual verification. This method effectively solves the problems of time-consuming manual annotation, poor annotation consistency, and the difficulty of traditional models in handling complex sentence structures.

[0037] 3. Significant technological progress:

[0038] Improved automation: Compared with the existing technology that requires a lot of manual participation, the present invention realizes the fully automatic construction from unlabeled text to knowledge data set, greatly improving the efficiency of data set construction and saving a lot of labor costs and time.

[0039] Improvement of data quality: By introducing a combination of confidence calculation and manual verification, the present invention ensures the high quality and accuracy of the knowledge extraction data set, so that the subsequent trained knowledge extraction model has better performance and generalization ability.

[0040] Enhanced model generalization ability: This invention uses a large language model for fine-tuning and combines it with an exploration-exploitation ratio strategy to improve the model's ability to understand complex texts in process industry scenarios, making the model more robust in diverse and complex industrial applications.

[0041] Efficient knowledge extraction process: By designing specific prompt word templates, the logic of extracting entities and relationships in the large language model is clarified, making the knowledge extraction process more efficient and clear, and suitable for diverse scenarios in the process industry.

[0042] The technical solution of the present invention effectively improves the efficiency and accuracy of extracting process industry text knowledge in industrial applications, promotes the advancement of automated processing of industrial knowledge, helps to solve the shortcomings of traditional technologies in efficiency and accuracy, and has significant technical progress and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a method for automatically constructing a process industry text knowledge extraction data set provided by an embodiment of the present invention;

[0044] Figure 2 It is a flowchart of the construction and application of industrial knowledge extraction data set based on a large model provided by an embodiment of the present invention;

[0045] Figure 3 It is a flow chart of the principle of constructing an industrial knowledge extraction data set based on a large model provided by an embodiment of the present invention;

[0046] Figure 4 It is a structural diagram of a system for automatically constructing a process industry text knowledge extraction data set provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] like Figure 1 As shown, the method for automatically constructing a process industry text knowledge extraction data set provided by an embodiment of the present invention includes:

[0049] S101: Select a suitable large language model according to the application scenario, segment the original unlabeled process industrial text into text blocks, generate a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tune the large language model;

[0050] S102, defining the entity and relationship types to be extracted, defining a large language model prompt word template, and filling the text blocks after the original unlabeled document is segmented into the template according to the prompt word template format according to the sentence granularity to form a process industry knowledge extraction problem to be input into the large language model;

[0051] S103, gradually adjusting the exploration-utilization ratio of the large language model in order from small to large, and performing frequency statistics on the corresponding entity and relationship instances identified by the large language model based on different exploration-utilization ratio parameters, and calculating the confidence of the corresponding entity and relationship according to the occurrence frequency to form an initial knowledge extraction sample set;

[0052] S104, according to application requirements, different confidence thresholds are set, entities and relationship items below the confidence level in the initial knowledge extraction sample set are screened out through rules, and they are manually verified and corrected to obtain the final sample data set;

[0053] S105, designing a knowledge extraction model for industrial field applications, and converting the generated final sample set into the format required by the model to train and test the model.

[0054] The method for automatically constructing a process industry text knowledge extraction dataset provided by an embodiment of the present invention realizes the automatic extraction of entity and relationship knowledge from unlabeled process industry text by combining the fine-tuning of a large language model with the knowledge extraction process. First, in step S101, a suitable large language model is selected based on the application scenario, the unlabeled process industry text is segmented, the long text is divided into smaller text blocks, and a fine-tuning dataset of the large language model is generated using a word segmentation tool and a TF-IDF algorithm. In this way, the large language model is customized and fine-tuned to adapt to specific process industry scenarios, thereby improving its recognition ability in knowledge extraction.

[0055] Next, in step S102, based on specific application requirements, define the entity and relationship types that need to be extracted, such as equipment and process parameters in the process flow. When defining the prompt word template, a series of question templates are designed based on the input format of the large language model, and the preprocessed text blocks are filled into these templates to form a complete knowledge extraction question. This step guides the large language model to targeted knowledge extraction tasks, ensuring that the model can focus on specific entity and relationship recognition according to the sentence granularity during input processing.

[0056] In step S103, the exploration-utilization ratio of the large language model is adjusted, that is, the exploration and robustness of the model are adjusted during the knowledge extraction process. Based on different exploration-utilization ratios, the model counts the frequency of the extracted entities and relationships, and calculates their confidence by the frequency, thereby generating an initial knowledge extraction sample set. The core of this process is to find the best entity and relationship recognition method through the adjustment of exploration parameters, and screen out high-frequency and high-confidence extraction results.

[0057] Finally, steps S104 and S105 further optimize the extraction results. In S104, the confidence threshold is set according to the requirements of the application scenario, and the entities and relationships below the threshold are manually verified and corrected to ensure the quality and accuracy of the data set. Finally, the sample data set after manual verification is used to design the knowledge extraction model, and the model is trained and tested in S105 by converting the data set into a format that adapts to the model. In this way, the process industry knowledge extraction data set can be efficiently constructed, and the accuracy and stability of the model in actual industrial applications can be improved.

[0058] With the application of information technology in process industry production management, a large amount of digital text knowledge containing production process optimization, product quality improvement, etc. to improve corporate decision-making and improvement has exploded. At the same time, these text knowledge faces the characteristics of large volume but low quality density. Therefore, how to improve the level of automatic extraction of industrial text knowledge has become a problem that needs to be solved urgently. Among them, knowledge extraction methods based on machine learning, especially deep learning algorithms, have achieved amazing achievements in the public domain. However, in the specific field applications of process industries, they are highly dependent on high-quality supervised learning methods. In the specific field knowledge extraction applications, due to the lack of annotation personnel with professional knowledge, the lack of high-quality model training data sets affects the development and application efficiency of knowledge extraction models in industrial fields.

[0059] The present invention proposes a method for automatically constructing a data set for textual knowledge extraction in process industries based on a large model. A data set is fine-tuned by generating a large language model based on the original documents already in the factory, and a high-quality data set for textual knowledge extraction in process industries is automatically constructed based on the fine-tuned large language model. This method solves the problem that the relevant algorithms are difficult to effectively evaluate and apply due to the lack of data sets for knowledge extraction in the field of process industries.

[0060] In order to achieve the above objectives, in the first aspect, the present invention provides a method for automatically constructing a large model domain knowledge fine-tuning dataset based on TF-IDF, such as Figure 2 As shown, including:

[0061] Select a suitable large language model according to the application scenario, segment the original unlabeled process industry text into text blocks, generate a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tune the large language model; specifically, the following steps are included:

[0062] Select a suitable large language model according to the application scenario. Due to the differences in the basic corpus used in the large model pre-training stage and the differences in the large model architecture, different large language models have different performances in specific tasks. Here we focus on process industry knowledge extraction, in which a large amount of text is described in Chinese, so we choose to use Qwen2-72B as the basic fine-tuning model.

[0063] Produce a dataset for fine-tuning the application of large models in the field; select a small number of representative documents such as operating procedures, emergency manuals, operation cards, hazard and operability analysis (HAZOP) reports, failure mode and effects analysis (FMEA) reports, etc. generated in the process of process industry production management according to the category coverage principle, and extract the text information in the documents through code tools, and remove special characters such as spaces, tabs, dollar signs, etc., and slice the cleaned document information into sentences or paragraphs to form a basic corpus for fine-tuning the large model.

[0064] Traverse all the original untagged text sequence blocks in the basic corpus, use the JIEBA word segmentation tool to segment and identify the part of speech for each text block, and store the recognition results in JSON format; after all text sequence blocks have completed word segmentation and part of speech tagging, based on the number of text blocks and the number of times a specific phrase appears in each text block, use the TF-IDF algorithm to identify high-importance phrases. The TF-IDF formula is as follows:

[0065]

[0066] Among them, n i,j Expressing words t i The frequency of occurrence in text j, ∑ k n k,j represents the sum of the frequencies of all phrases in text j; |D| represents the total number of documents in corpus d, |{j:t i ∈d j}| means that the corpus d contains phrase t i The number of documents;

[0067] By setting the threshold, we filter out high-importance phrases as the answers to the large model fine-tuning dataset, and replace the phrases at the corresponding positions in the original expected text sequence block with special characters, eventually forming a large language model fine-tuning dataset in the following format, where each fine-tuning sample format is as follows:

[0068]

[0069] Fine-tune the large language model based on the fine-tuning dataset; Based on the large model fine-tuning sample set, the general large model can be equipped with domain knowledge, thereby improving the adaptability of domain knowledge extraction. Fine-tune the large language model using the standard LORA fine-tuning method, fine-tune the model through Hugging Face, and save the fine-tuned model.

[0070] In order to achieve the above-mentioned object, in a second aspect, the present invention provides a method for automatically constructing a process industry text knowledge extraction dataset based on a fine-tuned large model, comprising:

[0071] Define the entity and relationship types to be extracted, define the large language model prompt word template, and fill the text blocks after the original unlabeled document is segmented into the template according to the prompt word template format according to the sentence granularity to form the process industry knowledge extraction problem to be input into the large language model;

[0072] The exploration-utilization ratio of the large language model is gradually adjusted in order from small to large, and the frequency of the corresponding entity and relationship instances identified by the large language model is counted based on different exploration-utilization ratio parameters, and the confidence of the corresponding entity and relationship is calculated according to the frequency of occurrence to form an initial knowledge extraction sample set;

[0073] According to application requirements, different confidence thresholds are set, and entities and relationship items with confidence levels below the initial knowledge extraction sample set are screened out through rules. The final sample data set is obtained by manually verifying and correcting them.

[0074] Specifically, if Figure 3 As shown, the following steps are included:

[0075] Industrial knowledge extraction dataset generation based on fine-tuning large language model; by cutting the original industrial text prediction library into sentence units, and storing the cut corpus in the following JSON format for subsequent dataset production process;

[0076] {

[0077] "sentence":"The pressure of the absorption tower is determined by the outlet pressure of the air compressor and is controlled by a pressure control instrument"

[0078] "entities":[]

[0079] }

[0080] According to the predefined entity and relationship types, the entity types included here are: entity types = [equipment, function, accident, cause, status], and the prompt template for industrial knowledge extraction of the fine-tuned large language model is designed. Here, knowledge extraction is performed in the common dialogue mode, and the entity extraction template is used as an example for explanation. The template for knowledge extraction of the large language model is as follows:

[0081] template="You are a text analysis expert. Please extract all named entities from a given sentence according to the given entity type. The given entity type list to be extracted is: [{entitytypes}], and the given sentence is: {sentence}. Please output all the identified named entity information in the following format: [{"index":[token1index,…,tokenN index],"type":"entity type"},…]". The identified named entities must exist in the original sentence. Please do not perform excessive reasoning. "

[0082] Based on the above template, the text cut into sentences is filled into the large language model named entity extraction template and input into the large language model. The output result is parsed to obtain the following results:

[0083] {

[0084] "sentence":"The pressure of the absorption tower is determined by the outlet pressure of the air compressor and is controlled by a pressure control instrument"

[0085] "entities":[{"index":[0,2],"type":"device"},{"index":[7,8,9],"type":"device"},…]

[0086] }

[0087] The knowledge extraction data set is generated based on the exploration-utilization ratio parameter of the large language model; in the process of entity extraction using the fine-tuned large model, in order to verify the correctness of the model output results, the exploration-utilization ratio parameter (temperature) of the large language model is gradually adjusted in the range of 0.0 to 1.0, where the larger the temperature value, the stronger the randomness. If the model can obtain the same output as that under the condition of smaller randomness under the condition of greater randomness, it means that the large language model has a more accurate feature judgment for the recognized entity, which will also be directly related to the correctness of the final output result of the model. The frequency statistics of entity recognition results under different temperature values ​​are performed, and the confidence of the named entity results recognized by the model is calculated based on the same number of sample experiments. The confidence calculation formula is:

[0088]

[0089] in, For entity e i The number of occurrences, M is the total number of times a single sample is identified. All sentences to be identified in the sample set are extracted through the fine-tuned large language model in a cyclic manner, and the results are summarized.

[0090] Calibration of the knowledge extraction dataset based on rules and manual verification; according to the recognition results of the fine-tuned large language model, first filter out the entity recognition results that are not in the original text sequence through the basic rule verification method, and set them as the identification that needs manual verification, and then filter the entities that pass the rule verification based on the confidence level, filter out the entity recognition results with lower confidence levels, and mark them as requiring manual review. During the manual review stage, correct or delete the erroneous entity recognition results, and set the confidence level of the modified retained results to 1.0, and no manual calibration is required again. In this way, the confidence level requirements of the recognition results are gradually improved until the requirements are met. The final corrected knowledge extraction sample set is saved to obtain the initial version of the basic dataset for training and testing process industry knowledge extraction models.

[0091] Finally, the existing knowledge extraction model can be designed or reused based on the needs of industrial field applications, and the final sample set generated can be converted into the format required by the model for training and testing, thereby greatly improving the level of automation of knowledge extraction in the process industry and improving the level of intelligence of process industry management.

[0092] like Figure 4 As shown, the automatic construction system of process industry text knowledge extraction data set provided by the embodiment of the present invention includes:

[0093] The large language model fine-tuning module selects a suitable large language model according to the application scenario, divides the original unlabeled process industrial text into text blocks, generates a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tunes the large language model;

[0094] The extraction question generation module defines the entity and relationship types to be extracted, defines the large language model prompt word template, and fills the text blocks after the original unlabeled document is segmented into the template according to the sentence granularity according to the prompt word template format to form the process industry knowledge extraction question to be input into the large language model;

[0095] The knowledge extraction sample set acquisition module gradually adjusts the exploration-utilization ratio of the large language model in order from small to large, and counts the frequency of the corresponding entity and relationship instances identified by the large language model based on different exploration-utilization ratio parameters, and calculates the confidence of the corresponding entity and relationship according to the frequency of occurrence to form an initial knowledge extraction sample set;

[0096] The final sample data set acquisition module sets different confidence thresholds according to application requirements, filters out entities and relationship items below the confidence level in the initial knowledge extraction sample set through rules, and manually verifies and corrects them to obtain the final sample data set;

[0097] The knowledge extraction model acquisition module designs the knowledge extraction model for industrial field application and converts the generated final sample set into the format required by the model for model training and testing.

[0098] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of a method for automatically constructing a process industry text knowledge extraction data set.

[0099] An application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a method for automatically constructing a process industry text knowledge extraction data set.

[0100] An application embodiment of the present invention provides an information data processing terminal, which includes an automatic construction system for process industry text knowledge extraction data set.

[0101] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0102] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for automatically constructing a process industry text knowledge extraction dataset, characterized in that: include: S101: Select a suitable large language model according to the application scenario, segment the original unlabeled process industrial text into text blocks, generate a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tune the large language model; S102, defining the entity and relationship types to be extracted, defining a large language model prompt word template, and filling the text blocks after the original unlabeled document is segmented into the template according to the prompt word template format according to the sentence granularity to form a process industry knowledge extraction problem to be input into the large language model; S103, gradually adjusting the exploration-utilization ratio of the large language model in order from small to large, and performing frequency statistics on the corresponding entity and relationship instances identified by the large language model based on different exploration-utilization ratio parameters, and calculating the confidence of the corresponding entity and relationship according to the occurrence frequency to form an initial knowledge extraction sample set; S104, according to application requirements, different confidence thresholds are set, entities and relationship items below the confidence level in the initial knowledge extraction sample set are screened out through rules, and they are manually verified and corrected to obtain the final sample data set; S105, designing a knowledge extraction model for industrial field applications, and converting the generated final sample set into the format required by the model to train and test the model.

2. The method for automatically constructing a process industry text knowledge extraction data set according to claim 1, characterized in that: A small number of documents including operating procedures, emergency manuals, operation cards, hazard and operability analysis reports, and failure mode and effect analysis reports in the process industry are selected according to the category coverage principle, and special characters are removed from the text characters to build a large model fine-tuning dataset; The basic process of constructing the fine-tuning dataset is as follows: divide the text sequence blocks into sentences or paragraphs, use the Chinese word segmentation tool to segment the original text sequence blocks, and select high-importance phrases as the answers of the large model fine-tuning dataset according to the word frequency-inverse file frequency method, use special characters to replace the high-importance phrases in the original text blocks, and use the replaced text blocks as the question context of the large model fine-tuning dataset. The question and answer pairs are combined into a large model fine-tuning dataset according to the large model fine-tuning dataset format, and the large model is fine-tuned based on the fine-tuning dataset to obtain a large language model with process industry domain knowledge.

3. The method for automatically constructing a process industry text knowledge extraction data set according to claim 1, characterized in that: Design safety knowledge entities and relationship types according to industrial safety knowledge application scenarios, where entity types include but are not limited to equipment, function, accident, cause, and state system safety analysis-related knowledge types, and relationship types are relationships between entities, including but not limited to monitoring, controlling, storing, generating, causing, and operating types that represent system connections and business-related relationships; The prompt word template of the large language model is defined according to the defined entity and relationship types, which is a question template input in the form of a dialogue when the large model performs security knowledge extraction. It contains fixed prompt words to prompt the large model what type of knowledge extraction task it needs to complete and the original text sequence placeholder, so as to finally generate the input question for the large model to perform knowledge extraction.

4. The method for automatically constructing a process industry text knowledge extraction dataset according to claim 1, characterized in that: The large language model is explored and utilized in a two-stage recognition method of multiple rounds of entities and relationships by gradually adjusting the large language model from low to high. The confidence scores of entities and relationships are obtained by statistically calculating the frequency of occurrence of entity and relationship pseudo-labels in the recognition results. The entity and relationship extraction is performed in two stages. First, an entity extraction question is generated according to a prompt word template, and then the corresponding entity is parsed based on the generated text data in a specific format. The identified head entity and tail entity are then filled with the relation hint word template to identify a specific predefined relation in the original text sequence. If it exists, the head entity, relation and tail entity triple information is output to form the initial knowledge extraction model training sample set.

5. The method for automatically constructing a process industry text knowledge extraction data set according to claim 1, characterized in that: The method verifies the extraction results of the large model based on rules or manual methods, and selects entities or relationship instances below a specific confidence level for correction based on a confidence threshold, thereby greatly improving the efficiency of data set generation; first, based on the rules, check whether the entity is in the original text sequence, and filter the entities that do not exist in the original sequence; then, the filtered entities and corresponding relationship instances are checked according to the confidence threshold, and the sample data below the confidence threshold will be manually verified to see whether the relationship is correct and whether the entity boundary is accurate, and the confidence after verification will be modified. Optionally, the verification threshold can be gradually increased until the quality of the data set meets the requirements; the calibrated entity and relationship data are integrated with the original text sequence data to form the final entity and relationship extraction data set.

6. The method for automatically constructing a process industry text knowledge extraction data set according to claim 1, characterized in that: Either pipeline or joint mode can be selected according to the actual knowledge extraction needs. There is no restriction on the selected model. The final knowledge extraction dataset format generated by the tool is converted according to the input requirements of different models, and the domain-specific knowledge extraction model is trained based on this.

7. A system for automatically constructing a data set for extracting textual knowledge from process industries, which implements the method for automatically constructing a data set for extracting textual knowledge from process industries as claimed in any one of claims 1 to 6, characterized in that: include: The large language model fine-tuning module selects a suitable large language model according to the application scenario, divides the original unlabeled process industrial text into text blocks, generates a large language model fine-tuning dataset based on the word segmentation tool and TF-IDF algorithm, and fine-tunes the large language model; The extraction question generation module defines the entity and relationship types to be extracted, defines the large language model prompt word template, and fills the text blocks after the original unlabeled document is segmented into the template according to the sentence granularity according to the prompt word template format to form the process industry knowledge extraction question to be input into the large language model; The knowledge extraction sample set acquisition module gradually adjusts the exploration-utilization ratio of the large language model in order from small to large, and counts the frequency of the corresponding entity and relationship instances identified by the large language model based on different exploration-utilization ratio parameters, and calculates the confidence of the corresponding entity and relationship according to the frequency of occurrence to form an initial knowledge extraction sample set; The final sample data set acquisition module sets different confidence thresholds according to application requirements, filters out entities and relationship items below the confidence level in the initial knowledge extraction sample set through rules, and manually verifies and corrects them to obtain the final sample data set; The knowledge extraction model acquisition module designs the knowledge extraction model for industrial field application and converts the generated final sample set into the format required by the model for model training and testing.

8. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for automatically constructing a process industry text knowledge extraction data set.

9. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the steps of the method for automatically constructing a process industry text knowledge extraction data set.

10. An information data processing terminal, comprising the automatic construction system for process industry text knowledge extraction data set.

Images