Industrial knowledge graph construction method based on large language model and thinking chain

Abstract

The invention relates to the technical field of natural language processing, and particularly discloses an industrial knowledge graph construction method based on a large language model and a thinking chain, which realizes comprehensive and accurate collection of information from a wider data source through a multi-source data acquisition system, and greatly improves the breadth and depth of data acquisition; the advanced natural language processing technology is adopted to preprocess the unstructured text data, key information is effectively extracted, the processing capacity of the unstructured data is remarkably improved, the method can adapt to specific requirements of different industrial fields, the practicability and adaptability of the knowledge graph are enhanced, and by combining a large language model and an improved thinking chain method, the knowledge graph can be quickly and efficiently processed. Automatic entity recognition and relation extraction greatly reduce the dependence on manual annotation, the extraction efficiency is remarkably improved, and a real-time monitoring and updating mechanism enables the knowledge graph to quickly respond to the change of a data source.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and more specifically, to a method for constructing an industrial knowledge graph based on a large language model and a thought chain. Background Art

[0002] By capturing and representing the connections between various data sources within the industry, industrial knowledge graphs help enterprises, research institutions, and government agencies better understand, analyze, and utilize the massive data related to them to support strategic planning, innovation, and decision-making. However, existing industrial knowledge graph construction methods usually rely on predefined structures, rules, and manual annotations. These methods may face limitations when dealing with diverse and dynamically changing data, and it is difficult to effectively deal with complex data types and diverse relationships. In addition, these traditional methods face challenges in large-scale data processing and rapid response to changes.

[0003] In recent years, with the rapid development of natural language processing and machine learning technologies, large-scale pre-trained language models have made significant progress in various fields. Learning rich language knowledge from a large amount of unlabeled data provides new ideas and opportunities for the construction of industrial knowledge graphs. However, the direct application of large language models to the construction of industrial knowledge graphs faces some challenges, such as:

[0004] 1. The breadth and depth of data collection are limited. Existing methods often cannot fully cover diverse data sources, resulting in incomplete information. Secondly, the complexity of unstructured data processing makes it difficult to effectively parse and utilize a large amount of industrial information in text form. In addition, existing knowledge representation methods have limitations in building complex knowledge graph structures that can be flexibly defined and expanded.

[0005] 2. The lack of domain adaptability and model generalization ability makes it difficult for general large language models to deeply understand the nuances of specific industry fields, affecting the accuracy of extraction. The lack of automated and intelligent extraction makes traditional methods that rely on rules or manual annotations inefficient. At the same time, the dynamic update and maintenance of knowledge graphs make it difficult for existing technologies to achieve real-time response to rapidly changing industry information.

[0006] 3. Since large language models are usually trained based on broad rather than domain-specific data, direct application may lead to inaccurate information extraction, insufficient domain specificity, and noise in the knowledge graph. At the same time, large language models are difficult to provide interpretability and cannot ensure data quality and consistency, which will affect the reliability and practicality of the knowledge graph.

[0007] 4. As the amount of data increases, the efficiency of large-scale data processing becomes increasingly prominent, and existing technologies may encounter performance bottlenecks when processing large-scale data sets. The need for multi-source data fusion and conflict resolution requires the ability to effectively integrate information from different data sources and resolve inconsistencies between data.

[0008] In view of this, the present invention provides a method for constructing an industrial knowledge graph based on a large language model and thought chain to solve the above problems. Summary of the invention

[0009] In order to overcome the problems in the prior art, the present invention proposes a method for constructing an industrial knowledge graph based on a large language model and a thinking chain, constructing a high-quality knowledge graph that is domain-specific and can adapt to data changes, overcoming the limitations of the prior art, and providing strong support for industrial analysis and decision-making.

[0010] In a first aspect, the present invention provides a method for constructing an industrial knowledge graph based on a large language model and a thought chain, comprising the following steps:

[0011] Step S1: Acquire task-related industry information text based on the task target industry;

[0012] Step S2: designing the architecture of the knowledge graph based on the industry information text, and annotating and fusing the industry information text based on the architecture of the knowledge graph to obtain the target industry text data;

[0013] Step S3: constructing a fine-tuning data structure used for fine-tuning based on the target industry text data;

[0014] Step S4: Use the large language model to process the target industry text data to obtain fine-tuning data;

[0015] Step S5: fine-tune the large language model using the LoRA method;

[0016] Step S6: Use the fine-tuned large language model combined with the improved thinking chain method to perform relationship extraction and entity recognition on the fine-tuned data; integrate the extracted entities and the relationships between entities to construct an industrial knowledge graph, and import the results into the graph database to complete the construction of the knowledge graph.

[0017] As a preferred technical solution of the first aspect of the present invention, the logic for obtaining the industrial information text is:

[0018] Use web crawler technology to collect industry information text related to the target industry from various external data sources;

[0019] Use crawler tools to determine the relevance of text information to the target industry based on the keywords and topics of the specific target industry, and perform preliminary filtering and classification of text information;

[0020] The crawled text information is cleaned and preprocessed to obtain industrial information text, removing irrelevant information, duplicate data and noise.

[0021] As a preferred technical solution of the first aspect of the present invention, the industry information text is stored in a database to construct an industry text data set.

[0022] As a preferred technical solution of the first aspect of the present invention, the design logic of the knowledge graph architecture is:

[0023] According to the entities related to the target industry in the industry information text, analyze the relationship types between the entities, including upstream and downstream relationships, competitive relationships, cooperative relationships, and affiliation relationships;

[0024] Based on the determined entity and relationship types, the architectural structure of the designed knowledge graph is formulated, which includes node types, edge types, attributes, and constraints.

[0025] As a preferred technical solution of the first aspect of the present invention, the industry information text constructs a large amount of industry initialization data through a template script, and the template script includes:

[0026] Task information, including task information and non-task information divided based on the evaluation of the industry information to be extracted;

[0027] Input, industry information;

[0028] Output: input industry information into the big model to analyze and output task information;

[0029] The constructed data is input into the locally deployed large language model, entity recognition and relationship extraction are performed on the text data, and the relevant entity data and the relationship between entities are obtained and saved.

[0030] As a preferred technical solution of the first aspect of the present invention, the key to the fusion method between entities is to calculate the degree of knowledge matching between two different entities, find out the similarities and differences between the entities, and fuse them to obtain fine-tuned data.

[0031] As a preferred technical solution of the first aspect of the present invention, the fused fine-tuning data is used to construct the training set and test set of the relationship extraction model, and the locally deployed large language model is fine-tuned locally based on the LoRA fine-tuning method. By adjusting the training parameters and verifying the fine-tuned large language model with a validation set, a large language model with high relationship extraction capability is finally obtained.

[0032] As a preferred technical solution of the first aspect of the present invention, in the extraction process, a method of improving the thinking chain is specifically used to construct multiple initial data, and the text information is extracted multiple times. The results of the multiple extractions are evaluated by a large language model to obtain the evaluation results, which specifically include industry information, upstream and downstream industries, and industry products extracted according to preset evaluation rules.

[0033] In a second aspect, the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0034] The processor executes the above-mentioned method for constructing an industrial knowledge graph based on a large language model and a thinking chain by calling a computer program stored in the memory.

[0035] In a third aspect, the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute the above-mentioned method for constructing an industrial knowledge graph based on a large language model and a thinking chain.

[0036] The specific advantages of the present invention are as follows:

[0037] Through a multi-source data acquisition system, the present invention realizes comprehensive and accurate information collection from a wider range of data sources, greatly improving the breadth and depth of data acquisition; and then uses advanced natural language processing technology to pre-process unstructured text data, effectively extract key information, and significantly improve the processing capabilities of unstructured data. It can adapt to the specific needs of different industrial fields and enhance the practicality and adaptability of the knowledge graph.

[0038] Through the LoRA fine-tuning method, we can quickly adapt to specific industrial fields. Combined with the large language model and the improved thought chain method, automated entity recognition and relationship extraction have greatly reduced the reliance on manual labeling and significantly improved the extraction efficiency. The real-time monitoring and update mechanism enables the knowledge graph to quickly respond to changes in data sources, maintaining the timeliness and dynamism of the graph. Combined with the verification and interpretation system, it ensures the interpretability of the extraction process and results, and improves the consistency and reliability of the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The basic architecture for retrieval enhancement generation based on two-stage retrieval optimization of Example 1 of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] See also Figure 1 The present invention provides an industrial knowledge graph construction method based on a large language model and thought chain, which is used to solve the problems of high manual construction cost, low precision, poor integrity, etc. in the traditional industrial graph construction method, and realize the efficient, accurate and automatic construction of industrial graph. The specific steps are as follows:

[0043] Step S1: Obtain industry information text

[0044] Specifically, the logic for obtaining the industrial information text is as follows:

[0045] Use web crawler technology to collect industry information text related to the target industry from various external data sources;

[0046] These data sources include government agencies, industry associations, academic institutions and research institutions, corporate websites, market research reports, news media, trade publications, online data sets and data warehouses, social media, patent and intellectual property databases, etc.

[0047] Use crawler tools to determine the relevance of text information to the target industry based on the keywords and topics of the specific target industry, and perform preliminary filtering and classification of the text information to ensure the relevance and quality of the data.

[0048] Clean and preprocess the crawled text information to obtain industry information text, remove irrelevant information, duplicate data and noise, and ensure the accuracy and consistency of the text data; store the industry information text in the database for subsequent analysis and processing. Data management should ensure the security and availability of data.

[0049] Step S2: Design the schema of the industry knowledge graph based on the industry information text;

[0050] Specifically, the design logic of the industry knowledge graph schema is:

[0051] According to the entities related to the target industry in the industry information text, the relationship types between entities are analyzed. The relationship types include upstream and downstream relationships, competitive relationships, cooperative relationships, and affiliation relationships.

[0052] Based on the determined entity and relationship types, a schema for the designed knowledge graph is formulated. The schema structure includes node types, edge types, attributes, and constraints.

[0053] Step S3: Construct fine-tuning data structure

[0054] Specifically, a data structure of a large language model for fine-tuning is constructed based on the target industry text data. The constructed data structure specifically includes the following parts: task information, which is the task that the large model needs to complete, specifically which industry's information needs to be extracted, what information needs to be extracted, such as the industry's products, attributes, and relationships, and what information is not needed, such as general data or irrelevant information; input, that is, industry information; output, the large model inputs the task results into the output.

[0055] Step S4: Large language model text processing to obtain fine-tuning data

[0056] Specifically, the industry information text is annotated according to the entity and relationship types in the architecture to obtain the annotation results. The annotation includes automatic annotation and manual annotation to ensure the accuracy of the data. The annotated data is injected into the fine-tuning data structure, and then the pre-trained large language model is used to analyze and process the data to identify the entity and relationship types of the target industry text data, and extract the entity and relationship types and store them in the database, and back them up for subsequent fine-tuning and construction of the knowledge graph.

[0057] The industry information text is used to construct a large amount of industry initialization data through template scripts. The template script is divided into several parts:

[0058] Task information, including task information and non-task information divided based on the evaluation of the industry information to be extracted;

[0059] Input, industry information;

[0060] Output: input industry information into the big model to analyze and output task information;

[0061] The constructed data is input into the locally deployed large language model, entity recognition and relationship extraction are performed on the text data, and the relevant entity data and the relationship between entities are obtained and saved.

[0062] Step S5: LoRA fine-tuning

[0063] Specifically, the fine-tuning data is input into the large model, and the LoRA method is used for fine-tuning. The specific process is to add a bypass next to the original large model to perform a dimensionality reduction and then dimensionality increase operation. During training, the parameters of the large model are fixed, and only the dimensionality reduction matrix A and the dimensionality increase matrix B are trained. The input and output dimensions of the model remain unchanged, and the parameters of BA and PLM are superimposed during output. A is initialized with a random Gaussian distribution, and B is initialized with a 0 matrix to ensure that the bypass matrix is ​​still a 0 matrix at the beginning of training to improve the large language model's ability to extract target industry information text.

[0064] Use the validation set to validate the fine-tuned large language model to ensure that the model performance meets the expected target. Based on the validation results, further optimize the model.

[0065] Step S6: Improve thought chain extraction

[0066] Specifically, traditional thinking chains often rely on linear and sequential logical reasoning processes, which may encounter limitations when dealing with complex problems. The reasoning ability of large models can be further enhanced by improving traditional thinking chains.

[0067] The specific steps of the improved thinking chain include: multiple information extraction, multiple information extraction of the input data; evaluation rules: that is, each extraction results are scored, the highest score is set to n, and the average value is taken after multiple extractions. The results with scores higher than half of n are retained; backtracking and re-extraction, the information directly extracted is used as the prompt information for information extraction again, to ensure that the large model can accurately extract the required information.

[0068] Example 2: Based on Example 1, this example uses a fine-tuned large language model combined with an improved thought chain method to perform relationship extraction and entity recognition on industrial texts. The specific operations are as follows:

[0069] Step A1. Define extraction rule information: Based on the actual needs of the target industry, define extraction rule information for extracting entity and relationship types.

[0070] Specifically, the actual needs of the target industry include logical structures based on domain knowledge, industry terminology and industry logic patterns to guide model understanding and extract rule information.

[0071] Step A2: Prediction of large language model: Use the large language model fine-tuned in step S5 of example 1 to predict the text data in step S1 to obtain preliminary entity and relationship data.

[0072] Specifically, the preprocessed industry information text is input into the fine-tuned large-scale language model to analyze the industry information text, identify and predict the entities and relationships in the industry information text; and collect the entities and relationships predicted by the model as preliminary extraction results.

[0073] Step A3: Logical analysis of thought chain: Using the improved thought chain method, the preliminary predicted entity and relationship data are logically analyzed and verified, the human reasoning process is simulated, and the prediction results are verified and corrected.

[0074] Step A4: Extraction optimization: Based on the analysis results, optimize the extraction process to ensure the accuracy and completeness of the extracted entity and relationship data.

[0075] The fine-tuned large language model combined with the improved thought chain method for relation extraction and entity recognition of industrial texts can effectively extract high-quality entities and relations from industrial texts, providing basic data for building knowledge graphs. This method combines the generalization ability of the large language model and the logical reasoning of the thought chain, improving the automation and accuracy of the extraction process.

[0076] Example 3: When the extracted entities and the relationships between the entities are integrated to construct an industry knowledge graph, the specific operations are as follows:

[0077] Step B1 Data integration: Integrate the entity and relationship data extracted from the text data to form the basic data set of the industry knowledge graph.

[0078] Step B2: Fusion algorithm: Use the fusion algorithm to calculate the knowledge matching degree between two different entities and perform data fusion based on the similarity and difference between the entities.

[0079] Step B3: Build a graph database: Use the Neo4j graph database to import the integrated entity and relationship data into the database to build a complete industry knowledge graph.

[0080] Step B4 Knowledge Graph Optimization: Optimize the constructed knowledge graph, including removing redundant data, improving relationship logic, and perfecting the graph structure.

[0081] Embodiment 4: For the parts not described in detail in this embodiment, please refer to the description of Embodiment 1. This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0082] The processor executes the above-mentioned method for constructing an industrial knowledge graph based on a large language model and a thinking chain by calling a computer program stored in the memory.

[0083] Embodiment 5: This embodiment provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the computer executes the above-mentioned method for constructing an industrial knowledge graph based on a large language model and a thinking chain.

[0084] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0085] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing an industrial knowledge graph based on a large language model and a thought chain, characterized in that: The following steps are involved: Step S1: Acquire task-related industry information text based on the task target industry; Step S2: designing the architecture of the knowledge graph based on the industry information text, and annotating and fusing the industry information text based on the architecture of the knowledge graph to obtain the target industry text data; Step S3: constructing a fine-tuning data structure used for fine-tuning based on the target industry text data; Step S4: Use the large language model to process the target industry text data to obtain fine-tuning data; Step S5: fine-tune the large language model using the LoRA method; Step S6: Use the fine-tuned large language model combined with the improved thinking chain method to perform relationship extraction and entity recognition on the fine-tuned data; integrate the extracted entities and the relationships between entities to construct an industrial knowledge graph, and import the results into the graph database to complete the construction of the knowledge graph.

2. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 1 is characterized in that: The logic for obtaining the industrial information text is as follows: Use web crawler technology to collect industry information text related to the target industry from various external data sources; Use crawler tools to determine the relevance of text information to the target industry based on the keywords and topics of the specific target industry, and perform preliminary filtering and classification of text information; The crawled text information is cleaned and preprocessed to obtain industrial information text, removing irrelevant information, duplicate data and noise.

3. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 2 is characterized in that: Store industry information text in the database to build an industry text dataset.

4. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 2 is characterized in that: The design logic of the knowledge graph architecture is: According to the entities related to the target industry in the industry information text, analyze the relationship types between the entities, including upstream and downstream relationships, competitive relationships, cooperative relationships, and affiliation relationships; Based on the determined entity and relationship types, the architectural structure of the designed knowledge graph is formulated, which includes node types, edge types, attributes, and constraints.

5. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 4 is characterized in that: The industry information text is used to construct a large amount of industry initialization data through template scripts. The template scripts include: Task information, including task information and non-task information divided based on the evaluation of the industry information to be extracted; Input, industry information; Output: input industry information into the big model to analyze and output task information; The constructed data is input into the locally deployed large language model, entity recognition and relationship extraction are performed on the text data, and the relevant entity data and the relationship between entities are obtained and saved.

6. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 5 is characterized in that: The key to the entity fusion method is to calculate the degree of knowledge matching between two different entities, find out the similarities and differences between the entities, and fuse them to obtain fine-tuned data.

7. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 6 is characterized in that: Use the fused fine-tuning data to build the training set and test set of the relationship extraction model. Fine-tune the locally deployed large language model based on the LoRA fine-tuning method. By adjusting the training parameters and verifying the fine-tuned large language model with the validation set, we finally get a large language model with high relationship extraction capabilities.

8. The method for constructing an industrial knowledge graph based on a large language model and a thought chain according to claim 6 is characterized in that: During the extraction process, we use the improved thinking chain method to construct multiple initial data, extract text information multiple times, and evaluate the results of multiple extractions using a large language model to obtain evaluation results. The specific information extracted according to preset evaluation rules includes industry information, upstream and downstream industries, and industry products.

9. An electronic device comprising: include: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; The processor executes the method for constructing an industrial knowledge graph based on a large language model and a thought chain as described in any one of claims 1 to 8 by calling the computer program stored in the memory.

10. A computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute a method for constructing an industrial knowledge graph based on a large language model and a thought chain as described in any one of claims 1 to 8.

Images