Knowledge graph construction method and system of tobacco pesticide standard
By constructing a knowledge map of tobacco pesticide standards, the problems of low standard correlation and difficulty in searching are solved, efficient query and digital management of standard information are achieved, and the efficiency and consistency of tobacco pesticide standards are improved.
Patent Information
- Application Number
- CN202411264399.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-15
AI Technical Summary
The correlation of tobacco pesticide standards is low, unable to be efficiently retrieved, difficult to implement, inefficient storage and utilization, lack of overall structural dismantling analysis, and there is little research on the knowledge graph in the field of tobacco pesticide standards.
A knowledge graph construction method of tobacco pesticide standard is adopted, and relevant standards are obtained through keyword search, divided into two levels: standard common elements and standard articles, knowledge modeling and data preprocessing are carried out, and knowledge extraction is used using the BERT-BILSTM-CRF model. , build a knowledge graph, and store the extracted concepts, relationships, and attributes in the Neo4j database in the form of triples.
It realizes accurate correlation query of standard information, quickly retrieves pesticide standard usage and other indicators, improves the query efficiency of standard users, eliminates the knowledge conflict between standards, reduces the workload of updates and revisions, and promotes the digital research of standards in the tobacco field. .
Smart Images

Figure CN118964637A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of agricultural information technology, and in particular to a knowledge graph construction method and system for tobacco pesticide standards. Background Art
[0002] As an important factor affecting the safe production of tobacco, the scientific use of pesticides to ensure the safety of tobacco producers, the quality of tobacco leaves and the ecological environment of tobacco areas is an important step in implementing this plan. As the necessary criteria and basis for the scientific and rational use of tobacco pesticides, how to fully interpret and utilize tobacco pesticide standards to ensure the high quality and appropriate production of tobacco leaves has become extremely critical. With the continuous deepening of national standardization reform, the number of pesticide use standard documents in the tobacco industry has continued to increase, but most of them are still stored in paper or PDF format. The correlation between standards is low and the content is repeated, resulting in the phenomenon that standard information is not fully mined, knowledge expression is not intuitive, users are inconvenient to use, and standards are difficult to implement. Due to its structured characteristics, the knowledge graph can effectively associate tobacco pesticide use standards to form a knowledge network, realize accurate correlation query of standard information, and help users obtain scientific knowledge on the use of tobacco pesticides. Therefore, it is an effective means to ensure the implementation of tobacco pesticide standards and the safe production of tobacco and tobacco products, and a powerful helper to solve the current difficulties faced by standards.
[0003] Since the "knowledge graph" technology was proposed by Google in 2012 and successfully applied to search engines, general domain knowledge graph technology has gradually matured. In pursuit of professionalism, accuracy and depth, research in various fields has gradually focused on the construction and application of vertical domain-specific knowledge graphs. At present, knowledge graphs have achieved good results in vertical fields such as medical care, finance, network security, news and aviation manufacturing, and have also been studied in the fields of tobacco and standard documents. Cheng Qiang constructed a knowledge graph based on the knowledge of tobacco redrying to achieve efficient use of knowledge; Wang Yongsheng et al. constructed a knowledge graph based on the relevant data of scientific research projects in the tobacco field to achieve in-depth data mining, analysis and visualization; Tan Song et al. combined the baking process knowledge to preliminarily construct a knowledge graph of tobacco baking process to meet actual needs; Lv Dongdong et al. extracted entity relationships from the knowledge graph based on agricultural product standards, and to a certain extent realized the association division and reuse of agricultural product standard information; Liu Peng et al. analyzed the structure of standard documents and constructed a knowledge graph DEP knowledge model based on coal mine-related standards to help users complete rapid retrieval and realize standard compliance uniqueness detection; CHEN Yi et al. constructed a knowledge graph based on food safety standards to promote efficient association query of specific information. At present, most of the standards field constructs graphs based on the structural content of some documents, lacks overall structure decomposition and analysis, and there is little involvement in the knowledge graph related research in the field of tobacco pesticide standards. Summary of the invention
[0004] The present invention aims to solve the problems of low correlation, inefficient retrieval, difficulty in implementation and inefficient storage and utilization of tobacco pesticide standards in the prior art, and proposes a knowledge graph construction method and system for tobacco pesticide standards, which comprehensively structures tobacco pesticide standards and helps tobacco pesticide standard users to quickly query indicators such as pesticide standard dosage.
[0005] To achieve the above purpose, the technical solution adopted is:
[0006] A knowledge graph construction method for tobacco pesticide standards, comprising:
[0007] Obtain national standards, industry standards, local standards, group standards and enterprise standards related to tobacco pesticides based on keyword searches;
[0008] The data structure analysis was conducted using tobacco pesticide standards as the data source, and tobacco pesticide standards were divided into two levels of knowledge units: standard common elements and standard provisions.
[0009] Conduct knowledge modeling based on the common elements of standards and standard clauses to comprehensively structure the knowledge content of standards;
[0010] Data preprocessing of tobacco pesticide standard text;
[0011] Extract knowledge from the common elements of standards based on template matching, and extract knowledge from the standard clauses using the BERT-BILSTM-CRF model;
[0012] For the extracted entities with the same name but different forms, high-similarity entities are merged; finally, a knowledge graph of tobacco pesticide standards is constructed.
[0013] According to the knowledge graph construction method of tobacco pesticide standards of the present invention, further, in the standard common element layer, common essential elements and important contents of the standard are set as concepts, and other elements are set as attributes;
[0014] The standard number, standard Chinese name and compiling unit in the cover are set as concepts, and the remaining elements are set as attributes of the standard Chinese name; the responsible unit, drafting unit and drafter in the preface are set as a type of concept, namely the organizational structure, and the technical scope and applicable scope in the scope are set as concepts; the terms in terms and term definitions are set as concepts, and the remaining elements are set as term attributes; the chapter titles and article titles in the core technical elements are set as concepts, and the specific article contents are set as relevant attributes; based on the established standard text concept set, the structural characteristics of the standard elements are analyzed to establish a concept relationship table of the common element layer of the standard.
[0015] According to the knowledge graph construction method of tobacco pesticide standards of the present invention, further, the standard clause layer uses specific standard contents as knowledge units, analyzes basic information on pesticide use in tobacco pesticide standards, and mines the relationships between concepts to obtain a standard clause layer concept relationship table.
[0016] According to the knowledge graph construction method of tobacco pesticide standards of the present invention, further, data preprocessing of tobacco pesticide standard texts includes: first, using pdfminer, TesseractOCR, and pdfplumber technologies for standard PDF files to convert editable texts according to different PDF types using different processing technologies; then correcting the converted text and preprocessing the header and footer; then using the Jiagu word segmenter to perform Chinese word segmentation; and using the BIOES annotation method to annotate the standard text data after Chinese word segmentation to obtain preprocessed standard data.
[0017] According to the knowledge graph construction method of tobacco pesticide standards of the present invention, further, using the BERT-BILSTM-CRF model to extract knowledge from the standard clause layer includes the following contents:
[0018] First, the preprocessed standard text sequence is input into the BERT model, and the BERT model generates a vector sequence that can express the semantic information of the text;
[0019] The vector sequence is then processed through the BILSTM model to generate a vector that can represent contextual information;
[0020] Finally, CRF is used to predict the optimal label sequence and identify the named entities in the data.
[0021] According to the knowledge graph construction method of tobacco pesticide standards of the present invention, further, the BOW model is used to calculate semantic similarity, realize high-similarity entity merging, and obtain the final concept relationship table.
[0022] According to the knowledge graph construction method of tobacco pesticide standards of the present invention, further, constructing the knowledge graph of tobacco pesticide standards includes: using the py2neo library to store the extracted concepts, relationships, and attributes in the form of triples in the Neo4j database, thereby completing the knowledge graph construction work.
[0023] Furthermore, the present invention also provides a knowledge graph construction system for tobacco pesticide standards, comprising a tobacco pesticide standard acquisition module, a data structure analysis module, a tobacco pesticide standard knowledge modeling module, a data preprocessing module, a standard knowledge extraction module and a knowledge fusion storage module, wherein:
[0024] The tobacco pesticide standard acquisition module is used to obtain national standards, industry standards, local standards, group standards and enterprise standards related to tobacco pesticides based on keyword searches;
[0025] The data structure analysis module is used to perform data structure analysis based on tobacco pesticide standards as the data source, and divide tobacco pesticide standards into two levels of knowledge units: standard common elements and standard provisions;
[0026] The knowledge modeling module of tobacco pesticide standards is used to conduct knowledge modeling based on the common elements of standards and standard clauses at two levels to comprehensively structure the knowledge content of standards;
[0027] Data preprocessing module, used for data preprocessing of tobacco pesticide standard text;
[0028] The standard knowledge extraction module is used to extract knowledge from the standard common element layer based on template matching, and to extract knowledge from the standard clause layer using the BERT-BILSTM-CRF model;
[0029] The knowledge fusion storage module is used to merge entities with high similarity that have the same name but different forms, and finally construct a knowledge graph of tobacco pesticide standards.
[0030] The beneficial effects achieved by adopting the above technical solution are:
[0031] 1. Query basic standard information.
[0032] Since standards are stored in Neo4j in the form of structured triples, corresponding information can be queried based on query requirements (a certain entity or a certain entity and its corresponding relationship). For example, by entering the standard name entity, you can query its corresponding reference documents, drafter, responsible unit, compilation unit, scope of application, technical scope, and other information. Figure 1 As shown in the figure, by querying the "Specifications for the Use of Flue-cured Tobacco Pesticides" (Q / YNYC(DL).13.J.17-2018), you can obtain the informative information of this standard, such as the drafter, responsible department, referenced standards and other entity information and their attributes.
[0033] 2. Standard consistency and compliance inspection.
[0034] The knowledge fusion and knowledge graph retrieval technology constructed by the knowledge graph can quickly obtain all standards related to the indicators and realize a comprehensive inspection of the standard indicators. At the same time, by analyzing the standard indicators of Henan tobacco agriculture, the rationality and compliance of the selected values, selection ranges, and selection basis of the current indicators can be evaluated. In addition, it is also possible to compare whether the standard indicators of the seven major production areas meet the Henan tobacco pesticide indicators. For example, by entering the pesticide entity, dosage form, and content (relationship), specific information can be queried to compare with the actual use indicators, assist in tobacco pesticide use indicator queries and compliance testing, and improve the efficiency of standard retrieval for pesticide users and supervisors (such as Figure 2 ).
[0035] By analyzing the knowledge graph of tobacco pesticide standards, we can detect whether there are inconsistencies and other knowledge conflicts between different standard documents. When it is found that the content of standard documents conflicts in different standards, the easy extensibility of the knowledge graph can be used to automatically update all standard documents involving this content by updating and modifying a single node (MATCH, SET statements). This can eliminate knowledge conflicts between different standards, reduce the workload when updating and revising standards, and better provide services to standard setters and users.
[0036] 3. Intelligent search and answering.
[0037] At present, most intelligent services such as the National Standard Information Public Service Platform, China Tobacco Standardization and China Standard Online Service Network can only realize simple retrieval of standard names, numbers, etc., and standard users cannot quickly locate standard documents and required information. The retrieval question-answering system developed with the help of the tobacco pesticide standard knowledge graph can realize the conversion from Chinese to SQL statements by using natural language algorithms, match relevant standards step by step, and give relevant answers under different constraints according to the retrieval requirements, greatly improving the retrieval efficiency and quickly acquiring standard knowledge. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.
[0039] Figure 1 This is a sample diagram of a standard basic information query;
[0040] Figure 2 This is a sample graph of standard indicator queries;
[0041] Figure 3 It is a flowchart of a method for constructing a knowledge graph of tobacco pesticide standards according to an embodiment of the present invention;
[0042] Figure 4 is a schematic diagram of a standard common element layer of an embodiment of the present invention;
[0043] Figure 5 This is a schematic diagram of the tobacco pesticide standard knowledge graph model layer of an embodiment of the present invention;
[0044] Figure 6 Schematic diagram of the process of the BiLSTM model of the embodiment of the present invention;
[0045] Figure 7 is a schematic diagram of a CRF process of an embodiment of the present invention;
[0046] Figure 8is a schematic diagram of a flow chart of calculating an optimal tag sequence according to an embodiment of the present invention;
[0047] Fig. 9 is a sample diagram of a tobacco pesticide standard entity according to an embodiment of the present invention;
[0048] Fig.10 This is a sample diagram of the entity relationship of tobacco pesticide standards according to an embodiment of the present invention;
[0049] Fig.11 This is an example of the tobacco pesticide standard knowledge graph of an embodiment of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.
[0051] This embodiment discloses a knowledge graph construction method for tobacco pesticide standards. A top-down approach is adopted to construct a knowledge graph based on tobacco pesticide standards through data structure analysis, entity and relationship extraction, knowledge fusion and other technologies, so as to complete the transformation of unstructured data into structured data. The knowledge graph of tobacco pesticide standards is used to perform standard compliance and consistency detection, and the relationship between tobacco pesticide standards is professionally and accurately displayed, so as to help pesticide standard users quickly retrieve pesticide standard dosages, etc. to achieve safe production, realize intelligent retrieval and question-answering of tobacco pesticide standards, and promote digital research on standards in the tobacco field.
[0052] like Figure 3 As shown, this embodiment discloses a method for constructing a knowledge graph of tobacco pesticide standards, which includes the following contents:
[0053] Step S1: Obtain national standards, industry standards, local standards, group standards and enterprise standards related to tobacco pesticides based on keyword search.
[0054] Based on keyword retrieval, the National Standard Information Public Service Platform, China Tobacco Standardization and China Standard Online Service Network publicly disclose the current tobacco pesticide standards and their reference standards, covering national standards, industry standards, local standards, group standards and enterprise standards in all aspects, and expert filtering of the collected standards to improve data quality.
[0055] Step S2: Perform data structure analysis using tobacco pesticide standards as data source, and divide tobacco pesticide standards into two levels of knowledge units: standard common elements and standard provisions.
[0056] Clarifying the standard data structure to slice and reorganize the standard text and construct the knowledge ontology is the basis for building the knowledge graph. Standards are written in accordance with unified rules and specifications, so they have the characteristics of standardized structure, clear hierarchy and conciseness. At the same time, the basic element information of the standard can be clarified according to the "Guidelines for Standardization Work Part 1: Structure and Drafting Rules for Standardization Documents" (GB / T 1.1-2020). Therefore, based on the working guidelines and tobacco pesticide use standards, the tobacco pesticide standard element analysis and structured ontology construction are carried out.
[0057] Standard elements can be divided into informative elements and normative elements. The informative elements (cover, table of contents, preface, etc.) and normative general elements (chapter title, article title) have a unified structure, while other normative technical elements are complex. Tobacco pesticide standards are divided into standard common elements (such as Figure 4 There are two levels of knowledge units: (as shown) and standard clauses.
[0058] Step S3: Conduct knowledge modeling based on the two levels of standard common elements and standard clauses to comprehensively structure the standard knowledge content.
[0059] (1) Standard common elements layer
[0060] The standard common elements layer takes the standard document as the overall knowledge unit and summarizes the knowledge fragments according to the standard structure hierarchy logic to form a concept (entity) set. Figure 4 ) to determine concepts and attributes, set common essential elements and important contents of standards as concepts, and set other elements as attributes.
[0061] The standard number, Chinese name of the standard and compiling unit in the cover are set as concepts, and the remaining elements are set as attributes of the Chinese name of the standard. The responsible unit, drafting unit and drafter in the preface are set as a concept, namely the organizational structure, and the technical scope and scope of application in the scope are set as concepts. The terms in terms and term definitions are set as concepts, and the remaining elements are set as term attributes. The chapter titles and article titles in the core technical elements are set as concepts, and the specific article contents are set as related attributes; based on the established standard text concept set, the structural characteristics of the standard elements are analyzed and the knowledge associations between the standards are mined to establish a standard common element layer concept relationship table (Table 1), and a total of 11 categories of concepts and related attributes and 11 categories of concept relationships are established.
[0062] Table 1 Conceptual relationship table of common elements of standards
[0063]
[0064] (2) Standard clause layer
[0065] The standard clause layer uses specific standard content as a knowledge unit to achieve in-depth mining of clause standards, focusing on mining the technical content of pesticide use specified in tobacco standards, such as tobacco-prohibited pesticides, pesticide use methods, etc. By analyzing the basic information on pesticide use in tobacco pesticide standards and mining the relationships between concepts, we can obtain the standard clause layer entities and the relationship table between entities (Table 2).
[0066] Table 2 Conceptual relationship table of standard clauses
[0067]
[0068] Analyze the characteristics of tobacco pesticide standards, combine the common elements layer and the standard clause layer, and construct the knowledge graph model layer of tobacco pesticide standards based on concepts and relationships, such as Figure 5 shown.
[0069] Step S4: preprocess the tobacco pesticide standard text.
[0070] Standard PDF files cannot be edited and knowledge extracted directly. PDFminer, TesseractOCR, and pdfplumber technologies are used to convert editable texts according to different PDF types (written, image, etc.). PDFminer determines the PDF structure. TesseractOCR technology is used for image-type PDFs, pdfminer is used for text, and pdfplumber is used for tables. The converted text is then corrected and preprocessed such as headers and footers to improve the quality of standard data. In order to improve the accuracy of knowledge extraction, Jiagu word segmenter is used for Chinese word segmentation. Since tobacco field standards have professional terms (such as GB_T18771.1-2015 Tobacco Terms Part 1: Tobacco Types and Tobacco Leaf Production, etc.), they cannot be split. In order to improve the accuracy and professionalism of word segmentation, before Chinese word segmentation, Kimi large model is combined to realize standard file term extraction to build a domain vocabulary. The standard text data after Chinese word segmentation is annotated using the BIOES annotation method to obtain preprocessed standard data.
[0071] Step S5: Extract knowledge from the standard common element layer based on template matching, and use the BERT-BILSTM-CRF model to extract knowledge from the standard clause layer.
[0072] The structure of the common elements of the standard is standardized, and the strict hierarchy can be regarded as semi-structured data. Based on the established standard ontology and the structural position of the standard information elements, regular expressions are constructed based on the rule template method to efficiently extract knowledge (concepts, attributes) and obtain entities such as the standard number, standard Chinese name, drafting unit, and centralized unit. Based on the extracted conceptual entities, multi-vectors are used to define the standard terminology structure for dynamic relationship extraction.
[0073] The standard clause layer has no fixed structural features and the terminology is unstructured. Deep learning methods are used for extraction, and the BERT-BILSTM-CRF model is used to realize knowledge extraction. The preprocessed data is input into the BERT model for text modeling, and the text is converted into word vectors. BILSTM is used for sequence feature extraction, and CRF is used to predict labels to obtain knowledge and realize knowledge extraction.
[0074] The specific steps for implementing knowledge extraction using the BERT-BILSTM-CRF model are as follows:
[0075] Step S51: The preprocessed standard text sequence is input into the BERT model, and a vector sequence capable of expressing the semantic information of the text is generated through the BERT model.
[0076] Step S511, process the text sequence into a sequence of words, add [CLS], [SEP], [PAD] tags, [CLS] and [SEP] are used as the start tag and separation tag of the sentence respectively, and add [PAD] padding sequence to keep the sentence length the same.
[0077] Step S512: Use the multi-layer bidirectional Transformer encoder of the BERT model to process the above text, convert each word in the text into a corresponding embedding vector; use the multi-layer bidirectional Transformer encoder to learn the interaction between words and generate a vector representation of each word. By pooling the vector representations of all words, a vector representation of the entire sentence is generated.
[0078] Step S513: Use the fully connected layer of the BERT model to process the above vector text to generate a vector sequence with semantic information.
[0079] Step S52: Process the vector sequence through the BILSTM model to generate a vector that can represent context information.
[0080] BiLSTM processes forward and backward sequential data by using two independent LSTM networks. The forward LSTM processes the sequence from front to back, and the backward LSTM processes the sequence from back to front, and then connects the outputs of the two networks to produce the final prediction. In the BiLSTM model structure, each LSTM unit has three gates that control the flow of information: forget gate, input gate, and output gate.
[0081] Ct o represents a memory unit, which is used to save the state information of the current LSTM and pass it to the LSTM at the next moment.
[0082] x t is the input at the current moment, Ct-1 is the memory of the previous moment, and ht-1 is the output of the previous moment.
[0083] The forget gate is used to control the information that needs to be forgotten. It consists of a sigmoid neural network layer and a bitwise multiplication operation. The working principle is:
[0084] f t =σ(W f ·[h t-1 ,x t ]+b f )
[0085] where f t is the activation value of the forget gate, which is a value between 0 and 1 and is used to control which information in the memory cell state Ct-1 at the previous moment needs to be forgotten; W f is the forget gate weight, b f is the bias vector of the forget gate; σ is the sigmoid activation function.
[0086] The input gate determines the new input information that needs to be retained. The working principle is:
[0087] i t =σ(W i ·[h t-1 ,x t ]+b i )
[0088]
[0089] where i t is the activation value of the input gate, which is also a value between 0 and 1 and is used to control the current input x t Which information should be updated to the memory unit? i is the input gate weight, b i is the bias vector of the input gate; The memory cell state at the current moment. It contains the sequence information from the initial moment to the current moment, and is calculated by the forget gate, input gate, and candidate memory cell state; tanh is the hyperbolic tangent function; W c is the memory unit weight matrix. It is used to calculate the candidate memory unit state; b c is the bias vector of the memory unit.
[0090] After the forget gate and input gate are calculated, according to Update memory cell Ct o.
[0091] The output gate determines how the information in the memory cell is output to the output of the current time step. It works together with the tanh function and the bitwise multiplication operation to pass the memory cell and the input signal to the output end. The working principle is:
[0092] o t =σ(Wo ·[h t-1 ,x t ]+b o )
[0093] h t =o t *tanh(C t )
[0094] Among them t is the activation value of the output gate, which is also a value between 0 and 1, used to control the current input x t Which information should be output to the hidden state h t ; W o is the output gate weight, b o is the bias vector of the output gate.
[0095] like Figure 6 As shown in the figure, the specific implementation steps of the BiLSTM model are as follows:
[0096] S521. Use the output of the BERT model as the input of the BiLSTM model.
[0097] S522: The forward LSTM network processes the vector sequence from front to back to generate a forward hidden state sequence.
[0098] S523: The backward LSTM network processes the vector sequence from back to front to generate a backward hidden state sequence.
[0099] S524. The outputs of the forward LSTM network and the backward LSTM network are spliced through the connector of the BiLSTM model to form a bidirectional output vector sequence, thereby obtaining a context vector for each word.
[0100] Step S53: Establish a model of feature vectors and labels through CRF, find the optimal label sequence, and identify the named entities in the data; Figure 7 As shown, the specific implementation steps are as follows:
[0101] S531. Use the output vector of the BiLSTM model as the input of the CRF model.
[0102] S532: Calculate a transmission probability matrix, where each element in the transmission probability matrix identifies the probability that each word in the sequence belongs to each tag.
[0103] S533. Calculate a state transfer matrix to represent the probability of transferring from one label to another.
[0104] S534. Use a dynamic programming algorithm to obtain an optimal label sequence.
[0105] The Viterbi algorithm in the dynamic programming algorithm is used to calculate the optimal label sequence, such as Figure 8 As shown, the following steps are included:
[0106] S5341, initialize the backtracking table for tracking the best path; initialize the Viterbi table V = (v ij ) m×n , where each cell represents from the beginning to position i; and position i belongs to label y j The shortest path length; initialize the backtracking table to track the best path.
[0107] S5342, update the value in the backtracking table through dynamic programming, for each position i and label y j , calculate the path length:
[0108] S ij =min(S i-1,k )+t kj , where k is the previous label position, t kj Indicates that from label y k to label y j The transition probability.
[0109] S5343. Use the backtracking table, starting from the last position, backtracking to the starting position, and constructing the entire shortest path, which is the shortest path of the entity label.
[0110] Integrate the named entities extracted at different levels of the standard and obtain the CSV table to obtain the extracted concept (entity) table ( Fig. 9 ) and concept (entity) relationship table ( Fig.10 ).
[0111] Step S6: Merge entities with high similarity for the extracted entities with the same name but different forms.
[0112] For the extracted entities with the same name but different forms, semantic similarity judgment and manual assisted judgment are performed according to the characteristics of tobacco pesticide standards to carry out knowledge fusion, realize the uniqueness verification and guarantee of standard knowledge information, and then make the extracted entities comprehensive and non-redundant. That is, after embedding with PaddleNLP preset tags, the BOW model is used to calculate semantic similarity, realize the merging of high-similarity entities, and obtain the final concept (entity) relationship table.
[0113] Step S7: Use the py2neo library to store the extracted concepts (entities), relationships, and attributes in the form of triples in the Neo4j database, and then complete the knowledge graph construction work. Write a Cypher statement and use match query and run to obtain a partial graph display as shown below: Fig.11 .
[0114] Corresponding to the above method, this embodiment also proposes a knowledge graph construction system for tobacco pesticide standards, including a tobacco pesticide standard acquisition module, a data structure analysis module, a tobacco pesticide standard knowledge modeling module, a data preprocessing module, a standard knowledge extraction module and a knowledge fusion storage module, wherein:
[0115] The tobacco pesticide standard acquisition module is used to obtain tobacco pesticide-related national standards, industry standards, local standards, group standards and enterprise standards based on keyword search.
[0116] The data structure analysis module is used to perform data structure analysis based on tobacco pesticide standards as the data source, and divide tobacco pesticide standards into two levels of knowledge units: common elements of standards and standard clauses.
[0117] The tobacco pesticide standard knowledge modeling module is used to perform knowledge modeling based on two levels: common elements of standards and standard clauses, in order to comprehensively structure the knowledge content of standards.
[0118] The data preprocessing module is used to perform data preprocessing on tobacco pesticide standard text.
[0119] The standard knowledge extraction module is used to extract knowledge from the standard common element layer based on template matching, and to extract knowledge from the standard clause layer using the BERT-BILSTM-CRF model.
[0120] The knowledge fusion storage module is used to merge entities with high similarity that have the same name but different forms, and finally construct a knowledge graph of tobacco pesticide standards.
[0121] Unless otherwise specifically stated, the relative steps, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0123] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.
[0124] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.
[0125] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A method for constructing a knowledge graph of tobacco pesticide standards, characterized in that: Include: Obtain national standards, industry standards, local standards, group standards and enterprise standards related to tobacco pesticides based on keyword searches; The data structure analysis was conducted using tobacco pesticide standards as the data source, and tobacco pesticide standards were divided into two levels of knowledge units: standard common elements and standard provisions. Conduct knowledge modeling based on the common elements of standards and standard clauses to comprehensively structure the knowledge content of standards; Data preprocessing of tobacco pesticide standard text; Extract knowledge from the common elements of standards based on template matching, and extract knowledge from the standard clauses using the BERT-BILSTM-CRF model; For the extracted entities with the same name but different forms, high-similarity entities are merged; finally, a knowledge graph of tobacco pesticide standards is constructed.
2. The method for constructing a knowledge graph of tobacco pesticide standards according to claim 1, characterized in that: In the standard common elements layer, the common essential elements and important contents of the standard are set as concepts, and other elements are set as attributes; The standard number, standard Chinese name and compiling unit in the cover are set as concepts, and the remaining elements are set as attributes of the standard Chinese name; the responsible unit, drafting unit and drafter in the preface are set as a type of concept, namely the organizational structure, and the technical scope and applicable scope in the scope are set as concepts; the terms in terms and term definitions are set as concepts, and the remaining elements are set as term attributes; the chapter titles and article titles in the core technical elements are set as concepts, and the specific article contents are set as relevant attributes; based on the established standard text concept set, the structural characteristics of the standard elements are analyzed to establish a concept relationship table of the common element layer of the standard.
3. The method for constructing a knowledge graph of tobacco pesticide standards according to claim 2, characterized in that: The standard clause layer uses specific standard contents as knowledge units, analyzes the basic information on pesticide use in tobacco pesticide standards, and mines the relationships between concepts to obtain the concept relationship table of the standard clause layer.
4. The method for constructing a knowledge graph of tobacco pesticide standards according to claim 1, characterized in that: Data preprocessing of tobacco pesticide standard text includes: first, for standard PDF files, pdfminer, TesseractOCR, and pdfplumber technologies are used to convert editable text using different processing technologies according to different PDF types; then the converted text is corrected and the header and footer are preprocessed; then the Jiagu word segmenter is used to perform Chinese word segmentation; the standard text data after Chinese word segmentation is annotated using the BIOES annotation method to obtain preprocessed standard data.
5. The method for constructing a knowledge graph of tobacco pesticide standards according to claim 1, characterized in that: Using the BERT-BILSTM-CRF model to extract knowledge from the standard clause layer includes the following: First, the preprocessed standard text sequence is input into the BERT model, and the BERT model generates a vector sequence that can express the semantic information of the text; The vector sequence is then processed through the BILSTM model to generate a vector that can represent contextual information; Finally, CRF is used to predict the optimal label sequence and identify the named entities in the data.
6. The method for constructing a knowledge graph of tobacco pesticide standards according to claim 1, characterized in that: The BOW model is used to calculate semantic similarity, merge entities with high similarity, and obtain the final concept relationship table.
7. The method for constructing a knowledge graph of tobacco pesticide standards according to claim 1, characterized in that: Constructing the knowledge graph of tobacco pesticide standards includes: using the py2neo library to store the extracted concepts, relationships, and attributes in the form of triples in the Neo4j database, thereby completing the knowledge graph construction work.
8. A knowledge graph construction system for tobacco pesticide standards, characterized in that: It includes tobacco pesticide standard acquisition module, data structure analysis module, tobacco pesticide standard knowledge modeling module, data preprocessing module, standard knowledge extraction module and knowledge fusion storage module, among which: The tobacco pesticide standard acquisition module is used to obtain national standards, industry standards, local standards, group standards and enterprise standards related to tobacco pesticides based on keyword searches; The data structure analysis module is used to perform data structure analysis based on tobacco pesticide standards as the data source, and divide tobacco pesticide standards into two levels of knowledge units: standard common elements and standard provisions; The knowledge modeling module of tobacco pesticide standards is used to conduct knowledge modeling based on the common elements of standards and standard clauses at two levels to comprehensively structure the knowledge content of standards; Data preprocessing module, used for data preprocessing of tobacco pesticide standard text; The standard knowledge extraction module is used to extract knowledge from the standard common element layer based on template matching, and to extract knowledge from the standard clause layer using the BERT-BILSTM-CRF model; The knowledge fusion storage module is used to merge entities with high similarity that have the same name but different forms, and finally construct a knowledge graph of tobacco pesticide standards.
9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
