A method and system for constructing a knowledge graph in the field of civil engineering specifications

Abstract

The present invention relates to a method and a system for constructing a knowledge graph in the field of civil engineering specifications, including screening out the sentences to be extracted from the specification text for modification, selecting the sentences that conform to the Schema tag definition to make them suitable for extraction by the neural network model; training the model with the screened part of the text, data annotating the training set according to the designed triple Schema tag to obtain a triple training set in the field of civil engineering specification text; constructing a CE-CasRel model, training it with the annotated data set, and validating it with a validation set; inputting the obtained specification text to be extracted into the trained CE-CasRel model, extracting the triples in the field of civil engineering specifications, and storing them in a json file; reading the json file, parsing the triple data, using the API interface of Neo4j to establish a knowledge graph in the field of civil engineering specifications and realizing visualization; meeting the requirements of intelligent drawing review for the knowledge graph.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent building drawing review, and relates to a method and a system for constructing a knowledge graph in the field of civil engineering specifications, and particularly relates to a method and a system for constructing a knowledge graph in the field of civil engineering specifications based on a deep neural network. Background Art

[0002] The entire life cycle of construction projects is subject to various specifications and standards. As a key link to ensure the quality of design and construction, drawing review still highly relies on manual review. There are currently many problems with manual review: the levels of drawing review personnel vary. During the drawing review process, drawing review personnel from different regions and institutions may have different understandings of the specifications, which will lead to differences in the judgment of whether the drawings are qualified; there are many repetitive review contents in the drawings, but the reviewers need to review and confirm their compliance one by one, resulting in a lot of repetitive labor and reducing work efficiency; there are many and extensive articles in building specifications, which are difficult to understand. For the same building component, different specifications may have different requirements. Therefore, it is difficult to ensure the comprehensiveness and error-freeness of drawing review in practice; the degree of intelligence is low and cannot meet the requirements of the country for the future development of intelligent construction.

[0003] For the above reasons, intelligent drawing review has received more and more attention and research, and its importance has become increasingly prominent. In the process of intelligent review, the most complex and important part is how the computer understands the knowledge of specification articles and conducts reasoning. Current research can be divided into two parts, one is the representation of specification knowledge, and the other is the interpretation of rules.

[0004] Knowledge representation is often abstracted as a triple of entity, attribute, and relationship for representation, and traditional storage is in formats such as RDF and XML; in recent years, based on ontology, a knowledge graph, that is, a graph relational network, has gradually emerged to represent knowledge more intuitively. However, domain ontology needs to be established according to specific application scenarios or actual needs. Ontologies in different professional fields are often different and often require a large amount of manual processing, lacking intelligence. For the civil engineering field, the relational network defined by ontology cannot highly refine specification articles. How to construct a unified, efficient, and ontology framework suitable for knowledge extraction for the civil engineering field is still a problem that needs to be further studied.

[0005] Rule interpretation is to process the specification clauses into a specific format that can be recognized by a computer. A knowledge graph can be regarded as a kind of rule interpretation. Traditional interpretation methods are all realized based on experts' manual work. In recent years, with the research and development in other fields, scholars have continuously started to study the use of methods such as natural language processing and ontology for interpretation. Existing interpretation methods can be divided into two categories: shallow structuring and pattern matching. However, both methods have limitations. For shallow structuring, it can only process the specification text at a coarser granularity level, cannot analyze at the word level, and requires a lot of manpower for maintenance. For pattern matching, it highly depends on regular expressions, has poor flexibility, and has a high maintenance cost when it needs to be updated later. Summary of the Invention

[0006] In view of this, in order to solve the problem that the two commonly used knowledge graph interpretation methods of shallow structuring and pattern matching have limitations in knowledge representation and rule interpretation, resulting in high later maintenance costs, the present invention provides a method and a system for constructing a knowledge graph in the field of civil engineering specifications, which can meet the requirements of intelligent drawing review for the knowledge graph.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A method for constructing a knowledge graph in the field of civil engineering specifications includes the following steps:

[0009] S1. Screen out the sentences to be extracted from the specification text, modify the text format and text sentence pattern of the screened text, and select the sentences that conform to the Schema label definition to make them suitable for extraction by the neural network model;

[0010] S2. Train the part of the text screened in step S1, segment the text into several sentences, and label the training set according to the designed triple Schema label to obtain a triple training set in the field of civil engineering specification text;

[0011] S3. Construct a CE-CasRel model, train it with the labeled data set, and verify it with the validation set; when constructing the CE-CasRel model, the pre-trained model used is the BERT model optimized on the civil engineering field data set;

[0012] S4. Repeat steps S1 and S2, input the specification text to be extracted into the trained CE-CasRel model, extract the triples in the field of civil engineering specifications, and store them in a json file;

[0013] S5. Read the json file, parse the triple data, and use the API interface of Neo4j to establish a knowledge graph in the field of civil engineering specifications and realize visualization.

[0014] Furthermore, the specific process of step S2 is as follows:

[0015] S21. Use regular expressions to organize the segmented sentences into the same folder to obtain the dataset to be annotated. The regular rules are defined by using full stops, semicolons, and colons as the markers for sentence segmentation;

[0016] S22. Divide the dataset into a training set and a test set according to a preset ratio, and use annotation software to annotate the training set to obtain a triple training set for training knowledge extraction;

[0017] S23. For triple extraction and knowledge graph construction, divide the extracted triple training set into entity labels and relationship labels according to the new Schema label definition rules. The entity labels MSub, Sub, and Prop represent the objects to be reviewed in drawing review. The relationship labels Sub_Prop, Prop_Bx, and Bx_RProp are the three most basic relationships in the code provisions, indicating that "a certain subject"'s "certain attribute" "should meet" "certain specification requirements".

[0018] Furthermore, in step S23, the entity labels MSub, Sub, and Prop represent the objects to be reviewed in drawing review. Among them, Sub represents the subject (Subject) in the object to be inspected, such as components and structures; Prop represents the attribute (Property) of Sub, which is restricted by RProp or LRProp, such as thickness, cross-sectional area, etc.; MSub represents the superset of Sub, and has an inclusion relationship with the Sub element.

[0019] The entity label RProp represents the requirement conditions connected to Prop, indicating the requirements (Requirement) for Prop. Only by meeting the constraint conditions provided by RProp can it comply with the code provisions.

[0020] The entity label Bx represents a certain "behavior" between Prop and RProp. Common behaviors include comparison or subordination relationships. For example: "greater than", "has", "meets", etc.

[0021] The entity label RSub, the parent element of RProp, is used when RProp is an attribute of a certain object or concept. For example: "beam height" in "the effective height should not be less than 1 / 3 of the beam height".

[0022] The entity label LRProp, the prerequisite condition (regarded as a kind of restriction) connected to Bx. Only by meeting the constraint of the prerequisite condition provided by LRProp will it continue to judge whether RProp meets the rule review later.

[0023] The entity label MProp, when there are multiple nested attributes, MProp is regarded as the upper-level attribute of Prop. For example, in "the concrete strength grade of precast components", "concrete / MProp" and "strength grade / Prop".

[0024] The entity labels CSub, CProp, CBx, and CRProp are condition groups (Condition). When the specification statement is too complex and there are complete clauses to limit the fact group, at this time LRProp can no longer fully summarize the semantics, so the condition group is introduced. Use the condition group to satisfy the most basic four elements of a sentence in the specification.

[0025] Among them, the relationship labels are:

[0026] The relationship labels Sub_Prop, Prop_Bx, and Bx_RProp are the three most basic relationships in the specification, indicating that "a certain subject"'s "certain attribute" "should meet" "a certain specification requirement".

[0027] The relationship label RSub_RProp indicates that "a certain specification requirement" is "an attribute of a certain subject".

[0028] The relationship label MProp_Prop represents the "attribute of attribute" relationship, that is, Prop is an attribute of MProp.

[0029] The relationship label LRProp_Bx represents the precondition of a certain element for the review main clause. In this patent, this element is regarded as a kind of modification and limitation to the Bx label element.

[0030] The relationship label MSub_Sub indicates that MSub is a superset of Sub, that is, MSub contains Sub.

[0031] The relationship labels CSub_CProp, CProp_CBx, and CBx_CRProp are similar to the three basic relationships. The only difference is that they are used to distinguish between the condition group and the fact group.

[0032] Furthermore, the specific process of step S3 is as follows:

[0033] S31. Build the CE-CasRel model. Use the BERT model that has been re-pre-trained with the civil engineering domain dataset to replace the original encoder at the encoder end. This patent calls it CEBERT; at the decoder end, to solve the entity nesting problem, use the stacked pointer annotation method; and before predicting the tail entity of the triple, use the feature vector of the head entity as a condition and perform a Conditional Layer Normalization (CLN) on the feature vector;

[0034] S32. Use the labeled training set text as input and enter the CEBERT encoder. The CEBERT encoder performs multi-head attention, layer normalization, and residual connection operations on each specification clause to extract text features and obtain sentence-level vector representations.

[0035] S33. Pass the vector representation obtained after the pre-trained model CEBERT into the head entity recognition layer of the decoder. First, use a linear layer and a sigmoid activation function to determine whether each token is the start or end of the head entity. Then, use the nearest matching principle to pair the recognized start and end to obtain a set of candidate head entities.

[0036] S34. Then randomly select a head entity and use this entity as a condition to perform a CLN on the sentence-level vector representation. Use the sentence vector representation after CLN processing as input, and also pass it through a linear layer and a sigmoid activation function to predict the position of the tail entity under different relationships.

[0037] S35. After the training of the CE-CasRel model is completed, use the test set data to verify the trained model. After reaching the required precision and recall rates, retain the model weight parameters obtained from training.

[0038] Furthermore, the CEBERT encoder described in step S32 is specifically as follows: Use the Python crawler method to crawl a large amount of text from civil engineering-related encyclopedia entries and specification texts, perform data cleaning such as long text splitting and irrelevant content screening, and construct a civil engineering field dataset. Select the Chinese version of the RoBERTa pre-trained model, use the dataset as input corpus, and pass it into the model for fine-tuning. The training method and evaluation metrics refer to the original design of RoBERTa. Finally, obtain the model CEBERT that has been fine-tuned on the civil engineering field dataset.

[0039] Furthermore, the CLN expression in step S31 is:

[0040]

[0041] where y is the feature information input to the CLN structure, and are the two conditional information to be fused as input respectively;

[0042] The sigmoid activation function expression in step S33 is:

[0043] .

[0044] Further, the specific process of step S4 is as follows: Repeat steps S1 and S2 to input the specification text to be extracted into the trained CE-CasRel model, and extract a set of triples that conform to the Schema tag definition in step S23; Through Python data processing, save the extracted triples line by line according to the articles, with one line for one specification article, and one article corresponding to several triples; Store all the articles into a json file.

[0045] Further, the specific steps of step S5 are as follows:

[0046] S51. Define the nodes in the Neo4j graph relationship library according to the entity tags described in the Schema in step S23; Define the edges in the Neo4j graph relationship library according to the relationship tags;

[0047] S52. Use the json and pands libraries of Python to read the json file and read the set of triples of each specification article line by line; Extract and add them to the Python list according to the defined Schema tags;

[0048] S53. Use the py2neo library of Python to connect Python to Neo4j; Correspondingly add the entities in the list in step 52 to the Neo4j nodes; Correspondingly add the relationships in the list in step 52 to the Neo4j edges to complete the construction of the knowledge graph;

[0049] S54. Start the Neo4j program through the console and view and save the graph in the browser.

[0050] A knowledge graph construction system in the field of civil engineering specifications includes a preprocessing module, a knowledge extraction module, and a construction module; Among them,

[0051] The preprocessing module is used to segment the specification articles and preprocess the segmented sentences to obtain convertible specification sentences;

[0052] The knowledge extraction module is used for the deep neural network model CE-CasRel to extract the triples in the specification articles to obtain structured specification articles saved in the form of triples;

[0053] The construction module is used to construct a knowledge graph and store the triples in the graph database based on the py2neo library for the structured specification articles.

[0054] The beneficial effects of the present invention are as follows:

[0055] 1. A method for constructing a knowledge graph in the field of civil engineering specifications disclosed by the present invention combines knowledge representation and rule interpretation. From the perspective of review, an abstract representation is refined from the specification articles to be reviewed: "a certain attribute of a certain subject must meet certain specification requirements". In this way, 4 basic entities and 3 basic relationships are abstractly summarized, and on this basis, 8 entities and 7 relationships are added as modifications or restrictions. This method abstractly extracts the core information of the specification articles, which is convenient for deep learning models to extract and can also serve rule interpretation and the construction of knowledge graphs; it can meet the requirements of intelligent drawing review for knowledge graphs; it solves the problems that the two common knowledge graph interpretation methods of shallow structuring and pattern matching have limitations in knowledge representation and rule interpretation, resulting in high later maintenance costs.

[0056] 2. In a method for constructing a knowledge graph in the field of civil engineering specifications disclosed by the present invention, intelligent drawing review is inseparable from existing specification articles, and the basic problem of storing specifications structurally in a database must be solved. As a graph database, the knowledge graph is one of the current research directions, but the specifications themselves are professional, and existing methods cannot be directly applied in the civil engineering field. This method proposes a knowledge graph construction method adapted to the civil engineering field and improves the existing information extraction model for the training data set in the civil engineering field, which is beneficial to subsequent research.

[0057] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, they will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0059] Figure 1 is a flowchart of the method for constructing a knowledge graph in the field of civil engineering specifications according to an embodiment of the present invention;

[0060] Figure 2 is a schematic diagram of the network structure of the CE-CasRel model according to an embodiment of the present invention;

[0061] Figure 3 is a schematic diagram of the extracted structured specification articles according to an embodiment of the present invention;

[0062] Figure 4 is a display of the knowledge graph in the field of civil engineering specifications according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand the other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0064] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as limiting the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0065] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, so the terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0066] As Figure 1 shown, a method and system for constructing a knowledge graph in the field of civil engineering specifications include the following steps:

[0067] S1. Screen out the sentences to be extracted from the specification text, and modify the sentence patterns of the screened text to make them suitable for extraction by the neural network model;

[0068] In this embodiment, specification documents are collected through methods such as web crawling. Most of the collected documents are in PDF or Word format, and are converted into TXT format files using text editing software such as ABBYY and Acrobat DC. The pictures and table contents in the specification are deleted, and irrelevant or non-extractable sentences are deleted using regular expressions, such as the table of contents, blank lines, page numbers, etc., and sentences that conform to the Schema tag definition are selected. Among the selected sentences, keywords such as comparison relationships and subordinate relationships should be included.

[0069] S2. Perform model training on the part of the text filtered out in step S1, segment the text into sentences, obtain several sentences, and annotate the training set with the designed triple Schema label to obtain a triple training set in the field of civil engineering specification text. Specifically, the following steps are included:

[0070] S21. Use regular expressions to save the segmented sentences as txt files; organize the txt files into the same folder to obtain a dataset to be labeled; wherein the regular rule is defined as using period, semicolon, and colon as signs for segmenting sentences;

[0071] S22, dividing the data set into a data set and a test set in a ratio of 8:2, annotating the training set using annotation software to obtain a triplet training set for training knowledge extraction;

[0072] S23. To extract triples and construct a knowledge graph, a new type of Schema labeling is defined. The extracted triple training set is divided into entity labels and relationship labels according to the new Schema labeling rules, as shown in Tables 1 and 2, respectively. From the perspective of rule review, this patent abstractly represents the regulatory clause under review: "A certain attribute of a certain subject must meet certain regulatory requirements." The label Sub represents "a certain subject," Prop represents "a certain attribute," and RProp represents "a certain regulatory requirement." The label Bx represents "satisfaction," essentially representing a "behavior" between Prop and RProp. Sometimes, the single label RProp cannot fully capture the semantics, and the label RSub is needed to expand the semantics. RProp can be considered an attribute of RSub. Similarly, when the single label Sub cannot fully capture the semantics, the label MSub is referenced as the parent set of Sub. The label LRprop is similar to the label Rprop, but the prefix L indicates a precondition; that is, the rule check is executed only when the comparison between the label Prop and the label LRprop is true. Sometimes, certain clauses contain attributes that modify properties. For example, in the "Concrete Strength Grade for Precast Components" section, "Concrete / MProp" and "Strength Grade / Prop" are used. In these cases, MProp is introduced as a parent attribute of Prop. Most of the time, the above labels fully summarize the semantics of the specification clause. However, sometimes conditional clauses serve as conditions, restricting the review requirements of the main clause. Therefore, the conditional groups CSub, CProp, CBx, and CRProp are introduced to describe conditional clauses. For other specific definitions, refer to Tables 1 and 2.

[0073] Table 1: Entity Tags

[0074]

[0075] Table 2: Relationship Tags

[0076]

[0077] Traditional triple extraction methods have a simple defined Schema framework, while the semantics of regulatory articles are complex and difficult to be defined by a simple Schema. The characteristics of regulatory articles are as follows: the semantics of regulatory articles are rich, and from the perspective of content, each sentence has multiple entities; to ensure the semantic integrity of the articles, multiple triples need to be nested to accurately represent the semantics. Therefore, from the perspective of examination, this patent extracts an abstract representation from the regulatory articles to be examined: "a certain attribute of a certain subject needs to meet certain regulatory requirements". In this way, 4 basic entities and 3 basic relationships are abstracted and generalized. On this basis, 8 entities and 7 relationships are added as modifications or restrictions. This method abstracts and extracts the core information of regulatory articles, which is convenient for deep learning models to extract and can also serve the rule interpretation and the construction of knowledge graphs.

[0078] S3. Construct the CE-CasRel model, train it with the labeled dataset, and verify it with the validation set; when constructing the CE-CasRel model, the pre-trained model used is the BERT model optimized on the civil engineering domain dataset; specifically, it includes the following steps:

[0079] S31. As Figure 2 shown, the overall idea of the CE-CasRel model is: identify the head entity, that is, Subject; on the basis of the entity, identify all the tail entities (Object) corresponding to all the given relationships (Relation), that is, use the feature vector of the head entity as a condition and perform a Conditional Layer Normalization (CLN) on the feature vector.

[0080] The expression of Conditional Layer Normalization (CLN) is:

[0081]

[0082] Among them, y is the feature information input into the CLN structure, and are respectively the two conditional information to be fused input.

[0083] Traditional information fusion methods are all addition, multiplication, etc. These methods ignore the directionality between information, but in triples, this directionality is very important. It can be seen from the CLN expression that through the trainable weights and , the conditional information is mapped to different spaces to reflect the direction information.

[0084] Replace the original encoder with a BERT model that has been re - pre - trained using a dataset in the civil engineering field at its encoder end. This patent refers to it as CEBERT; at the decoder end, to solve the entity nesting problem, a stacked pointer annotation method is used. In this embodiment, the loss function of the CE - CasRel model is as follows:

[0085]

[0086] In the formula: sentence belongs to the training set D and represents the j - th input sample;

[0087] is the text that contains all triples;

[0088] represents a head entity that appears in ; = represents the learnable weights W and bias b when predicting the head entity; represents the prior probability after taking the logarithm. The prior probability is equivalent to the probability of predicting the head entity here;

[0089] represents a set of relationships that appear in and whose head entity is s; represents the learnable weights W and bias b when predicting the tail entity and the relationship; represents the probability of predicting the tail entity after taking the logarithm;

[0090] R is the set of all relationships; is the difference set between R and , representing other relationships that do not appear in ; represents that for the sentence with respect to the head entity s and the relationship r that does not appear in , the tail entity recognition should be empty; represents the predicted probability after taking the logarithm for tail entities that are not in the sentence and tail entities without corresponding head entities.

[0091] S32. Take the labeled training set text as input and enter the CEBERT encoder; the encoder CEBERT performs multi - head attention, layer normalization processing, and residual connection operations on each specification clause, extracts text features, and obtains a sentence - level vector representation.

[0092] In this example, the design of the encoder CEBERT is as follows: Using the Python crawler method, a large amount of text is crawled from civil engineering-related encyclopedia entries and specification texts, and data cleaning such as long text splitting and irrelevant content screening is performed to construct a dataset in the field of civil engineering. The Chinese version of the RoBERTa pre-trained model is selected, and the dataset is used as the input corpus and fed into the model for fine-tuning; the training method and evaluation metrics refer to the original design of RoBERTa; finally, the model CEBERT fine-tuned on the civil engineering field dataset is obtained.

[0093] Pre-training the model is essentially a form of transfer learning. A neural network model is trained on a large-scale dataset and then applied to the target task for further training, and this model is the pre-trained model. Currently, due to the lack of relevant research in the civil engineering field, there is no fine-tuning for the civil engineering field dataset. This patent fine-tunes the data in the civil engineering field, which can improve the accuracy of related natural language processing tasks in this field by 5.4%.

[0094] S33. Input the vector representation obtained after passing through the pre-trained model CEBERT into the head entity recognition layer of the decoder. First, use a linear layer and a sigmoid activation function to determine whether each token is the start or end of the head entity; then, use the nearest matching principle to pair the identified start and end to obtain a set of candidate head entities, where the expression of the sigmoid activation function is:

[0095] 。

[0096] The function finally used for classification in the conventional knowledge extraction model is the Softmax activation function, but the Softmax outputs mutually exclusive data and is more suitable for multi-class classification problems. There are a large number of entity overlap problems in the dataset involved in the knowledge extraction of this patent. Therefore, it is more appropriate to use the sigmoid function to handle this multi-label classification problem.

[0097] In this embodiment, the expressions of the linear layer and the sigmoid activation function for predicting the head entity are:

[0098]

[0099]

[0100] In the formula: represents the i-th token in the sentence, and the token represents a single Chinese character in this patent;

[0101] represents the learnable weights W and biases b in the linear layer;

[0102] represents the sigmoid activation function;

[0103] and represents the probability that the i-th token is the tail position of the head position of the head entity;

[0104] such as Figure 2 As shown, in this embodiment, the meaning of the nearest matching principle is: after obtaining all start_s and end_s, the nearest adjacent start and end are regarded as a complete entity.

[0105] S34. Then enter the relationship-specific tail entity recognition layer: randomly select a head entity, use this entity as a condition to perform a CLN on the sentence-level vector representation; use the sentence vector representation after CLN processing as input, and also pass through a linear layer and a sigmoid activation function to predict the tail entity position under different relationships.

[0106] In this embodiment, the linear layer and sigmoid activation function expressions for predicting the tail entity and relationship are:

[0107]

[0108]

[0109] In the formula: represents using CLN to fuse the sequence encoding and the encoding of the head entity. Since there is only one entity in the input condition, the two conditions are set to the same entity.

[0110] S35. After the CE-CasRel model training is completed, use the test set data to verify the trained model. After reaching the required precision and recall rate, retain the model weight parameters obtained from training.

[0111] Using the features of the BERT pre-trained model as the encoder can not only eliminate the need for manual feature extraction, but also make its feature representation more reasonable, improve the model accuracy, and save the training parameter tuning time. The decoder uses the stacked pointer annotation method, which can effectively solve the problem of a large number of entity relationship overlaps in the canonical text; using CLN for the head entity can better fuse the feature vectors of the head entity and the sentence than the traditional feature addition.

[0112] S4. Input the canonical text to be extracted into the trained CE-CasRel model, extract the triples in the civil engineering specification field, and store them in a json file;

[0113] In this embodiment, the specific steps are as follows: Repeat steps S1 and S2 to obtain more processed standard texts. Input the standard text to be extracted into the trained CE-CasRel model to extract a set of triples that conform to the Schema tag definition. Through Python data processing, save the extracted triples line by line according to the articles, with one line being one standard article, and one article corresponding to several triples. Store all articles in a json file. As Figure 3 shown, it is a schematic diagram of the stored structured standard articles.

[0114] S5. Read the json file, parse the triple data, and use the API interface of Neo4j to build a knowledge graph of civil engineering specifications and realize visualization.

[0115] As Figure 4 shown, it shows the results of the knowledge graph of civil engineering specifications. In this embodiment, the specific steps are as follows:

[0116] S51. Define the nodes in the Neo4j graph relationship library according to the entity tags described in the Schema; define the edges in the Neo4j graph relationship library according to the relationship tags.

[0117] S52. Use the json and pands libraries of Python to read the json file, and read the triple set of each standard article line by line; according to the defined Schema tags, extract and add them to the Python list from the triple set.

[0118] S53. Use the py2neo library of Python to connect Python to Neo4j; add the entities in the list of step 52 to the Neo4j nodes correspondingly; add the relationships in the list of step 52 to the Neo4j edges correspondingly to complete the construction of the knowledge graph.

[0119] S54. Start the Neo4j program through the console, and enter "http: / / localhost:7474 / " in the browser to view and save the graph.

[0120] Using py2neo, the Neo4j database can be operated using the Python language, thus avoiding using the Cypher query syntax of Neo4j; the Schema tag system defined in this patent converts the relationship between triples from two-dimensional to one-dimensional, so it can be directly used to build a knowledge graph without redundant conversion operations.

[0121] A knowledge graph construction system for the field of civil engineering specifications based on a deep neural network includes a preprocessing module, a knowledge extraction module, and a construction module; among them,

[0122] A preprocessing module, which is used to perform sentence segmentation on the specification clauses and preprocess the segmented sentences to obtain convertible specification sentences;

[0123] A knowledge extraction module, which is used for the deep neural network model CE-CasRel to extract triples from the specification clauses and obtain structured specification clauses saved in the form of triples;

[0124] A construction module, which is used to construct a knowledge graph and store the triples in a graph database based on the py2neo library for the structured specification clauses.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for constructing a knowledge graph in the field of civil engineering specifications, characterized in that It includes the following steps: S1. Screen out the sentences to be extracted from the specification text, modify the text format and sentence patterns of the screened-out text, and select the sentences that conform to the Schema tag definition to make them suitable for extraction by the neural network model; S2. Train the part of the text screened out in step S1, segment the text into several sentences, and label the training set according to the designed triple Schema tag to obtain the triple training set in the field of civil engineering specifications; S3. Build a CE-CasRel model, train it with the dataset labeled in step S2, and verify it with the validation set; Among them When building the CE-CasRel model, the pre-trained model used is the BERT model optimized on the civil engineering domain dataset; S4. Repeat steps S1 and S2, input the specification text to be extracted into the trained CE-CasRel model, extract the triples in the field of civil engineering specifications, and store them in a json file; S5. Read the json file, parse the triple data, use the API interface of Neo4j to establish a knowledge graph in the field of civil engineering specifications and realize visualization; The specific process of step S2 is as follows: S21. Use regular expressions to organize the segmented sentences into the same folder to obtain the dataset to be labeled; the regular rule is defined as using full stops, semicolons, and colons as the signs for splitting sentences; S22. Divide the dataset into a training set and a test set according to a preset ratio, and use annotation software to annotate the training set to obtain the triple training set for training knowledge extraction; S23. For triple extraction and knowledge graph construction, divide the extracted triple training set into entity labels and relationship labels according to the new Schema tag definition rule. The entity labels MSub, Sub, and Prop represent the objects to be reviewed in the drawing review. The relationship labels Sub_Prop, Prop_Bx, and Bx_RProp are the three most basic relationships in the specification clauses, indicating that "a certain subject"'s "certain attribute" "should meet" "certain specification requirements"; The specific process of step S3 is as follows: S31. Build a CE-CasRel model. Use the BERT model re-pre-trained with the civil engineering domain dataset to replace the original encoder at the encoder end, which is called CEBERT; at the decoder end, to solve the entity nesting problem, use a stacked pointer annotation method; and before predicting the tail entity of the triple, use the feature vector of the head entity as a condition to perform a Conditional Layer Normalization on the feature vector; S32. Take the text of the labeled training set as input and enter the CEBERT encoder; The encoder CEBERT performs multi-head attention, layer normalization processing, and residual connection operations on each specification clause to extract text features and obtain a sentence-level vector representation; S33. Input the vector representation obtained after passing through the pre-trained model CEBERT into the head entity recognition layer of the decoder. First, use a linear layer and a sigmoid activation function to determine whether each token is the start or end of the head entity; then, use the nearest matching principle to pair the recognized start and end to obtain a set of candidate head entities. S34. Then, randomly select a head entity and use this head entity as a condition to perform a CLN on the sentence-level vector representation; use the sentence vector representation after CLN processing as input, and also pass through a linear layer and a sigmoid activation function to predict the position of the tail entity under different relationships. S35. After the CE-CasRel model training is completed, use the test set data to verify the trained model. After reaching the required precision and recall rates, retain the model weight parameters obtained from training.

2. The method for constructing a knowledge graph in the field of civil engineering specifications according to claim 1, wherein, The CEBERT encoder described in step S32 is specifically as follows: Use the Python crawler method to crawl a large amount of text from civil engineering-related encyclopedia entries and specification texts, perform data cleaning such as long text splitting and irrelevant content screening, and construct a civil engineering domain dataset; select the Chinese version of the RoBERTa pre-trained model, use the dataset as the input corpus, and input it into the model for fine-tuning. The training method and evaluation metrics refer to the original design of RoBERTa; finally, obtain the model CEBERT fine-tuned on the civil engineering domain dataset.

3. The method for constructing a knowledge graph in the field of civil engineering specifications according to claim 2, wherein, The CLN expression in step S31 is: Among them, y is the feature information input into the CLN structure, and c β and c β are respectively two pieces of conditional information to be fused as input; The sigmoid activation function expression in step S33 is:

4. The method for constructing a knowledge graph in the field of civil engineering specifications according to claim 3, characterized in that, The specific process of step S4 is: Repeat steps S1 and S2 to input the to-be-extracted specification text into the trained CE-CasRel model, and extract a set of triples that conform to the Schema label definition in step S23; through Python data processing, save the extracted triples line by line according to the articles, with one line for one specification article, and one article corresponding to several triples; store all articles into a json file.

5. The method for constructing a knowledge graph in the field of civil engineering specifications according to claim 4, characterized in that The specific steps of step S5 are: S51. Define the nodes in the Neo4j graph relationship library according to the entity labels described in Schema in step S23; define the edges in the Neo4j graph relationship library according to the relationship labels. S52. Use the json and pands libraries in Python to read the json file and read the triple set of each specification article line by line; extract and add to the Python list according to the defined Schema labels from the triple set. S53. Use the py2neo library in Python to connect Python to Neo4j; add the entities in the list in step 52 to the Neo4j nodes correspondingly. Add the relationships in the list in step 52 to the Neo4j edges correspondingly to complete the construction of the knowledge graph. S54. Start the Neo4j program through the console and view and save the graph in the browser.

6. A graph construction system for the civil engineering specification field knowledge graph construction method according to any one of claims 1 to 5, characterized in that It includes a preprocessing module, a knowledge extraction module, and a construction module; among them, A preprocessing module, which is used to perform sentence segmentation on the specification clauses and preprocess the segmented sentences to obtain convertible specification sentences; A knowledge extraction module, which is used for the deep neural network model CE-CasRel to extract triples from the specification clauses and obtain structured specification clauses saved in the form of triples; A construction module, which is used to construct a knowledge graph and store the triples in a graph database based on the py2neo library for the structured specification clauses.

Images