A Fine-Grained Entity Classification Method for Road and Utility Tunnel Standard Knowledge Graph Based on Prompt Learning

Abstract

The present invention discloses a fine-grained entity classification method for a road and integrated utility tunnel standard knowledge graph based on prompt learning, comprising the following steps: S1: Collect data, perform knowledge modeling and data annotation; S2: Train a model according to the training data and optimize the parameters; S3: Decode the sample labels according to the paths in the label ontology structure. It can make full use of the hierarchical structure of the label ontology and the ability of prompt learning to solve the few-shot problem in the low-resource professional field scenario, making the prediction results of the model more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly relates to a fine-grained entity classification method for a road and integrated utility tunnel standard knowledge graph based on prompt learning. Background Art

[0002] Currently, the paradigm of pre-trained language models and fine-tuning has been widely applied to downstream tasks in natural language processing. This method often requires a large amount of labeled sample data to obtain good results. In vertical fields, especially in professional and emerging fields such as road and integrated utility tunnel standard digitization, it is undoubtedly time-consuming and laborious to obtain a large amount of data and perform annotation.

[0003] With the emergence of prompt learning, it has demonstrated powerful capabilities in low-resource scenarios. However, there are still some problems in the current entity classification methods based on prompt learning. For example, in complex professional fields, there are often hierarchical relationships between ontologies, but most of the current models use flat inputs and do not fully utilize the relationships between labels. Currently, there are methods that use external data for enhancement and have achieved good results in general fields, but due to the lack of resources, such methods are difficult to implement in downstream vertical fields. Summary of the Invention

[0004] The present invention aims to solve the technical problems existing in the prior art, and particularly innovatively proposes a fine-grained entity classification method for a road and integrated utility tunnel standard knowledge graph based on prompt learning, which can make full use of the hierarchical structure of label ontologies and the ability of prompt learning to solve few-shot problems in low-resource professional field scenarios, making the prediction results of the model more accurate.

[0005] To achieve the above object, the present invention provides a fine-grained entity classification method for a road and integrated utility tunnel standard knowledge graph based on prompt learning, including the following steps:

[0006] S1: Collect data, perform knowledge modeling and data annotation;

[0007] S2: Train a model according to the training data and optimize the parameters;

[0008] S3: Decode the sample labels according to the paths in the label ontology structure.

[0009] In the above solution: Step S1 further includes:

[0010] Perform annotation on the collected data, divide the annotated data, and construct label words

[0011] This task is a multi-label classification task, and there is a tree-like hierarchical relationship among the labels. After collecting the relevant text data, entities and categories in the data are labeled according to the results of ontology modeling, and the labeled data is divided to construct a small-sample dataset.

[0012] In the above solution: Step S2 further includes:

[0013] S2-1: Initialize or update the representation of the label;

[0014] S2-2: Construct a mixed prompt template, dynamically aggregate hierarchical label information and inject it into the template;

[0015] S2-3: Encode the generated embedding through the BERT encoder;

[0016] h(T mix ) = [h(x), …, h(m), …, h([SOFT1]), h 1 ([MASK]), …, h([SOFT l )], h l ([MASK])]

[0017] Among them, h(·) represents the representation of the corresponding symbol after being encoded by BERT;

[0018] S2-4: Obtain the output of the model at the [MASK] position Calculate the probabilities belonging to each label respectively within the label range of each level

[0019] Calculate through the following formula:

[0020]

[0021] Among them, represents the probability that the output of the i-th layer is predicted as the j-th label of this layer, and Sigmoid(·) is the Sigmoid activation function;

[0022] S2-5: Calculate the loss of the model.

[0023] In the above solution: Step S2-1 further includes:

[0024] S2-1-1: Calculate the average vector after embedding;

[0025] Calculate the average vector of the label word after embedding through the BERT model;

[0026] The formula is as follows:

[0027]

[0028] Among them, v i represents the i-th labeled word, and d i represents the average vector after the embedding of the i-th labeled word. e(·) represents the embedding operation that converts the input text symbols into dense vector representations;

[0029] S2-1-2: Initialize or update the label representation D;

[0030] The d calculated through step S2-1 i Initializes or updates the label representation D:

[0031]

[0032] Among them, N d represents the total number of labels, and dim represents the dimension of the embedding.

[0033] In the above solution: Step S2-2 further includes:

[0034] S2-2-1: Randomly select a sample (x, m) from the collected data, where x and m represent the sample sentence and the given entity reference respectively;

[0035] S2-2-2: Pass the selected sample (x, m) through the pre-defined mixed prompt template T mix (x, m) is constructed into the following sample:

[0036] T mix (x, m) = x In this sentence, m belongs to [SOFT1][MASK]...[SOFT l [MASK]

[0037] Among them, the mixed prompt template contains natural language and l soft prompt words [SOFT], and l represents the level of the label ontology structure; each soft prompt word is followed by a [MASK] special token for predicting the label at this level;

[0038] S2-2-3: Pass the sample constructed in step S2-2-2 through the BERT embedding layer to convert the symbols into the following vector representation:

[0039] e(T mix ) = [e(x), …, e(m), …, e([SOFT1]), e([MASK]), …, e([SOFT l ),e([MASK])]

[0040] Among them, the representation e([SOFT i) Dynamically generated from the representations of labels and samples, representing the label information of each layer, and inserted into the embedded sequence;

[0041] S2-2-4: Obtain the soft prompt representation;

[0042] First, encode the label representation D through a graph attention network, which can better learn and represent the relationships between labels. Denote the encoding result as

[0043] D enc = CAT(D)

[0044] where GAT(·) represents the graph attention network;

[0045] S2-2-5: Group the labels according to the hierarchical relationship, then Obtain the soft prompt representations of each label level through cross-attention:

[0046]

[0047] where W cq 、W ck and W cv are all learnable weight parameters.

[0048] In the above solution: Step S2-2-3 further includes:

[0049] S2-2-3-1: Generate the representation of the sample instance; respectively obtain the representations of the context by the self-attention mechanism and average pooling for the left and right contexts of the input sample;

[0050] S2-2-3-2: Generate the representation of the sample instance for the left context:

[0051]

[0052] where e left represents the left context representation, AvePool(·) represents the average pooling operation, Softmax(·) represents the Softmax activation function, W sq 、W sk 、W sv are all learnable parameters; X left represents the left context;

[0053] S2-2-3-3: Generate the representation of the sample instance e right for the right context in the same way as in step S2-2-3-2;

[0054] S2-2-3-4: Obtain the reference representation e men through the following formula:

[0055]

[0056] Among them, m is the character referring to m in the sample (x, m).

[0057] S2-2-3-5: Fuse the context representation and the reference representation through a feed-forward neural network. The formula is as follows:

[0058] e ins = Concat(e left , e mem , e right )W a + b a

[0059] Among them, the representation of the generated sample instance dim represents the dimension of the embedding, W a and b a are learnable weights and biases, and Concat(·) represents the concatenation of vectors.

[0060] In the above solution: Step S2-5 further includes:

[0061] S2-5-1: Calculate the loss of the model through the binary cross-entropy loss function

[0062]

[0063] Among them, represents the true label. represents the j-th label in the i-th layer, represents the probability that the output of the i-th layer is predicted to be the j-th label of this layer; n i represents the number of labels in the i-th layer;

[0064] S2-5-2: Calculate the mean squared error loss of the model In order to make the output representation h i ([MASK]) of the special symbol "[MASK]" in the model as close as possible to the representation of the true label, the mean squared error loss is also added

[0065] Calculate the mean squared error loss of the model through the following formula

[0066]

[0067] Among them, t i is the representation of the true label of the sample in the i-th layer,

[0068] S2-5-3: Calculate the actual loss of the calculation model

[0069]

[0070] Where λ is the weight.

[0071] In the above solution: Step S3 further includes:

[0072] S3-1: Record the tree structure of the label ontology Where Y is the set of label nodes, E is the set of edges, and the true label of a certain sample is represented as one or more paths from the root node to the leaf node in the label ontology structure;

[0073] S3-2: Classify the nodes according to the paths, and there is Where N p is the number of leaf nodes;

[0074] S3-3: Assume that the node set of a certain path is represents the jth label in the ith layer, then the probability of selecting this path is calculated as:

[0075]

[0076] Where, represents the probability that the output of the lth layer is the kth label in this layer;

[0077] S3-4: Select the path according to the probability that each path calculated in step S3-3 is the true path.

[0078] In the above solution: Step S3-4 further includes:

[0079] S3-4-1: If the dataset labeled in step S1 is a single-path dataset, then execute S3-4-3; if it is a multi-path dataset, then execute S3-4-2;

[0080] S3-4-2: If it is a multi-path dataset, then first set a threshold according to the effect of the validation set. If the probability of each path is less than this threshold, execute S3-4-3; otherwise execute S3-4-4;

[0081] S3-4-3: Select the path with the highest probability, and output the node labels on this maximum path as the predicted label of the final decoding, and execute S3-4-5;

[0082] S3-4-4: Output all the labels on the paths that exceed this threshold as the predicted labels of the model, and execute S3-4-5;

[0083] S3-4-5: End.

[0084] In summary, the beneficial effects of the present invention are as follows: a hierarchical perception hybrid prompt template suitable for entity classification tasks is proposed, and the hierarchical structure of the labels is reflected in the prompt template; dynamic label information aggregation is proposed, and according to the samples and contexts, the label information at each level is dynamically aggregated and injected into the input template; a label word decoding strategy based on label paths is proposed, and by comprehensively considering the label prediction results at different levels, the performance of the model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 is a schematic flowchart of the present invention;

[0086] Figure 2 is a schematic diagram of the overall model structure;

[0087] Figure 3 is a schematic diagram of dynamic label aggregation;

[0088] Figure 4 is the label ontology structure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0089] The present invention will be further described below through embodiments in conjunction with the drawings:

[0090] As Figure 1 shown, a fine-grained entity classification method for a road and integrated utility tunnel standard knowledge graph based on prompt learning includes the following steps:

[0091] S1: Collect data, perform knowledge modeling and data annotation;

[0092] Annotate according to the collected data, divide the annotated data, and construct label words

[0093] This task is a multi-label classification task and there is a tree-like hierarchical relationship between the labels. After collecting the relevant text data, the entities and categories in the data are annotated according to the results of ontology modeling, and the annotated data is divided to construct a small sample data set.

[0094] S2: Train the model according to the training data and optimize the parameters. Build a hierarchical perception fine-grained entity classification model based on prompt learning, train it with the constructed data set, and complete the optimization and update of the model parameters, including the following steps:

[0095] S2-1: Initialize or update the representation of the labels;

[0096] S2-1-1: Calculate the average vector after embedding;

[0097] Calculate the average vector after embedding of the label words through the BERT model ;

[0098] The formula is as follows:

[0099]

[0100] Among them, v i represents the i-th label word, and d i represents the average vector after the embedding of the i-th label word. e(·) represents the embedding operation, which converts the input text symbols into dense vector representations;

[0101] S2-1-2: Initialize or update the label representation D;

[0102] The d calculated through step S2-1 i Initializes or updates the label representation D:

[0103]

[0104] Among them, N d represents the total number of labels, and dim represents the dimension of the embedding;

[0105] S2-2: Construct a mixed prompt template, dynamically aggregate hierarchical label information and inject it into the template;

[0106] S2-2-1: Randomly select a sample (x, m) from the collected data, where x and m represent the sample sentence and the given entity reference respectively;

[0107] S2-2-2: Pass the selected sample (x, m) through the pre-defined mixed prompt template T mix (x, m) is constructed into a sample as follows:

[0108] T mix (x, m) = x In this sentence, m belongs to [SOFT1][MASK]...[SOFT l [MASK]

[0109] Among them, the mixed prompt template contains natural language and l soft prompt words [SOFT], where l represents the level of the label ontology structure; each soft prompt word is followed by a [MASK] special token for predicting the label at that level;

[0110] S2-2-3: Pass the sample constructed in step S2-2-2 through the BERT embedding layer to convert the symbols into a vector representation as follows:

[0111] e(T mix ) = [e(x), …, e(m), …, e([SOFT1]), e([MASK]), …, e([SOFT l), e([MASK])]

[0112] Among them, the representation e([SOFT i ) of the soft prompt is dynamically generated from the representations of the labels and samples, representing the label information of each layer, and inserted into the embedded sequence; for the convenience of expression, the hard prompt and special tokens (such as "[CLS]" and "[SEP]") are removed;

[0113] The generation of the representation e([SOFT i ) includes the following steps:

[0114] S2-2-3-1: Generate the representation of the sample instance; respectively obtain the representations of the context through the self-attention mechanism and average pooling for the left and right contexts of the input sample;

[0115] S2-2-3-2: Generate the representation of the sample instance for the left context:

[0116]

[0117] Among them, e left represents the left context representation, AvePool(·) represents the average pooling operation, Softmax(·) represents the Softmax activation function, and W sq , W sk , W sv are all learnable parameters; X left represents the left context;

[0118] S2-2-3-3: Generate the representation of the sample instance for the right context in the same way as in step S2-2-3-2, that is:

[0119]

[0120] Among them, e right represents the right context representation, and X right represents the right context;

[0121] S2-2-3-4: Obtain the reference representation e men through the following formula:

[0122]

[0123] Among them, m is the character of the reference m in the sample (x, m);

[0124] S2-2-3-5: Fuse the context representation and the reference representation through a feed-forward neural network, and the formula is as follows:

[0125] e ins = Concat(eleft , e mem , e right )W a +b a

[0126] Among them, the representation of the generated sample instance dim represents the dimension of the embedding, W a and b a are learnable weights and biases, and Concat(·) represents the concatenation of vectors;

[0127] S2-2-4: Obtain the soft prompt word representation;

[0128] First, encode the label representation D through the graph attention network, which can better learn and represent the relationship between labels. Denote the encoding result as

[0129] D enc = GAT(D)

[0130] Among them, GAT(·) represents the graph attention network;

[0131] S2-2-5: Group the labels according to the hierarchical relationship, then Obtain the soft prompt word representation of each label level through cross-attention:

[0132]

[0133] Among them, W cq , W ck and W cv are all learnable weight parameters;

[0134] S2-3: Encode the generated embedding through the BERT encoder;

[0135] h(T mix ) = [h(x), …, h(m), …, h([SOFT1), h 1 ([MASK]), …, h([SOFT l ), h l ([MASK])]

[0136] Among them, h(·) represents the representation of the corresponding symbol after being encoded by BERT;

[0137] S2-4: Obtain the output at the [MASK] position of the model Calculate the probability belonging to each label within the label range of each level respectively

[0138] Calculate through the following formula:

[0139]

[0140] Among them, represents the probability that the output of the i-th layer is predicted as the j-th label of this layer, and Sigmoid(·) is the Sigmoid activation function;

[0141] S2-5: Calculate the loss of the model;

[0142] S2-5-1: Calculate the loss of the model through the binary cross-entropy loss function

[0143]

[0144] Among them, represents the true label. represents the j-th label of the i-th layer, represents the probability that the output of the i-th layer is predicted as the j-th label of this layer; n i represents the number of labels in the i-th layer;

[0145] S2-5-2: Calculate the mean square error loss of the model In order to make the output representation h i ([MASK]) of the special symbol "[MASK]" in the model as close as possible to the representation of the true label, the mean square error loss is also added

[0146] The mean square error loss of the model is calculated through the following formula

[0147]

[0148] Among them, t i is the representation of the true label of the sample in the i-th layer,

[0149] S2-5-3: Calculate the actual loss of the model

[0150]

[0151] Among them, λ is the weight.

[0152] S3: Decode the sample label according to the path in the label ontology structure;

[0153] Through the above steps, the probabilities belonging to the labels of each layer can be obtained If the label with the highest probability or exceeding the set threshold is directly selected as the output of the model, the relationship between the labels of each level will be ignored;

[0154] S3-1: Record the tree structure of the label ontology Where Y is the set of label nodes, and E is the set of edges. The true label of a certain sample is represented as one or more paths from the root node to the leaf node in the label ontology structure;

[0155] S3-2: Classify the nodes according to the paths, and there is Where N p Is the number of leaf nodes;

[0156] S3-3: Assume that the set of nodes of a certain path is Indicates the j-th label in the i-th layer, then the probability of selecting this path is calculated as:

[0157]

[0158] Among them, Indicates that the output of the l-th layer is the probability of the k-th label in this layer;

[0159] S3-4: Select the path according to the probability that each path calculated in step S3-3 is the true path;

[0160] S3-4-1: If the dataset labeled in step S1 is a single-path dataset, then execute S3-4-3; if it is a multi-path dataset, then execute S3-4-2;

[0161] S3-4-2: If it is a multi-path dataset, then first set a threshold according to the effect of the validation set. If the probability of each path is less than this threshold, execute S3-4-3; otherwise execute S3-4-4;

[0162] S3-4-3: Select the path with the highest probability, and output the node labels on this maximum path as the predicted label of the final decoding, and execute S3-4-5;

[0163] S3-4-4: Output all the labels on the paths that exceed this threshold as the predicted labels of the model, and execute S3-4-5;

[0164] S3-4-5: End.

[0165] The following is an example based on the field of road and utility tunnel standard digitization to illustrate this embodiment, and it should not be construed as a limitation of the present invention.

[0166] (1) Collect data and perform knowledge modeling and data annotation.

[0167] Collect standards related to roads and utility tunnels, and obtain standard texts through OCR conversion and manual adjustment. Refer to the Classification and Coding Standard for Building Information Models (GB / T 51269-2017) and the actual standard provisions for organization, and design 5 categories of coarse-grained entity categories and 10 categories of fine-grained entity categories, such as Figure 4 shown. Then, manually annotate the standard data. To simulate a low-resource scenario, for the annotated data, randomly sample 5 pieces for each category to construct the training set and the validation set, and the remaining data is used as the test set.

[0168] (2) Train the model based on the training data and optimize the parameters.

[0169] 1) Initialize or update the representation of the labels.

[0170] For the label words V = {v1, v2,..., v 15} = {construction achievements, construction attributes,..., modifiers} designed in the previous step, perform word segmentation and pass through the embedding layer of the BERT model, and average the embedding vectors to obtain the label representation D, and the gradient can be calculated for parameter update.

[0171]

[0172] D = [d1; d2;...; d 15

[0173] 2) Construct a mixed prompt template, dynamically aggregate hierarchical label information and inject it into the template.

[0174] Given a sample, first wrap it with the mixed prompt template. For example, for the sample x: "The maximum longitudinal slope of the road in the tunnel should not be greater than 3.0%." and the mention m: "maximum longitudinal slope", the following template T mix can be obtained: "The maximum longitudinal slope of the road in the tunnel should not be greater than 3.0%. In this sentence, the maximum longitudinal slope belongs to [SOFT1][MASK][SOFT2][MASK]".

[0175] Pass the above template through the BERT embedding layer, and the symbols can be converted into vector representations:

[0176] e(T mix ) = [e(x),..., e(m),..., e([SOFT1]), e([MASK]), e([SOFT2]), e([MASK])]

[0177] In the above formula, for the sake of convenience of expression, the hard prompt words and special markers (such as "[CLS]" and "[SEP]") are deleted.

[0178] Among them, the representation of the soft prompt word e([SOFT i ​) Dynamically generated from the representations of labels and samples, representing the label information of each layer, and inserted into the embedded sequence. The generation method of the soft prompt representation is as follows:

[0179] ① Generate the representation of the sample instance:

[0180] a) Respectively obtain the representations of the left and right contexts of the input sample through the self-attention mechanism and average pooling. Taking the left context as an example:

[0181]

[0182] Among them, W sq 、W sk 、W sv are all learnable parameters. The right context representation e right is obtained in a similar way as the above formula.

[0183] b) Obtain the representation e men of the reference, which is obtained by averaging the embedding vectors obtained from the reference passing through the embedding layer of the BERT model.

[0184]

[0185] c) Fuse the context representation and the reference representation through a feed-forward neural network to generate the representation dim represents the dimension of the embedding.

[0186] e ins = Concat(e left , e mem , e right )W a + b a

[0187] In the above formula, W a and b a are learnable weights and biases.

[0188] ② Obtain the soft prompt representation:

[0189] Denote the representation of the label as Among them, N d represents the total number of labels. First, encode the label representation D through a graph attention network, and the result is

[0190] D enc = GAT(D)

[0191] Group the labels according to the hierarchical relationship, and there is Then obtain the soft prompt representations of each label level through cross-attention:

[0192]

[0193] In the above formula, W cq , W ck and W cv are all learnable weight parameters.

[0194] 3) Encode the generated embeddings through the BERT encoder:

[0195] h(T mix ) = [h(x), …, h(m), …, h([SOFT1]), h 1 ([MASK]), h([SOFT2]), h 2 ([MASK])]

[0196] 4) Obtain the output at the [MASK] position of the model Calculate the probabilities belonging to each label within the label range of each layer respectively:

[0197]

[0198] 5) During the training process of the model, the loss functions used are binary cross-entropy and mean squared error loss:

[0199]

[0200]

[0201] (3) Decode the sample labels according to the paths in the label ontology structure.

[0202] Through the above steps, the probabilities belonging to the labels of each layer can be obtained According to the ontology structure of the labels, the nodes can be divided into multiple paths

[0203] Among them, P1 = {construction achievements, buildings}, P2 = {construction achievements, building spaces}, and so on. Then, calculate the probabilities of each path. For example, the probability of path P1 is:

[0204]

[0205] The probabilities of the remaining paths are calculated by following the above formula, and p = {p1, p2,..., p 10} is obtained.

[0206] Since the labels of the samples in this dataset are all on a single path, select the nodes on the path with the highest probability as the decoded labels. For example, if max(p) = p4 is output for the sample, the decoded label is P4 = {construction attributes, attribute indicators}.

[0207] Compared with traditional fine-tuning methods and prompt learning methods (hard templates, soft templates), the experimental results are shown in the following table, and the evaluation metrics include strict accuracy (ACC), macro F1 score (MaF), and micro F1 score (MiF):

[0208] model ACC MaF MiF VanillaFT 48.96 53.67 53.67 Hard Prompt 45.54 61.35 61.35 Soft Prompt 46.93 62.87 62.87 the method of the present invention 61.29 67.94 67.94

[0209] It can be seen from this that by comparing the various values in the table, the method of the present invention has significantly improved in all metrics.

Claims

1. A fine-grained entity classification method for standard knowledge graphs of roads and integrated pipe corridors based on prompt learning, characterized by: The following steps are involved: S1: Collect data, conduct knowledge modeling and data annotation; S2: Train the model based on the training data and optimize the parameters; S2-1: Initialize or update the label representation; S2-2: Construct a hybrid prompt template, dynamically aggregate hierarchical label information and inject it into the template; Step S2-2 also includes: S2-2-1: Randomly select a sample (x, m) from the collected data, where x and m represent the sample sentence and the given entity reference respectively; S2-2-2: The selected sample (x, m) is passed through the pre-defined mixed prompt template T mix (x, m) is constructed into a sample as shown below: T mix (x, m) = x In this sentence, m belongs to [SOFT1][MASK]...[SOFT l ][MASK] The hybrid prompt template contains natural language and l soft prompt words [SOFT], l represents the level of the label ontology structure; each soft prompt word is followed by a [MASK] special tag to predict the label of that level; S2-2-3: The sample constructed in step S2-2-2 is converted into a vector representation as shown below through the BERT embedding layer: e(T mix )=[e(x),...,e(m),...,e([SOFT1]),e([MASK]),...,e([SOFT l ]),e([MASK])] Among them, the representation of soft prompt words ([SOFT i ]) is dynamically generated by the representation of labels and samples, representing the label information of each layer and inserted into the embedded sequence; S2-2-3-1: Generate representation of sample instance; respectively obtain the representation of context by using self-attention mechanism and average pooling for the left and right context of input sample; S2-2-3-2: Characterization of sample instances generated from the left context: Among them, e left represents the left context representation, AvePool(·) represents the average pooling operation, Softmax(·) represents the Softmax activation function, and W sq , W sk , W sv are all learnable parameters; X left Indicates the left context; S2-2-3-3: Generate a representation of the sample instance for the right context in the same way as step S2-2-3-2 right ; S2-2-3-4: Obtain the referential representation e through the following formula men : Among them, m is the reference in the sample (x, m). m Characters; S2-2-3-5: The context representation and the reference representation are fused through a feedforward neural network. The formula is as follows: e ins =Concat(e left ,e mem ,e right )W a +b a Among them, the representation of the generated sample instance dim represents the dimension of embedding, W a and b a are learnable weights and biases, Concat(·) represents the concatenation of vectors; S2-2-4: Get soft prompt word representation; First, the label representation D is encoded through the graph attention network, which can better learn and represent the relationship between labels. The encoding result is recorded as <h2 style=";text-align:left;direction:ltr">D<h2 style=";text-align:left;direction:ltr"> enc <h2 style=";text-align:left;direction:ltr"> (GAT(D)) Among them, GAT(·) represents the graph attention network; S2-2-5: Group the tags according to the hierarchical relationship. Then, cross attention is used to obtain the soft prompt word representations at each tag level: Among them, W cq , W ck and W cv All are learnable weight parameters; S2-3: Encode the generated embedding through the BERT encoder; h(T mix )=[h(x),...,h(m),...,h([SOFT1]),h 1 ([MASK]),...,h([SOFT l ]),h l ([MASK])] Among them, h(·) represents the representation of the corresponding symbol after BERT encoding; S2-4: Get the output of the model [MASK] position Calculate the probability of belonging to each label within the label range of each level Calculate using the following formula: in, represents the probability that the output of the i-th layer is predicted to be the j-th label of the layer, and Sigmoid(·) is the Sigmoid activation function; S2-5: Calculate the loss of the model; S3: Decode and classify sample labels according to the paths in the label ontology structure.

2. According to the prompt learning-based fine-grained entity classification method of road and integrated pipe gallery standard knowledge graph according to claim 1, it is characterized by: Step S1 also includes: Label the collected data, divide the labeled data, and construct label words 3. According to the prompt learning-based fine-grained entity classification method of road and integrated pipe gallery standard knowledge graph according to claim 1, it is characterized by: Step S2-1 also includes: S2-1-1: Calculate the average vector after embedding; Calculate label words through BERT model The average vector after embedding; The formula is as follows: Among them, v i represents the i-th label word, d i represents the average vector of the i-th label word after embedding, and e(·) represents the embedding operation, which converts the input text symbol into a dense vector representation; S2-1-2: Initialize or update the tag representation D; The d calculated in step S2-1-1 i Initialize or update the label representation D: in, N d represents the total number of labels, and dim represents the dimension of embedding.

4. According to the prompt learning-based fine-grained entity classification method of road and integrated pipe gallery standard knowledge graph according to claim 1, it is characterized by: Step S2-5 also includes: S2-5-1: Calculate the model loss through the binary cross entropy loss function in, Indicates the true label; represents the jth label of the i-th layer, represents the probability that the output of the i-th layer is predicted to be the j-th label of the layer; n i Indicates the number of labels in the i-th layer; S2-5-2: Calculate the mean square error loss of the model In order to make the output of the special symbol "[MASK]" in the model represent h i ([MASK]) Try to be as close to the representation of the true label as possible, and also increase the mean square error loss The mean square error loss of the model is calculated by the following formula Among them, t i is the representation of the true label of the sample at layer i, S2-5-3: Calculate the actual loss of the model Among them, λ is the weight.

5. According to the prompt learning-based fine-grained entity classification method of standard knowledge graph of roads and integrated pipe corridors according to claim 1, it is characterized by: Step S3 also includes: S3-1: Tree structure of the tag body Where Y is the set of label nodes, E is the set of edges, and the true label of a sample is represented by one or more paths from the root node to the leaf node in the label ontology structure; S3-2: Classify nodes according to paths. Where N p is the number of leaf nodes; S3-3: Assume that the node set of a certain path is represents the jth label of the i-th layer, then the probability of selecting this path is calculated as: in, Indicates the probability that the output of the lth layer is the kth label of this layer; S3-4: Select a path based on the probability that each path calculated in step S3-3 is a true path.

6. According to claim 5, a method for fine-grained entity classification of standard knowledge graphs of roads and integrated utility tunnels based on prompt learning is characterized by: Step S3-4 also includes: S3-4-1: If the data set marked in step S1 is a single-path data set, execute S3-4-3; if it is a multi-path data set, execute S3-4-2; S3-4-2: If it is a multi-path data set, first set a threshold according to the effect of the verification set. If the probability of each path is less than the threshold, execute S3-4-3; otherwise, execute S3-4-4; S3-4-3: Select a path with the highest probability, and output the node label on the highest probability path as the predicted label for final decoding, and execute S3-4-5; S3-4-4: Output all labels on the path that exceed the threshold as the model's predicted labels and execute S3-4-5; S3-4-5: Over.

Images