Urban operation work order recommendation method and application of knowledge graph

Abstract

The invention relates to the technical field of work order processing, and discloses an urban operation work order recommendation method, which comprises the following steps: S1, data preprocessing: preprocessing an input work order text, and eliminating invalid information; s2, calculating the similarity by using a CoSENT model: the model adopts annotation data in a sentence pair form, two sentences are subjected to BERT to respectively obtain vectors u and v, and the similarity of positive samples can be improved through a loss function based on cosine similarity (in a prediction stage); s3, work order entity identification is carried out, wherein a bi-directional long and short time memory network-conditional random field model (Bi-Directional Long Short Term Model-Conditional Random Field, BiLSTM-CRF) is utilized to carry out named entity identification; according to the method, the feature values of the urban transportation hotline work orders are calculated, so that processing methods required by order sending personnel for different events can be quickly searched for processing methods of historical similar work orders in combination with entity relationships between knowledge maps in historical data, and a reference basis is provided for traditional order sending.

Description

Technical Field

[0001] The present invention relates to the technical field of work order processing, and specifically to an application of a method for recommending urban operation work orders and a knowledge graph. Background Art

[0002] In recent years, grass-roots social governance practices across the country have been undergoing a transformation towards digital and intelligent governance; how to improve the level of intelligent social governance and efficiently process a large amount of governance data is the top priority in building a social governance pattern of co-construction, co-governance and sharing.

[0003] Hotline work orders are relatively effective in resolving residents' complaints about public affairs, and the number of work orders in each city is now relatively large. During the process of handling hotline work orders, by screening work orders with relatively high similarity, the root causes of such work orders can be effectively identified, better disposal cases can be selected from similar work order handling methods, and problems of the same type of work orders can be solved at one time through merging, correlation analysis, etc.

[0004] As an important branch of knowledge engineering, the knowledge graph has unique advantages in modeling social governance data. The "entity-relationship" representation method can intuitively represent intricate relationships, and the node-based logical storage method can also flexibly expand the storage scale. Therefore, this project will build a professional knowledge graph for social governance based on large-scale social governance data to achieve various downstream tasks such as policy text recommendation, intelligent question answering, semantic understanding, and intelligent retrieval, effectively improving the efficiency of government departments in handling various grass-roots social governance events.

[0005] The search and query of existing work order systems cannot solve the problem of similar queries for the semantic description of work orders, and can only perform simple keyword retrieval on data through preset SQL statements. It is also impossible to query the association relationship between work orders in the existing large amount of data. Therefore, it is necessary to establish relevant algorithm models to structure and visualize work order data, extract its key information, so as to achieve flexible use of the data, find the most similar work orders, and provide a reference basis and the association relationship between work orders. Summary of the Invention

[0006] (1) Technical Problems to be Solved

[0007] Aiming at the deficiencies of the existing technology, the present invention provides an application of a method for recommending urban operation work orders and a knowledge graph, which has the advantages of extracting the description feature of the demands of the urban hotline, capturing keywords, and analyzing conditional probabilities, and solves the problem that the search and query of the existing work order system cannot solve the problem of similar queries for the semantic description of work orders, and can only perform simple keyword retrieval on data through preset SQL statements.

[0008] (2) Technical Solutions

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for recommending urban operation work orders, comprising the following steps:

[0010] S1, data preprocessing:

[0011] Pre-process the input work order text to remove invalid information;

[0012] S2, calculate the similarity using the CoSENT model:

[0013] The model uses labeled data in the form of sentence pairs. After running the two sentences through BERT, the vectors u and v are obtained respectively. The loss function based on cosine similarity (prediction stage) is used to improve the similarity of positive samples.

[0014] S3, work order entity recognition:

[0015] Use Bi-Directional Long Short Term Model-Conditional Random Field (BiLSTM-CRF) for named entity recognition to meet the entity extraction requirements in knowledge graph data import;

[0016] S4, relation extraction:

[0017] Use Piece-Wise-CNN (PCNN) for relation extraction. PCNN is an improvement on CNN. It divides sentences into three segments and performs independent maximum pooling when entities are known to achieve feature extraction.

[0018] S5, build domain-level knowledge graph:

[0019] After completing entity recognition and relationship classification, the data is imported into the neo4j graph database to generate a visual knowledge graph;

[0020] S6, build a question-answering model based on knowledge graph semantics:

[0021] The algorithm implements knowledge graph question-answering queries through the entity linking system and Cypher statements, meeting the requirements of knowledge graph statistical question-answering functions and returning highly similar semantic data.

[0022] In some embodiments, the S1 includes:

[0023] S11: Eliminate invalid structured information such as "Current demand:" and "Original task number:", perform traditional and simplified Chinese conversion, full-width character conversion, data encoding, and Chinese word segmentation and part-of-speech tagging to achieve the unification of Chinese and English symbols, thereby providing a more reliable data foundation for subsequent text mining, sentiment analysis, text classification and other tasks.

[0024] In some embodiments, the S2 includes:

[0025] S21 uses the CoSENT model, which is based on the BERT pre-trained language model and meets the requirements of text semantic matching tasks. The goal of the CoSENT model is to supervise the training of a sentence vector representation model based on the BERT pre-trained language model architecture, so that the model can achieve the best performance in text semantic matching tasks.

[0026] In some embodiments, the S3 includes:

[0027] S31, during the entity recognition process, the BIO annotation method is used to set a corresponding label for each word in the sentence. B-[type] indicates that the word is the beginning of a named entity of type type; I-[type] indicates that the word is a non-starting word of a named entity of type type; O indicates that the word is an ordinary word.

[0028] In some embodiments, the S6 includes:

[0029] S61, the data input of the algorithm is a well-constructed event knowledge graph and the corresponding formalized question. The data output returns the corresponding answer entity.

[0030] (3) Beneficial effects

[0031] Compared with the existing technology, the present invention provides an urban operation work order recommendation method and the application of knowledge graph, which has the following beneficial effects:

[0032] 1. The application of this urban operation work order recommendation method and knowledge graph can provide dispatchers with the required handling methods for different events by calculating the characteristic values of urban operation hotline work orders. By combining the entity relationships between knowledge graphs in historical data, it can quickly find the handling methods of similar historical work orders, providing a reference basis for traditional dispatching.

[0033] 2. The city's work order recommendation method and the application of knowledge graphs can assist managers in discovering similarities and correlations between work orders through the domain-level hotline work order knowledge graph, and solve the problem of regionalized multiple complaints, thereby achieving the ability to intervene in advance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1Schematic diagram of the structure of the BERT model of the present invention;

[0035] Figure 2 Schematic diagram of the architecture of the BiLSTM-CRF sequence labeling model of the present invention;

[0036] Figure 3 Schematic diagram of the architecture of the PCNN model of the present invention;

[0037] Figure 4 Domain-level knowledge graph of the present invention. Detailed implementation manners

[0038] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0039] As an important branch of knowledge engineering, the knowledge graph has unique advantages in modeling social governance data. The "entity-relationship" representation method can intuitively represent complex relationships, and the node-based logical storage method can also flexibly expand the storage scale. Therefore, this project will build a professional knowledge graph for social governance based on large-scale social governance data to achieve various downstream tasks such as policy text recommendation, intelligent question answering, semantic understanding, and intelligent retrieval, and effectively improve the efficiency of government departments in handling various grass-roots social governance events.

[0040] The search and query of the existing work order system cannot solve the similar query for the semantic description of the work order, and can only perform simple keyword retrieval on the data through preset SQL statements. It is also impossible to query the association relationship between work orders in the existing massive data. Therefore, it is necessary to establish relevant algorithm models to structure and visualize the work order data, extract its key information, so as to achieve flexible use of the data, find the most similar work orders, and provide a reference basis and the association relationship between work orders.

[0041] To solve the problems in the related technologies to a certain extent, the embodiments of the present application provide an application of a method for recommending urban operation work orders and a knowledge graph. By calculating the characteristic values of the urban operation hotline work orders, it is possible to quickly find the handling methods of historical similar work orders for the dispatchers' handling methods required for different events in the historical data in combination with the entity relationships in the knowledge graph, providing a reference basis for traditional dispatching.

[0042] Next, the present application will be described in conjunction with the accompanying drawings and with reference to specific embodiments:

[0043] In conjunction withFigures 1 - 4 , embodiments of the present application provide a method for recommending urban operation work orders, including the following steps:

[0044] S1, data preprocessing:

[0045] Preprocess the input work order text to eliminate invalid information;

[0046] BERT first encodes the input text into a vector representation, and then obtains a vector representation rich in semantic information in the output layer through multiple layers of Encoders.

[0047] The internal architecture of the Encoder module is as follows, consisting of the attention mechanism represented by the Multi-Head Attention module and the non-linear mapping represented by the Feed Forward module. The role of the attention mechanism is to enable the model to focus on the more important parts of the input text, that is, to distinguish the influence of each word and phrase in the input text on the output and its importance to the overall text semantics. The Feed Forward module maps the input data from one space to another high-dimensional space, which can extract more complex and abstract text features and enhance the model's semantic understanding ability of the text.

[0048] When training the CoSENT model, for positive sample pairs (xi, xi+) and negative sample pairs (xi, xi-), it is desired to satisfy the following relationship:

[0049] cos(h i , h i+ ) > cos(h i , h i- )

[0050] Where h i is the sentence vector of x i . Optimize this loss, the greater the difference, the more beneficial. Based on this, the following cosine-based ranking loss function is designed:

[0051]

[0052] Where t is an adjustable hyperparameter.

[0053] In this patent, CoSENT uses the text2vec-base-chinese-paraphrase pre-trained weights to extract the sentence vectors of the work order text, and calculates the final text similarity value.

[0054] S2, calculate the similarity using the CoSENT model:

[0055] The model uses labeled data in the form of sentence pairs. After passing two sentences through BERT, vectors u and v are obtained respectively. Through a loss function based on cosine similarity (in the prediction stage), the result can improve the similarity of positive samples.

[0056] S3, Work order entity recognition:

[0057] Use the Bi-Directional Long Short Term Model-Conditional Random Field (BiLSTM-CRF) to perform named entity recognition, meeting the entity extraction requirements in the import of knowledge graph data.

[0058] BiLSTM-CRF learns the probability that each word in the current context may be labeled with different labels as its feature representation through the BiLSTM network, and then uses CRF for decoding to obtain the annotation sequence with the maximum joint probability.

[0059] 1) Path score calculation

[0060] Each word in the sentence corresponds to different labels with different probabilities, forming multiple paths. To obtain the path with the highest score, calculate the score of each path through the following formula:

[0061]

[0062] Where x is the input vector, which can use a randomly initialized vector or a pre-trained word vector; θ is the model parameter of BiLSTM; is the score of the t-th word corresponding to the i-th label; A is the transition matrix, the model parameter of CRF, A i，j represents the transition probability from state i to state j; s is the path score.

[0063] 2) Model training

[0064] The loss function of the model is calculated through the CRF score, and y′ is the label

[0065]

[0066] To optimize the calculation process, take the logarithmic function of the function

[0067]

[0068] This loss function is a convex function, and the model training obtains the global optimal solution through the gradient descent method.

[0069] 3) Entity recognition

[0070] For the calculated optimal path, use the Viterbi algorithm to decode, obtain the corresponding sequence annotation, and then obtain the entities and their types in the sentence through the annotation logic.

[0071] θ = {B-Entity: [0.2, 0.3, 0.5], I-Entity: [0.1, 0.4, 0.6], O: [0.3, 0.2, 0.4]}

[0072] The transition probability matrix A = {

[0073] B-Entity I-Entity O STOP

[0074] B-Entity 0.1 0.2 0.7 0.9

[0075] I-Entity 0.3 0.1 0.6 0.8

[0076] O 0.4 0.2 0.4 0.7

[0077] START 0.1 0.1 0.1 0.1

[0078] }

[0079] S4, Relation Extraction:

[0080] Use Piece-Wise-CNN (PCNN) for relation extraction. PCNN is an improvement of CNN. The sentence is divided into three segments, and independent max-pooling is performed in the case of known entities to achieve feature extraction;

[0081] Given the sentence to be extracted and the corresponding identified entities, the algorithm finally performs the relation extraction task through the softmax operation and outputs the relation category labels for each pair of entities.

[0082] Vector Representation

[0083] In natural language processing, in addition to using randomly generated or pre-trained word vectors, position encoding needs to be added to consider the distance between each word and entity in the sentence. For example: "There is noise disturbing residents in Spring Community", this sentence contains two entities, so two encodings of the same length as the sentence are generated, namely p1 = [0, 0, 0, 0, 1, 2, 3, 4, 5, 6], and p2 = [-6, -5, -4, -3, -2, -1, 0, 0, 0, 0].

[0084] Assume the dimension of the word vector is d w , the dimension of the position encoding is d p , and the finally used word vector dimension is d = d w +d p *2.

[0085] Convolution and Pooling

[0086] Perform a convolution operation on the input data using a convolution kernel to capture local features of entity pairs and their contexts. The size of the convolution kernel can be adjusted to capture context information of different sizes. For each convolution kernel, piecewise pooling extracts the "piecewise" max-pooling value associated with the entity pair. This helps capture the local relationships between entity pairs.

[0087] Objective Function

[0088] During training, a multi-instance learning design is adopted for the objective function. Assume the input to the model is a bag, and there are a total of T bags, {M1, M2,..., M T}, the i-th bag has q i sentences, and each sentence is an instance, and the instances are independent of each other. After passing through the softmax operation, we get:

[0089]

[0090] where o is the output result of the instance after passing through the model, and o r represents the score for the r-th class. For the training set (M i , y i ), the objective function uses cross-entropy loss at the bag level:

[0091]

[0092] During training, select the instance with the highest score for the annotation result from the bag for error backpropagation calculation.

[0093] Final output, each entity is labeled with a specific relationship category, such as "department", "category", etc.

[0094] S5, Construct a domain-level knowledge graph:

[0095] After completing entity recognition and relationship categories, introduce the data into the neo4j graph database to generate a visual knowledge graph;

[0096] S6, Build a question-answering model based on the semantics of the knowledge graph:

[0097] The algorithm realizes knowledge graph question-answering queries through an entity linking system and Cypher statements, meeting the requirements of knowledge graph statistical question-answering functions and returning highly similar semantic data.

[0098] The data input of the algorithm is a well - constructed event knowledge graph and the corresponding formalized question. The data output returns the corresponding answer entity.

[0099] The query graph consists of four types of nodes: fixed entities (i.e., topic entities, represented by rounded rectangles in the graph), existing variables (represented by circles), λ - variables (i.e., unknown variables, represented by shaded circles), and aggregation functions (represented by diamonds). Fixed entities represent existing entities in the knowledge graph, and the corresponding existing variables and entities with unfixed λ - variables. It is hoped to retrieve all entities that can be mapped to λ - variables as the final answers. Here, the design of the aggregation function is used to perform function operations on specific entities, usually able to capture some numerical features. In addition, the nodes in the query graph are connected by edges, and the edges are labeled with predicates in the knowledge graph.

[0100] For each entity e in the knowledge graph, the system first prepares a surface - form vocabulary list that contains all possible ways in which entity e may be mentioned in the text. Then, for each pair of mentions and entities, a score is calculated based on the word - frequency statistics in the surface vocabulary list. To avoid potential errors in the entity - linking system and explore more possible query graphs, the top 10 entities ranked by score are retained as topic entities.

[0101] After the topic entity is determined, it is necessary to construct a query graph on the entire kg to determine the relationship between the topic entity and the answer. Map the question into a pattern, i.e., using a general symbol <e>To replace the entity mention, compare it with the candidate reasoning chain, and compare the keywords cut by the question with the subgraph sequence. The model consists of two neural networks, one for calculating the pattern and the other for calculating the reasoning chain. The outputs of both models are vectors, and cosine similarity is used to represent the semantic similarity between them.

[0102] The calculation formula for measuring the matching degree between the problem pattern and the subgraph sequence is:

[0103]

[0104] Among them, Q is the question entity, S is the subgraph sequence, Match(Q i , S i ) is the matching degree between the i-th element in the pattern and the i-th entity element in the subgraph sequence.

[0105] The above process forms a query graph of the core reasoning chain, which can be searched in the knowledge graph to obtain answers.

[0106] To ensure a limited set of candidate answer entities, constraints are applied to the query graph containing the core reasoning chain. The algorithm first uses the core reasoning chain as the query graph to search for connections between variables x and y in the knowledge graph. It then enumerates all neighboring nodes of these entities to derive a complete set of constraints. Finally, the corresponding Cypher statements are used to find the resulting values of the relevant entities in the knowledge graph and the question.

[0107] In some embodiments, the S1 includes:

[0108] S11: Eliminate invalid structured information such as "Current demand:" and "Original task number:", perform traditional and simplified Chinese conversion, full-width character conversion, data encoding, and Chinese word segmentation and part-of-speech tagging to achieve the unification of Chinese and English symbols, thereby providing a more reliable data foundation for subsequent text mining, sentiment analysis, text classification and other tasks.

[0109] In some embodiments, the S2 includes:

[0110] S21 uses the CoSENT model, which is based on the BERT pre-trained language model and meets the needs of text semantic matching tasks. The goal of the CoSENT model is to supervise the training of a sentence vector representation model based on the BERT pre-trained language model architecture, so that the model can achieve the best performance in text semantic matching tasks.

[0111] In some embodiments, the S3 includes:

[0112] S31. During the entity recognition process, the BIO annotation method is used to set corresponding tags for each word in the sentence. B-[type] indicates that the word is the beginning of a named entity of type [type]; I-[type] indicates that the word is a non-beginning word of a named entity of type [type], and O indicates that the word is an ordinary word.

[0113] In some embodiments, S6 includes:

[0114] S61. The data input of the algorithm is a well-constructed event knowledge graph and the corresponding formalized question. The data output returns the corresponding answer entity.

[0115] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0116] In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0117] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.< / e>

Claims

1. A method for recommending urban operation work orders, characterized in that, The following steps are involved: S1, data preprocessing: Pre-process the input work order text to remove invalid information; S2, calculate the similarity using the CoSENT model: The model uses labeled data in the form of sentence pairs. After running the two sentences through BERT, the vectors u and v are obtained respectively. The loss function based on cosine similarity (prediction stage) is used to improve the similarity of positive samples. S3, work order entity identification: Use the bidirectional long short-term memory network-conditional random field model for named entity recognition to meet the entity extraction requirements in knowledge graph data import; S4, relation extraction: Perform relationship extraction; S5, build domain-level knowledge graph: After completing entity recognition and relationship classification, the data is imported into the neo4j graph database to generate a visual knowledge graph; S6, build a question-answering model based on knowledge graph semantics: The algorithm implements knowledge graph question-answering queries through the entity linking system and Cypher statements, meeting the requirements of knowledge graph statistical question-answering functions and returning highly similar semantic data.

2. The application of a method for recommending urban operation work orders and a knowledge graph according to claim 1, characterized in that: Said S1 includes: S11: Eliminate invalid structured information such as "Current demand:" and "Original task number:", perform traditional and simplified Chinese conversion, full-width character conversion, data encoding, and Chinese word segmentation and part-of-speech tagging to achieve the unification of Chinese and English symbols.

3. The application of a method for recommending urban operation work orders and a knowledge graph according to claim 2, characterized in that: Said S2 includes: S21 uses the CoSENT model, which is based on the BERT pre-trained language model and meets the needs of text semantic matching tasks. The goal of the CoSENT model is to supervise the training of a sentence vector representation model based on the BERT pre-trained language model architecture, so that the model can achieve the best performance in text semantic matching tasks.

4. The application of a method for recommending urban operation work orders and a knowledge graph according to claim 3, characterized in that: Said S3 includes: S31, during the entity recognition process, the BIO annotation method is used to set a corresponding label for each word in the sentence. B-[type] indicates that the word is the beginning of a named entity of type type; I-[type] indicates that the word is a non-starting word of a named entity of type type; O indicates that the word is an ordinary word.

5. The application of a method for recommending urban operation work orders and a knowledge graph according to claim 4, characterized in that: The S6 includes: S61, the data input of the algorithm is a well-constructed event knowledge graph and the corresponding formalized question. The data output returns the corresponding answer entity.

6. The application of a method for recommending urban operation work orders and a knowledge graph according to claim 1, characterized in that: Piece-Wise-CNN (PCNN) is used for relationship extraction. PCNN is an improvement on CNN. It divides sentences into three segments and performs independent maximum pooling when entities are known to achieve feature extraction.

7. Application of the work order recommendation method described in claim 1 in knowledge graph.

Images