A knowledge enhancement-based multi-granularity government affair service item recommendation method

Abstract

The present application relates to a kind of knowledge enhancement-based multi-granularity government affair service matter recommendation method, the method includes obtaining data, constructs and trains KEMG model, the KEMG model includes embedding layer of the fusion word granularity information, based on CNN enhancement's encoding layer, based on the interaction layer of attention mechanism, fusion layer and the similarity prediction score of multilayer fully connected layer are obtained.According to the data obtained, KEMG model is trained, and the optimal KEMG model is obtained.For new user, S1 is used to obtain user vector input optimal KEMG model, the optimal KEMG model calculates the correlation degree of the new user with all service matters, and is arranged in descending order according to the size of correlation degree value, and the service matter sequence TOP-k corresponding to correlation degree value is output.Proved by test, the method can guarantee the retrieval efficiency while having higher information retrieval accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of government service recommendation, and in particular to a multi-granularity government service matter recommendation method based on knowledge enhancement. BACKGROUND

[0002] Government service is a very important public service provided by the government for the people, which is directly related to the production and life of the people and is an important duty of the people's government. With the continuous advancement of informatization, "Internet + government service" has become an important direction of government work. In recent years, the types and scope of service information covered by the government platform have become more and more extensive, and the data sources have shown heterogeneous characteristics. At the same time, the platform access volume has increased explosively year by year. Taking Chongqing as an example, according to the Government Website Work Annual Report of Chongqing Statistical Bureau in 2022, by the end of 2022, the total number of independent user visits to the municipal government department website reached 683934, and the total number of website visits reached 2094310, an increase of 59.8% and 320.7% respectively compared with 2021. In the face of huge government information and massive access requests, the traditional data management mode cannot effectively cope with it, resulting in low service quality of the government platform.

[0003] On the other hand, service matter information retrieval is one of the core needs of users, and through search and question and answer functions, the required information can be quickly obtained. In the retrieval process, the text similarity matching between the input query and the matter name to be retrieved is the core. Related research shows that most of the current government platforms still use traditional word frequency statistical algorithms to realize matching. These methods, although fast in calculation and small in memory occupation, have very limited text semantic feature mining. In the government service matter retrieval scenario, users have different cultural levels, and the query contains colloquial content, which has a large difference with the matter name to be retrieved and cannot accurately describe the retrieval demand. For example, the user inputs "driving license lost to reissue", which should return the matter name "motor vehicle certificate lost to reissue registration", but due to the lack of the same keywords, the system returns other matters containing the "reissue" keyword.

[0004] Therefore, to improve the experience of the people in querying and handling government services, a precise government service matter retrieval model needs to be constructed based on deep learning and knowledge graph technology.

[0005] With the rapid development of artificial intelligence, deep learning technology has been widely used in many fields, especially in image and natural language processing. As a subtask of retrieval tasks in natural language processing, many deep learning-based text matching models have been proposed, and the effect has been significantly improved. However, deep learning models still have limitations when processing text, such as being unable to truly grasp the deep semantic of text and lacking logic in generating text. In 2012, Google proposed knowledge graph, and knowledge graph technology can convert text data into structured graph data to better combine deep learning models and play their advantages. For the application scenario of government service, knowledge graph technology can assist in understanding and utilizing a large amount of scattered service matter data, and through the conversion of data into structured knowledge graph, deep analysis and mining are realized, and more intelligent and accurate government services are provided for users, and data support is provided for government decision-making. Using deep learning, the rich entity and relationship information in the knowledge graph can be used to assist the training of the deep learning model to improve the semantic understanding ability of the model.

[0006] However, there is less research on related technologies in the field of government services. On the one hand, the data in the field of government services usually exists in a structured and semi-structured form, which is different from the structured data required by the knowledge graph. At the same time, the government service resources are large and scattered, with many types and complex levels, which brings great difficulty to the construction and maintenance of the knowledge graph. On the other hand, the deep text matching model has a complex structure, and the operation and maintenance and online retrieval efficiency after actual deployment become problems that need to be solved in applying deep learning technology to the field of government services.

[0007] Therefore, based on the graph technology and deep learning technology, the government service matter retrieval method is researched to ensure the retrieval efficiency and improve the information retrieval accuracy, which is of great significance to improve the level of government services. SUMMARY

[0008] In view of the above problems existing in the prior art, the technical problem to be solved by the present application is how to ensure the retrieval efficiency while improving the information retrieval accuracy based on the graph technology and deep learning technology.

[0009] To solve the above technical problems, the present application adopts the following technical scheme: a multi-granularity government service matter recommendation method based on knowledge enhancement, comprising the following steps:

[0010] S1: obtaining a plurality of user input query texts and to-be-matched government matter names, and constructing a government knowledge graph.

[0011] S2: constructing and training a KEMG model, the KEMG model comprising an embedding layer fusing word granularity information, an encoding layer based on CNN enhancement, an interaction layer based on an attention mechanism, a fusion layer, and a multi-layer fully connected layer to obtain a similarity prediction score.

[0012] S2-1: Embedding the word vectors of all user input query text and the to-be-matched government affair name to obtain query word granularity vectors and government word granularity vectors, embedding the character vectors of all user input query text and the to-be-matched government affair name to obtain query character granularity vectors and government character granularity vectors, and splicing and fusing the query word granularity vectors and the query character granularity vectors to form a query final text sentence representation vector, and splicing and fusing the government word granularity vectors and the government character granularity vectors to form a government final text sentence representation vector;

[0013] S2-2: The query final text sentence representation vector and the government final text sentence representation vector obtained in S2-1 are both first encoded by using ESIM to encode the context sequence information, and the encoded vectors are then secondly encoded by using a CNN network to obtain query keyword feature encoding values and government keyword feature encoding values.

[0014] S2-3: An improved graph attention mechanism is used to assign different weights to each node in the government knowledge graph, and the weights are used to enhance the government word granularity vectors. The query keyword feature encoding values and the government keyword feature encoding values are interacted by using a soft attention mechanism to obtain enhanced representations of each word after weighting and input a feature vector obtained by a pooling layer for splicing to obtain a user query final representation vector and an affair name final representation vector, respectively.

[0015] S2-4: A fusion layer realizes multi-feature cross fusion by vector pair subtraction and vector pair multiplication to obtain a matching degree between the query and the affair name.

[0016] S2-5: The matching degree between the query and the affair name is directly input into a multi-layer fully connected neural network to predict a matching score, and the government affair names corresponding to the predicted matching scores are input in descending order according to the predicted matching scores.

[0017] S3: For a new user, a user vector obtained by using S1 is input into an optimal KEMG model, the optimal KEMG model calculates the correlation degree of the new user and all service affairs, and outputs a service affair sequence TOP-k corresponding to the correlation degree value in descending order of the correlation degree value.

[0018] As an improvement, the process of obtaining the word granularity vectors in S2-1 is as follows:

[0019] The purpose of the embedding layer is to map the text after tokenization from a high-dimensional space to a low-dimensional space, so as to perform feature augmentation on the text and participate in vector operations of the neural network. Word embedding, as a commonly used method of the embedding layer, can represent the sentence vector to a great extent through the combination of word vectors in short text tasks.

[0020] The user input query text and the to-be-matched government affair matter text are embedded by using Word2vec to obtain word vectors, as shown in equations (1) and (2).

[0021]

[0022]

[0023] wherein Q w and S w respectively represent the vector matrix composed of word vectors after the user input query (Query) and the to-be-matched government service matter name (Service Name) are processed by Word2vec, represents the w-dimensional vector obtained after each segmented word in the two texts is embedded, d w represents the dimension of the word vector. l a and l b respectively represent the length of the user input query and the to-be-matched government service matter name.

[0024] As an improvement, the process of obtaining the character granularity vector by S2-1 is as follows:

[0025] The joint learning word embedding model JWE is used to realize the character embedding of the text, as shown in equation (3).

[0026]

[0027] wherein are respectively the context words, the context Chinese characters, and the context sub-characters, w i is the target word, and L(w i ) represents the sum of the logarithmic likelihoods of the three prediction conditional probabilities of the target word w i .

[0028] The prediction probability is defined by the Softmax function, as shown in equation (4).

[0029]

[0030] wherein is the output vector of the target word w i , represents the current target word, represents any one of the target words in the sum of all target words from 1 to N.

[0031] The user input query and each character in the to-be-matched government service matter name are represented by a character vector by JWE. The character vector representations of the user input query and the to-be-matched government service matter name are denoted as Q j and S j , as shown in equations (5) and (6).

[0032]

[0033]

[0034] wherein, represent the character vector representation of the i-th, j-th Chinese character in the user input query and the to-be-matched government service matter name, respectively, d c represents the dimension of the character vector. z q , z s represent the number of characters in the user input query and the to-be-matched government service matter name, respectively.

[0035] As an improvement, the process of splicing and fusing the word granularity vector and the character granularity vector in S2-1 to form the final text pair embedding representation is as follows:

[0036] The new combination vector of the i-th word in the user input query is denoted as Q i The calculation formula is shown in equation (7).

[0037]

[0038] wherein represents the feature vector obtained by performing a pooling operation on all character granularity vectors constituting the i-th word in the text;

[0039] The final text sentence representation vector Q of the query is calculated according to equation (8).

[0040] Q = (Q1,..., Q l ) (8)

[0041] wherein, The embedding dimension of the fused character granularity word vector of the defined model is defined as d = d w +d c .

[0042] The government final text sentence representation vector S can be obtained by using the same method as obtaining Q.

[0043] As an improvement, the steps of obtaining the keyword feature encoding in S2-2 are as follows:

[0044] The ESIM is used to encode the context sequence information of the final text sentence representation vector, and the BiLSTM is used in the ESIM to obtain the text sequence features. The calculation process of the encoded representation vector is shown in formulas (9) and (10).

[0045]

[0046]

[0047] wherein, respectively represent the new encoded values of the user input query text and the to-be-matched government affair name obtained after BiGRU at the i-th time step and the j-th time step, d g represents the hidden layer dimension set in the BiGRU.

[0048] The CNN network is used for the second encoding. The CNN network uses convolution kernels with sizes of 2, 3, and 4, respectively, for convolution operation. The convolution window extracts text features as follows. First, formula (11) is used:

[0049]

[0050] wherein, represents the concatenation matrix of the word vectors . represents the concatenation operation. In the convolution operation, the convolution kernel slides on the text in the form of a window with a size of w to extract new features, as shown in formula (12).

[0051]

[0052] wherein C i represents the i-th feature extracted by the convolution kernel in the convolution process.

[0053] The input vector that is out of the range (i < 1 or i > l) is regarded as a zero vector, represents the concatenation matrix of the word vectors q i , q i+1 , q i+2 , …, q w , and b represents the bias in the convolution network.

[0054] The convolution kernel with a size of w is applied to each possible hidden state in the sentence to extract the feature map as shown in formula (13).

[0055] q = [q1, q2, …, q l-w+1 ] T (13)

[0056] where The obtained six feature maps are input into the max-pooling layer to obtain a fixed-length feature vector representation:

[0057] Q i = P({q i1 , q i2 ,..., q i(l-w+1)}) (14)

[0058] where, Q i represents the feature vector of the i-th feature map after the max-pooling operation, q i1 represents the first feature of the i-th feature map. q i(l-w+1) represents the (l-w+1)th feature of the i-th feature map.

[0059] Finally, the pooled feature vectors are concatenated to form the final query keyword feature encoding by formula (15)

[0060]

[0061] The same method as obtaining the government keyword feature encoding can be used to obtain the government keyword feature encoding

[0062] As an improvement, the process of obtaining the final user query representation vector and the final transaction name representation vector in S2-3 is as follows:

[0063] The SIF sentence vector generation method includes:

[0064] 1) Obtain a preliminary sentence vector: traverse all sentences in the corpus, and calculate the preliminary sentence vector v s of the current sentence s by formula (16).

[0065]

[0066] where |s| represents the number of words in the sentence s, v w represents the word vector of the word w in the sentence, p(w) is the word frequency probability of the word w in the entire corpus, and a is an adjustable parameter.

[0067] 2) Principal component analysis of all preliminary sentence vectors: calculate the first principal component u of all preliminary sentence vectors, called the public discourse vector, which is the first principal vector of the matrix composed of the word vectors of the sentence.

[0068] 3) Obtain the target sentence vector: perform secondary processing on the preliminary sentence vector by formula (17) to obtain the target sentence vector.

[0069] vs = v s - uu T v s (17)

[0070]

[0071] wherein denotes the sentence vector value of the SIF-embedded query text, w denotes the sentence and each word, respectively.

[0072] The sentence vector value of the SIF-embedded query text is obtained by using formula (18) The sentence vector value of the SIF-embedded government affair matter name to be matched

[0073] The improved graph attention calculation formula is shown in formula (19).

[0074]

[0075] wherein, denotes the sentence vector representation of the government affair matter name enhanced by fusing the government affair knowledge graph, denotes all government affair knowledge nodes directly associated with the government affair matter, including the government affair matter itself, denotes the sentence vector value of the relevant government affair knowledge after SIF embedding.

[0076] The soft attention mechanism interacts the query keyword feature encoding value and the government keyword feature encoding value:

[0077] First, the correlation score is calculated, and the score value is calculated as shown in formula (20).

[0078]

[0079] In the formula, e ij denotes the in the final text sentence representation vector of the query, and the correlation score obtained by dot product.

[0080] The correlation score is normalized to convert the correlation score into a weight coefficient for subsequent weighted representation, and the process is shown in formula (21).

[0081]

[0082] wherein, is the weighted sum result of the i-th word in the final text sentence representation of the query and each word in the final text sentence representation of the government affairs after attention coefficient distribution, e ik is the word vector of any word in the final text representation of the government affairs.

[0083] For the j-th word in the matter name, The calculation process and meaning are the same, as shown in equation (22).

[0084]

[0085] is the weighted sum result of the j-th word in the final text sentence representation of the government affairs and each word in the final text sentence representation of the query after attention coefficient distribution, e kj is the word vector of any word in the query text representation.

[0086] The dimension reduction is performed on and using the pooling strategy, and the average pooling combined with the maximum pooling method is used, and the calculation process is shown in equations (23) and (24).

[0087]

[0088]

[0089] v q,ave , v q,max , v s,ave , v s,max respectively represent the four vectors obtained by average pooling and maximum pooling on the user input query text and the matter name to be searched.

[0090] Then, the representation vector obtained by soft attention interaction operation on the user input query text and the government affairs matter name to be matched is obtained by splicing:

[0091]

[0092]

[0093] wherein, respectively represent the user query final representation vector corresponding to the user input query text and the matter name final representation vector corresponding to the government affairs matter name to be matched after the soft attention interaction operation. d g is the hidden layer dimension set in GRU.

[0094] As an improvement, the process of S2-4 to get the matching degree between query-transaction name is as follows:

[0095] Multi-feature cross fusion is realized by vector pair subtraction and vector pair multiplication. The user input query text and the to-be-matched government transaction name are spliced with their respective three features to store in a sequence:

[0096]

[0097]

[0098] The two sequences are respectively subjected to pair subtraction and pair multiplication, and then the results of the two are spliced with the original sequence value:

[0099] m out =[m Q ;m S ;m Q -m S ;m Q ⊙m S ] (29)

[0100] Through the fusion layer, the matching degree between query-transaction name is represented by vector .

[0101] As an improvement, the process of S2-5 to calculate the matching score is as follows: as shown in formula (30):

[0102] m score =F s (W·m out +b) (30)

[0103] Wherein, W, b represents the parameters of multi-layer full connection layer, F s (·) is Sigmod activation function.

[0104] As an improvement, Margin Ranking Loss introduces the minimum relevance gap between two transactions for each sample pair when calculating the loss. For each sample pair (x i ,x j ), the loss function is as follows:

[0105] l(x k ,x j )=max(0,m-(s(x k )-s(x j ))) (31)

[0106] Wherein, m represents the Margin parameter, s(x i ) and s(x j ) are sample xk and x j the correlation score of x

[0107] The ServiceRank loss function is proposed:

[0108]

[0109] wherein, represents a data set composed of user input text and to-be-retrieved matter name text, represents the number of samples in the data set, s i represents the number of government affair matter names corresponding to the i-th user input query text, represents the set of all government affair matter names corresponding to the i-th user input query text, l(x j ,x k is the MarginRanking Loss calculated by (31).

[0110] Compared with the prior art, the present application has at least the following advantages:

[0111] 1. Comparative experiment results show that in the government service matter retrieval task, the retrieval effect of the method of the present application is better than that of other non-BERT models, and the parameter quantity is greatly less than that of the BERT model, ensuring the feasibility of online deployment. The loss function strategy comparison experiment shows the effectiveness of ServiceRank on the data set constructed in the present application.

[0112] 2. Ablation experiments are performed to verify the influence of the introduced word embedding, CNN network and government knowledge on the retrieval ability of the model. The influence of the value of the parameter a in the SIF sentence vector generation, the value of m in ServiceRank, the number of neurons in the full connection layer and the Dropout rate of the last layer on the overall retrieval effect of the model is analyzed by experiment, and the performance indicators of the model in the optimal state are determined. BRIEF DESCRIPTION OF DRAWINGS

[0113] Figure 1 is the architecture diagram of the method of the present application.

[0114] Figure 2 is a triple instance diagram.

[0115] Figure 3 is a knowledge graph hierarchical diagram.

[0116] Figure 4 is a principle diagram of government knowledge enhanced query-matter pair matching capability.

[0117] Figure 5 is a JWE structure diagram.

[0118] Figure 6 Pooling fused word vectors.

[0119] Figure 7 Word embedding flowchart of fused word granularity.

[0120] Figure 8 Convolutional text feature extraction flowchart.

[0121] Figure 9 Government affair knowledge fusion process diagram.

[0122] Figure 10 Soft attention operation schematic diagram.

[0123] Figure 11 Labeling details of the same problem situation.

[0124] Figure 12 Transaction subdirectory.

[0125] Figure 13 Search effect result comparison, wherein Figure 13 (a) represents the result comparison of different models under ServiceRank, Figure 13 (b) represents the result comparison of the model in this paper under six kinds of loss functions, Figure 13 (c) and (d) both represent the training / test time comparison of each model.

[0126] Figure 14 Model training convergence process. Figure 14 (a) respectively represents the training set and validation set loss function change diagram in the model training iteration, Figure 14 (b) represents the change of the three indexes in the training iteration process.

[0127] Figure 15 Modular ablation experiment results, Figure 15 (a) and 15(b) respectively represent the model single module ablation comparison and the model two module combination comparison.

[0128] Figure 16 The influence of each parameter on the NDCG of the model, Figure 16 (a)-16(d) respectively represent the influence of the parameter a value, the hyperparameter m value, the neuron number and the Dropout rate on the NDCG value of the model. DETAILED DESCRIPTION

[0129] The present application will be further described in detail below.

[0130] To address the shortcomings of ESIM, the text proposes a knowledge enhanced multi-granularity government service item retrieval model (KEMG), as shown in the overall structure of Figure 1 .

[0131] In the text embedding layer, the model obtains the text word vector matrix and the text character vector matrix through Word2vec and JWE respectively, and obtains the text embedding vector after splicing. At the same time, the model uses the SIF sentence vector generation method to obtain the query and the name of the item and the sentence vector of the government knowledge directly connected with the item to prepare for the subsequent interactive layer to fuse the government knowledge.

[0132] In the encoding layer, the model first sends the text embedding vector into BiGRU. BiGRU is used to extract the context information before and after the text. In order to capture the key word and other feature information of the text, the application proposes to add CNN structure, which uses the characteristics of CNN convolution kernel to capture the feature information of word granularity. Finally, the sequence encoding value output by BiGRU and the feature encoding value obtained by CNN secondary encoding are obtained respectively.

[0133] In the interaction stage, the retrieval government service item and the related government knowledge are retrieved and the corresponding sentence vector is read. The application uses an improved graph attention mechanism to assign different weights to each node in the government knowledge, and uses them to enhance the representation of the item name sentence vector. The soft attention mechanism is used to interact with the two text sequence values after sequence encoding, to obtain the enhanced representation of each word after weighting and input into the pooling layer. The model splices the secondary encoding features obtained by the convolutional neural network, the enhanced text sentence vector of the government knowledge, and the feature vector obtained by pooling after soft attention interaction, to obtain the final representation vector of the user query and the item name respectively. Through the interaction strategy of the model, the multi-dimensional features extracted by the text through multiple modules can be effectively extracted and retained.

[0134] Finally, the fusion layer includes operations such as bitwise subtraction and bitwise multiplication on vectors, which are used to fuse the respective representation vectors of the two texts. Finally, the similarity prediction score is obtained through multiple fully connected layers.

[0135] The process of the government knowledge graph is as follows

[0136] The knowledge graph can be logically divided into a schema layer and a data layer. The schema layer is the core of the knowledge graph, and stores refined knowledge, defines and standardizes the hierarchy and categories within the domain. The schema layer of the graph is usually managed using an ontology library, which uses the rules, axioms and constraint conditions in the ontology library to standardize the association between entities, relationships and the types and attributes of entities in the graph.

[0137] The data layer is responsible for the specific storage of specific triples in the knowledge graph, and is structurally below the schema layer, and is the actual form of the entire knowledge graph. In the data layer, triples are stored in a graph database in two forms: <entity1, relation-entity2> and <entity, attribute, value>, as shown in Figure 2 .

[0138] A large amount of heterogeneous knowledge will form a huge entity relationship network through the structural definition of the schema layer and the processing of the data layer, forming a knowledge graph. The structure of the schema layer and the data layer is shown in Figure 3 .

[0139] The construction of the knowledge graph starts from obtaining the original knowledge data, adopts a series of knowledge processing techniques (automatic or semi-automatic) to extract the required knowledge elements from the original data, and then stores them according to the definition and constraints of the schema layer and the data layer. The construction method of the knowledge graph is divided into bottom-up and top-down.

[0140] The bottom-up construction method is to define the most basic concepts, attributes and relationships based on business on the basis of a large amount of text and data tables, then extract triples and add them to the data layer, cluster the concepts with high overlapping attributes / relationships based on data, and abstract the schema layer of the knowledge graph, and then guide the subsequent knowledge processing.

[0141] The top-down construction method is to define the schema layer according to the domain concepts, and then add triples to the data layer based on the constraints of the schema layer. Specifically, it includes four steps: 1) defining the schema layer of the knowledge graph; 2) extracting triples from multiple heterogeneous data sources based on the constraints of the schema layer; 3) eliminating redundant information in the triple data through knowledge fusion related technologies; 4) storing the cleaned triples in the knowledge graph. This method is suitable for domain knowledge graphs. Domain knowledge graphs have higher requirements for entity information, and domain concepts are more concentrated, so the schema layer of the graph can be constructed comprehensively and efficiently.

[0142] The government service matter knowledge graph belongs to the category of domain knowledge graph, and the present application adopts a top-down method for construction.

[0143] A multi-granularity government service matter recommendation method based on knowledge enhancement, comprising the following steps:

[0144] S1: Obtain a plurality of user input query texts and to-be-matched government matter names, and construct a government knowledge graph;

[0145] S2: Construct and train a KEMG model, the KEMG model comprising an embedding layer fusing word granularity information, a coding layer based on CNN enhancement, an interaction layer based on an attention mechanism, a fusion layer, and a plurality of fully connected layers to obtain a similarity prediction score.

[0146] S2-1: The embedding of the word vectors of all user input query text and the to-be-matched government affair name corresponds to obtain query word granularity vectors and government word granularity vectors, the embedding of the character vectors of all user input query text and the to-be-matched government affair name corresponds to obtain query character granularity vectors and government character granularity vectors, the query word granularity vectors and the query character granularity vectors are spliced and fused to form query final text sentence representation vectors, and the government word granularity vectors and the government character granularity vectors are spliced and fused to form government final text sentence representation vectors;

[0147] S2-2: After the user input query and the to-be-matched government affair name pass through the embedding layer of the model, they are mapped into a vector space. At this time, the sentence features are only composed of word vectors fused with character granularity, and lack of context and keyword features, so further encoding is needed to extract sequence features and text keyword information.

[0148] The application improves the feature extraction method of context information on the basis of ESIM and introduces a CNN network for secondary coding to capture finer-grained text features. Due to the characteristics of the CNN network, the model effect is improved while the overall parameter quantity is greatly reduced.

[0149] The query final text sentence representation vectors and the government final text sentence representation vectors obtained in S2-1 are both first encoded by ESIM to obtain context sequence information, and then the encoded vectors are secondarily encoded by a CNN network to obtain query keyword feature encoding values and government keyword feature encoding values.

[0150] The convolutional neural network (CNN) performs secondary coding on the basis of BiGRU coding, combines the advantages of both, and the model can capture and retain finer-grained text features. Meanwhile, due to the parameter sharing characteristics of the CNN network, the overall parameter quantity of the model is further reduced.

[0151] S2-3: An improved graph attention mechanism is used to assign different weights to each node in the government knowledge graph, and the weights are used to enhance the government word granularity vectors. A soft attention mechanism is used to interact the query keyword feature encoding values and the government keyword feature encoding values to obtain enhanced representations of each word after weighting and input the feature vectors obtained by the pooling layer for splicing to obtain user query final representation vectors and affair name final representation vectors. The user only inputs query text, and the to-be-retrieved affair text and related knowledge text vectors are stored in the server side.

[0152] The text features have certain semantic information after model embedding and coding, but still lack of interaction operation between texts, and lack of comparison of lexical and syntactic information between texts, which means that a large amount of semantic information is lost. On the other hand, the current obtained semantic information only focuses on the query and the name of the matter to be searched, and it is difficult for the model itself to realize deeper semantic mining in the government field.

[0153] Fusing the government knowledge graph can help the model better understand the deep connection between the user input query and the government service matter, such as the user input "What materials are needed for reimbursement of surgical expenses?", the corresponding matter is "Basic medical insurance expense settlement", the model retrieves the associated knowledge in the government knowledge graph centering on the service matter entity, and uses them to enhance the model's text understanding ability of the government service matter name, such as shown in the following formula. Figure 4

[0154] As can be seen from the figure, the knowledge connected by the "acceptance condition" relationship in the government knowledge graph centering on the matter contains keywords such as "reimbursement" and "materials", which are highly similar to part of the words in the user input sentence. Fusing them into the text of the matter to be matched can enhance the representation of the matter name and improve the overall matching ability of the model.

[0155] The existing knowledge fusion method treats each entity node in the knowledge graph as equal, without reflecting the importance distinction. The government matters as the center node are not all helpful to semantic understanding, on the contrary, too much introduction of government information will bring noise and affect the model performance. In the above example, the clause text content connected by the "acceptance condition" relationship contains the most relevant information, while other knowledge such as "handling place" and "examiner" should weaken their influence to avoid negative effects on the model.

[0156] To solve the above problems, the present application proposes to better fuse government knowledge based on graph attention mechanism. The graph attention mechanism is different from the knowledge embedding and knowledge representation method, it combines attention mechanism with graph structure, can distribute weights to other nodes directly connected with the node, and by assigning more weights to nodes with higher relevance to strengthen the representation of the center node.

[0157] S2-4: The fusion layer realizes multi-feature cross fusion by vector pair subtraction and vector pair multiplication, and obtains the matching degree between the query and the matter name;

[0158] ​S2-5: The degree of matching between the obtained query-service name is directly input into a multi-layer fully connected neural network to predict a matching score, and the predicted matching score is ranked in descending order according to the predicted matching score, and the government affair item name corresponding to the predicted matching score is input.

[0159] S3: For a new user, the user vector obtained by S1 is input into the optimal KEMG model, the optimal KEMG model calculates the correlation degree of the new user and all service items, and outputs the service item sequence TOP-k corresponding to the correlation degree value in descending order of the correlation degree value.

[0160] Specifically, the process of obtaining the word granularity vector by S2-1 is as follows:

[0161] The purpose of the embedding layer is to map the text after tokenization from a high-dimensional space to a low-dimensional space, so as to perform feature augmentation on the text and participate in vector operation of the neural network. Word embedding as a commonly used method of embedding layer, through the combination of word vectors, it can represent the sentence vector to a great extent in short text tasks.

[0162] The word vectors of the user input query text and the to-be-matched government affair item text are embedded by using Word2vec, as shown in equations (1) and (2).

[0163]

[0164]

[0165] Wherein, Q w and S w represent the vector matrix composed of word vectors after the user input query (Query) and the to-be-matched government service item name (Service Name) are processed by Word2vec, represents the w-dimensional vector obtained after each token in the two texts is embedded, d w represents the dimension of the word vector. l a and l b represent the length of the user input query and the to-be-matched government service item name, respectively. In order to simplify the calculation, the length of the two texts is uniformly defined as length l.

[0166] Specifically, the process of obtaining the word granularity vector by S2-1 is as follows:

[0167] In natural language processing tasks, word-level features usually have better representation capabilities than character-level features. However, there are problems with simply using word embeddings as the representation vector matrix for sentences in the tasks of this invention. For Chinese texts, word segmentation needs to be performed first before further operations can be carried out. Due to the particularity of Chinese, the word segmentation results of different word segmentation tools (such as jieba, Paoding JieNiu word segmentation package, etc.) may be inconsistent and deviate from the original meaning of the text, affecting the model's performance. Secondly, due to the real-time changes in policies, new government service words may emerge, resulting in out-of-vocabulary words and deteriorating the model's performance. Finally, from the perspective of semantic information, Chinese characters contain certain semantic information, and only using word embeddings will lose the semantic information at the character level.

[0168] The concept of sub-characters is defined in JWE. Sub-characters refer to the components within a single Chinese character. For example, in the character "照", in addition to having the radical "灬", it also has three sub-components: "日", "刀", and "口". JWE regards these components as the sub-characters of "照". Based on the CBOW model, JWE uses the average of context word vectors, the average of context character vectors, and the average of context sub-character vectors to predict the target word and jointly learn the embeddings of the three. The model structure is as Figure 5 shown.

[0169] Among them, w i is the target word, w i-1 and w i+1 are the words on the left and right sides of the target word in the text, c i-1 and c i+1 represent the Chinese characters in the context, s i-1 and s i+1 represent the sub-characters in the context, and s i represents the sub-characters of the target word w i . <0>

[0170] The joint learning word embedding model JWE is adopted to achieve character embedding of the text. JWE aims to maximize the sum of the logarithmic likelihoods of the three prediction conditional probabilities of the target word w i , as shown in formula (3).

[0171] .

[0172] Among them refer to context words, context Chinese characters, and context sub-characters respectively, w i is the target word, and L(w i ) represents the sum of the logarithmic likelihoods of the three prediction conditional probabilities of the target word w . i . [[HIDDEN]]

[0173] The prediction probability is defined by the Softmax function, as shown in formula (4).

[0174]

[0175] where is the target word w i The output vector here is defined by the softmax formula, represents the current target word, represents any one of the target words in the sum of all target words from 1 to N.

[0176] For each character in the query input by the user and the name of the government service item to be matched, a character vector representation is obtained through JWE. With the help of the JWE model, the present invention realizes the character-level embedding of the query and the name of the government service item to be retrieved. The character vector representations corresponding to the query input by the user and the name of the government service item to be matched are respectively denoted as Q j and S j , as shown in formulas (5) and (6).

[0177]

[0178]

[0179] where respectively represent the character vector representations of the i-th and j-th Chinese characters in the query input by the user and the name of the government service item to be matched, and d c represents the dimension of the character vector. z q , z s respectively represent the number of characters in the two texts of the query input by the user and the name of the government service item to be matched.<9000635>

[0180] Specifically, the process of concatenating and fusing the word-level vector and the character-level vector to form the final text pair embedding representation in S2-1 is as follows:

[0181] After obtaining the word-level vector, it is necessary to concatenate and fuse the word-level representation and the character-level representation to form the final text pair embedding representation.

[0182] A word in a Chinese text can be regarded as composed of multiple characters. Therefore, directly concatenating the vectors of the two granularities will cause the fused vector to not reflect the inclusion relationship between words and characters, losing semantic information.

[0183] A pooling strategy is adopted to fuse the word and character granularity feature vectors. Specifically, the maximum pooling or average pooling is used to process the character vectors and then concatenate them. The following takes the word "handle" in the user query q as an example to show the vector fusion process using the pooling strategy, as Figure 6 shown.

[0184] The word "handling" is represented by a word vector through Word2vec. "Handling" is split into two characters "B" and "Li", and their respective word vectors are obtained through the JWE model, and then the word vectors are spliced and subjected to maximum pooling or average pooling operation to obtain a new combined vector, as shown in .

[0185] wherein c j represents the word granularity vector of the jth character of the word, and w represents the number of characters of the word. represents the feature vector of all word granularity vectors constituting the ith word in the text after the pooling operation. P represents the average pooling operation or the maximum pooling operation. Finally, the word granularity vector and the word granularity combined vector of the word are spliced by column to obtain the final new combined vector of the word "handling".

[0186] The new combined vector Q i of the ith word in the query input by the user is calculated as shown in formula (7).

[0187]

[0188] wherein represents the feature vector of all word granularity vectors constituting the ith word in the text after the pooling operation;

[0189] The final text sentence representation vector Q of the query is calculated as shown in formula (8).

[0190] Q = (Q1,..., Q l ) (8)

[0191] wherein, the embedding dimension of the word vector of the fusion word granularity of the model is defined as d = d w +d c .

[0192] The final text sentence representation vector S of the government affairs can be obtained by the same method as obtaining Q.

[0193] At this point, the embedding layer of the model is constructed. The word vector matrix of the text after tokenization is obtained by using Word2vec, and the word vector of each character in the text is obtained based on the JWE word granularity embedding model. The final word embedding matrix is obtained by processing the word vector using the pooling strategy to fuse the word vector matrix. The representation vector matrix of the sentence is obtained by using the new word embedding of the text. Taking the user input query "handling the procedure for children studying abroad" as an example, the embedding vector conversion process is shown in Figure 7 .

[0194] Specifically, the step of obtaining the keyword feature code by S2-2 is as follows:

[0195] The final text sentence representation vector is encoded with context sequence information by ESIM, and BiLSTM is used in ESIM to obtain text sequence features. The calculation process of the encoded representation vector is shown in formulas (9) and (10).

[0196]

[0197]

[0198] wherein, respectively represent the new encoded values of the user input query text and the to-be-matched government affair name after BiGRU at the i-th time step and the j-th time step, d g represents the hidden layer dimension set in BiGRU.

[0199] The CNN network is used for the second encoding, and the CNN network uses convolution kernels with sizes of 2, 3, and 4, respectively, for convolution operation, and the detailed process is shown in formula (11). Figure 8

[0200] The left side of the figure is the vector matrix of the query "handle the procedure of overseas children going to school", and the model regards the matrix as the original pixel point in the image, extracts the multi-level feature information in the text through convolution, and then inputs the matrix into one-dimensional convolution layers with convolution kernel sizes of 2, 3, and 4, respectively, for convolution operation. Each size of the convolution kernel has two filters, that is, the number of channels is 2, and finally a total of 6 feature maps are output.

[0201] The text feature extraction process of the convolution window is as follows. First, formula (11) is used:

[0202]

[0203] wherein, represents the concatenation matrix of the word vector , and represents the concatenation operation. In the convolution operation, the convolution kernel slides on the text in the form of a window with a size of w to extract new features, as shown in formula (12).

[0204]

[0205] wherein C i represents the i-th feature extracted by the convolution kernel in the convolution process. At the same time, a special padding mark is set for the sentence text, because when the window slides to the vicinity of the boundary, the window may be outside the sentence boundary (l). After such processing, the input vectors (i < 1 or i > l) outside the range are regarded as zero vectors. ​Word vector q i ,q i+1 ,q i+2 ,…,q w The concatenated matrix, where b represents the bias in the convolutional network.

[0206] A convolution kernel of size w is applied to a sentence Each possible hidden state is represented by the extracted feature map as shown in Equation (13).

[0207] q = [q1, q2, ..., q l-w+1 ] T (13)

[0208] in The resulting 6 feature maps are input into the max pooling layer to obtain a fixed-length feature vector representation:

[0209] O i =P({q i1 ,q i2 ,...,q i(l-w+1)}) (14)

[0210] Among them, Q i Let q represent the feature vector of the i-th feature map after max pooling. i1 q represents the first feature of the i-th feature map. i(l-w+1) This represents the (l-w+1)th feature of the i-th feature map. A feature map has (l-w+1) features, where w represents a convolution kernel with a window size of w, and l is the total number of iterations after text segmentation.

[0211] Finally, the pooled feature vectors are concatenated using formula (15) to form the final query keyword feature code.

[0212]

[0213] Adoption and Acquisition The same method can be used to obtain the feature encoding of government keywords.

[0214] Two types of text encoding feature values ​​can be obtained through the enhanced encoding layer: one type is the sequence encoding value obtained by BiGRU. and Another type is the keyword feature encoding value obtained by using CNN for secondary encoding. and

[0215] Specifically, the process of obtaining the final representation vector of the user query and the final representation vector of the item name in S3-3 is as follows:

[0216] A SIF sentence vector (Smooth Inverse Frequency) and an improved graph attention method are proposed. In order to maximize the guarantee that the features are not lost, the application adopts a SIF sentence vector generation method.

[0217] The SIF sentence vector generation method comprises:

[0218] 1) Obtain a preliminary sentence vector: traverse all sentences in a corpus, and calculate a preliminary sentence vector v of a current sentence s through formula (16) s .

[0219]

[0220] Where |s| represents the number of words of the sentence s, v w represents the word vector of the word w in the sentence, p(w) is the word frequency probability of the word w in the whole corpus, and a is an adjustable parameter. The core idea of the formula maintains the idea of weighted average, and the weight calculation uses a new word frequency probability plus an adjustable parameter.

[0221] 2) Principal component analysis is performed on the whole preliminary sentence vector: the first principal component u of the whole preliminary sentence vector is calculated, which is called a common discourse vector, representing the "meaning" shared by all words in the sentence, and can be understood as the core "component" in the sentence, which represents the meaning of the sentence to some extent. u is the first principal vector of the matrix composed of the word vectors of the sentence (the first principal component in PCA).

[0222] 3) Obtain a target sentence vector: the preliminary sentence vector is processed twice through formula (17) to obtain a target sentence vector. It can be understood as deleting the "common part" u of the word vector and retaining the features possessed by each word vector.

[0223] v s =v s -uu T v s (17)

[0224]

[0225] Where represents the SIF embedded sentence vector value, w respectively represent the sentence and each word.

[0226] The SIF embedded sentence vector value of the user input query text is obtained by formula (18) The SIF embedded sentence vector value of the government affair matter name to be matched Here represents and are calculated by (18).

[0227] On the other hand, in the process of fusing government affair knowledge, the role of knowledge is to make the matter name closer to the query in similarity. However, according to the original formula, the model will calculate the correlation between the matter name text itself and its associated knowledge to give the knowledge node weight value and then represent itself, which is contradictory. Therefore, in view of the unreasonable problem of the graph attention center node selection, the application proposes to improve the graph attention calculation formula, and use the query instead of the matter name to calculate the attention correlation, which more accurately captures the influence of government affair knowledge on the matter.

[0228] The improved graph attention calculation formula is shown in (19).

[0229]

[0230] wherein, represents the enhanced government affair matter name sentence vector representation of the government affair knowledge graph, represents all government affair knowledge nodes directly associated with the government affair matter, including the government affair matter itself, represents the sentence vector value obtained after the relevant government affair knowledge is embedded by SIF. Taking "What materials are needed for reimbursing surgical expenses" as an example, Figure 9 shows the process of fusing service matter knowledge and matter name by the model.

[0231] Based on the graph attention, the government affair knowledge is fused to enhance the matter name representation, and the soft attention is used for interaction by referring to ESIM, which further enhances the matter retrieval capability of the model;

[0232] The soft attention mechanism interacts the query keyword feature encoding value and the government affair keyword feature encoding value:

[0233] The soft attention is used to realize the interaction operation between the query and the matter name. The soft attention emphasizes that although the weights of different words are different, each word will be considered, and the words with strong relevance will be assigned with larger weights. The weighted average value of all input information is calculated during output. In the text matching task, a word usually has association with multiple words in the relative sentence, and different associations will affect the word itself to different degrees, which is very consistent with the implementation method of the soft attention.

[0234] In the soft attention interaction step, the model performs attention operation based on the query keyword feature encoding value and the government affair keyword feature encoding value obtained in the encoding layer. First, the correlation score is calculated, and the calculation method of the score value is shown in formula (20).

[0235]

[0236] wherein eij represents the correlation score in the query final text sentence representation vector represents the correlation score in the query final text sentence representation vector The correlation score is obtained by dot product. Specifically, the greater the correlation between the two words, the greater the value obtained by formula (20), Figure 10 An example of soft attention operation based on formula (20) is shown.

[0237] The figure shows the interaction operation details between the query "How to reissue the lost cost consultant certificate" and the matter name "Loss of secondary cost consultant registration certificate and reissue". The color depth in each cell intuitively reflects the correlation between the two words, and the number represents the correlation score. As can be seen from the example in the figure, the correlation scores of "lost"-"lost" and "do"-"reissue" are high, indicating that they have a high degree of similarity. In contrast, the correlation scores of "lost"-"registration" and "how"-"certificate" are low, indicating that their similarity is also low.

[0238] The correlation score is normalized to convert the correlation score into a weight coefficient for subsequent weighted representation, as shown in formula (21).

[0239]

[0240] wherein, represents the new representation of the user input query text, which is the weighted sum of the i-th word in the query final text sentence representation vector and each word in the government final text sentence representation vector after attention coefficient distribution, e ik represents the word vector of any word in the government final text representation vector, represents the new representation of the i-th word after being weighted and enhanced by all words in the matter name.

[0241] For the j-th word in the matter name, The calculation process and meaning of are the same, as shown in formula (22).

[0242]

[0243] represents the new representation of the government matter name to be matched, which is the weighted sum of the j-th word in the government final text sentence representation vector and each word in the query final text sentence representation vector after attention coefficient distribution, e kj represents the word vector of any word in the user input query text representation vector.

[0244] After alignment through the soft attention mechanism, the new representation vectors of the user input query text and the to-be-matched government affair matter name after interaction are obtained and The dimensionality reduction is performed on and by using a pooling strategy, so as to be subsequently fused with other features. The average pooling combined with the maximum pooling is used, and the calculation process is shown in formulas (23) and (24).

[0245]

[0246]

[0247] v q,ave , v q,max , v s,ave , v s,max respectively represent four vectors obtained after the average pooling and the maximum pooling of the user input query text and the to-be-retrieved matter name.

[0248] Then, the representation vectors of the user input query text and the to-be-matched government affair matter name obtained through the soft attention interaction operation are spliced to obtain:

[0249]

[0250]

[0251] wherein, respectively represent the final representation vector of the user query corresponding to the user input query text and the final representation vector of the matter name corresponding to the to-be-matched government affair matter name after the soft attention interaction operation. d g represents the hidden layer dimension set in the GRU.

[0252] Specifically, the process of obtaining the matching degree between the query and the matter name by S2-4 is as follows:

[0253] Before predicting the query-transaction name matching score, in order to more effectively combine the features obtained by each module of the model, the fusion layer designed by the application fuses the obtained secondary coding values of CNN, the name sentence vectors enhanced by government affairs knowledge (the query is the SIF sentence vector) and the sequence feature values represented by soft attention interaction. The cross fusion of multiple features is realized by vector pair subtraction and vector pair multiplication. The user input query text and the to-be-matched government transaction name are spliced respectively with the three features of each of them

that is, the query keyword feature coding value of the user input query text, the sentence vector value after SIF embedding of the user input query text, and the fusion of the final representation vector of the user query; the government keyword feature coding value of the to-be-matched government transaction name, the sentence vector value after SIF embedding of the to-be-matched government transaction name, and the final representation vector of the transaction name

[0254]

[0255]

[0256] The two sequences are respectively subjected to vector pair subtraction and vector pair multiplication, and then the results and the original sequence values are spliced:

[0257] m out =[m Q ;m S ;m Q -m S ;m Q ⊙m s ] (29)

[0258] Through the fusion layer, the matching degree between the query and the transaction name is represented by vector .

[0259] Specifically, the process of S2-5 calculating the matching score is as follows: as shown in formula (30):

[0260] m score =F s (W·m out +b) (30)

[0261] Wherein, W, b represent the parameters of the multi-layer fully connected layer, and F s (·) is a Sigmod activation function.

[0262] The fusion layer cross-fuses the multi-granularity features of the text, further improving the semantic matching ability of the model.

[0263] Specifically, the application introduces learning to rank (LTR) to optimize the training mode of the model and improve the retrieval capability of the government service matter retrieval model. Learning to rank refers to applying a machine learning method to a ranking task. The main goal of learning to rank is to learn a ranking function that maps a query and a document pair to a score for ranking relevant documents in front and irrelevant documents in back.

[0264] The application proposes a ServiceRank loss function based on Pairwise design. For a large number of consistent matters corresponding to a query, a certain bias is introduced when randomly selecting positive and negative samples to ensure that the relevance difference between positive and negative samples is large enough.

[0265] Margin Ranking Loss introduces the minimum relevance gap (Margin) between two matters for each sample pair when calculating the loss. For each sample pair (x i ,x j ), the loss function is as follows:

[0266] l(x k ,x j )=max(0,m-(s(x k )-s(x j ))) (31)

[0267] Where m represents the Margin parameter, s(x i ) and s(x j ) are the relevance scores of samples x k and x j , respectively. If the relevance of x i is greater than that of x j , then s(x i )>s(x j ), and vice versa. If the relevance difference between the two matters is less than m, the loss is 0, otherwise the loss is m minus the relevance difference between the two matters.

[0268] This method can avoid the problem of randomly selecting positive and negative samples, while encouraging the model to learn the relevance difference between documents. On the other hand, the amount of calculation is very small, avoiding the non-convexity of the loss function, and the overall efficiency is better than FRank.

[0269] On the other hand, to solve the problem of uneven distribution of the number of matters corresponding to the query, the application improves the IR-SVM based on the matter retrieval dataset, and proposes to directly introduce the number of matters corresponding to each query as a weight coefficient to balance different numbers of relevant matters. Combined with Margin Ranking Loss, the application proposes a ServiceRank loss function:

[0270]

[0271] wherein, denotes the data set consisting of user input texts and name texts of affairs to be retrieved, one sample is one user input and two name texts, denotes the number of samples in the data set, s i denotes the number of name texts of affairs corresponding to the i-th user input query text, denotes the set of all name texts of affairs corresponding to the i-th user input query text, l(x j ,x k is the Margin Ranking Loss calculated by (31).

[0272] The weighted method increases the weight coefficient s i for considering the number of affairs contained in each query, and minimizes the influence of the imbalance of the number of affairs corresponding to different queries on the model.

[0273] Experimental design and result analysis

[0274] 1. Experimental data set

[0275] To verify the effect of KEMG on the task of government service affair retrieval, the model needs to be applied to the data set for experiment. Since there is no public government service affair data set for matching or searching in the academic field at present, the present application attempts to construct a government service affair retrieval data set based on the “Common Questions” block data of the Chongqing Yukuai government service platform.

[0276] The “Common Questions” block is located in the affair detail page, and each question belongs to the corresponding affair. The present application takes this as the relevance annotation reference. After cleaning the original data by removing empty characters and empty fields, it is found that there are 5158 service affairs in the question and answer block, corresponding to 19084 questions and answers. In the information retrieval data set of the ranking learning field, one query corresponds to n documents and is manually annotated with multiple levels of relevance. Since it is impossible to obtain data such as government website backend click logs, and the cost of manual annotation of a large number of multi-level labels is high, it is temporarily impossible to achieve, therefore, the present application selects to construct a Pairwise format retrieval data set, the specific steps are as follows:

[0277] ① The Pairwise format data contains four parts, which are query, service1, service2 and two-level relative relevance label (1 for the relevance of the former document greater than the latter, and vice versa). The form of a single sample is shown in the table.

[0278] Table 1 Format of a single sample of the data set

[0279]

[0280] ②Different matters exist under the same problem, such as "mechanic school cancellation" and "mechanic school establishment" both contain the problem "what business can be applied for by newly established mechanic school?". For such cases, the problem and the corresponding multiple matters are grouped and extracted, and there are a total of 403 groups, with an average of 3.2 matters per group. These special samples are manually annotated, and three-level classification labels are defined: 0 (irrelevant), 1 (definitely relevant), and 2 (very relevant), Figure 11 Some labeled samples are shown. Then according to the size of the relevance, a script is written to construct Table 1 format samples. A query containing n matters can construct samples. It should be noted that due to the existence of the number of matters greater than the number of label categories, considering that the proportion is small, in the case of the same relevance label, the original relative position is followed.

[0281] ③In addition to the case in ②, the regular problem-matter pair is a many-to-one relationship. According to the above, the relevant positive examples in the data sample are the matter names belonging to the problem. For negative sampling, unlike random sampling and other methods, the invention adopts a special sampling method for the characteristics of the subdirectory of the government affairs website matters (see Figure 12 ).

[0282] Matters under a large category are located in the same directory and have certain similarity in name. Using them as negative examples to construct samples can improve the generalization ability of the model. Here, the judgment is added: if the problem-matter pair conflicts with ②, discard it to avoid repeated sample construction. For a directory containing m sub-matters, assuming that each sub-matter contains an average of k questions and answers, we can construct k×(m-1)×m samples. For matters that do not contain subdirectories, use random negative sampling to obtain negative examples to construct Pairwise format samples. The number of samples corresponding to a query refers to the average number of samples corresponding to each query in the final statistics.

[0283] Finally, the data set is divided into training set, validation set and test set using the 6:2:2 ratio after shuffling. The information statistics of the government service matter retrieval data set constructed by the invention are shown in Table 2 and Table 3:

[0284] Table 2 Overall information of data set

[0285]

[0286] Table 3 Detailed information of data set

[0287]

[0288] 2. Model evaluation indicators

[0289] In the field of text retrieval, commonly used evaluation methods include MAP (Mean Average Precision), MRR (Mean Reciprocal Rank), NDCG (Normalized Discounted Cumulative Gain), and other indicators. The present application uses MAP, MRR, and NDCG (only considering the first 5 elements) as model retrieval performance evaluation indicators.

[0290] ① MAP

[0291] MAP is based on AP (Average Precision), which is an indicator used in the field of information retrieval to evaluate the quality of ranking results for a single query. For a single user query sentence, AP refers to the average reciprocal of the position of relevant items in the ranking results, defined as follows:

[0292]

[0293] where |R| represents the total number of relevant items in the retrieval results, P(k) represents the precision of the top k retrieval results, rel k indicates the relevance of the kth retrieval result, with relevance being 1 and irrelevance being 0.

[0294] MAP considers the quality of all retrieval results and averages the AP values for all queries:

[0295]

[0296] where |Q| represents the number of queries, i.e., the total number of queries to be evaluated.

[0297] ② MRR

[0298] MRR is used to measure the average reciprocal of the position of the first relevant item in the ordered list returned by an information retrieval system, defined as follows:

[0299]

[0300] where |Q| represents the total number of queries, rank i represents the ranking of the first correct result for the ith query. Unlike MAP, MRR does not consider the order and number of retrieval results, but only focuses on the position of the first correct result, so it is more suitable for evaluating tasks with a single correct answer.

[0301] ③ NDCG

[0302] MAP and MRR focus on the ranking of all relevant search items, while NDCG focuses more on the weight of the top-ranked items. NDCG is based on Discounted Cumulative Gain (DCG), which is an indicator for measuring the performance of ranking models, defined as follows:

[0303]

[0304] where rel i represents the relevance of the i-th item, and k represents the top k items for calculating DCG. The numerator of the formula represents the weight of the item relevance, and the denominator log2(i+1) represents the weight of the item ranking position. When the item is ranked high and has high relevance, its contribution value will be greater. Based on DCG, DCG is divided by an ideal DCG value to eliminate the influence of the number of items, and NDCG is defined as follows:

[0305]

[0306] where DCG@k represents the DCG value of the top k items of the ranking model, and IDCG@k represents the DCG value of the top k items in the ideal case. The value of NDCG ranges from 0 to 1.

[0307] 3. Experimental model

[0308] To verify the retrieval ability of KEMG, the present application selects representative algorithm models in the field of text matching and retrieval according to the progress of related engineering research as a control for comparison experiment. The control models are mainly divided into four categories, which are based on word frequency statistics, representation model, interactive model and Bert-based model. Specifically, they include:

[0309] Based on word frequency statistics: TF-IDF, BM25

[0310] Representation model: ARC-I, DSSM, CDSSM

[0311] Interactive model: ARC-II, ESIM, BiMPM

[0312] BERT-Based: BERT-base, Sentence-Bert (S-BERT)

[0313] On the other hand, to verify the effectiveness of ServiceRank designed for the data set of the present application, representative algorithms are selected from three types of ranking learning algorithms and applied to all comparison models, mainly including:

[0314] Pointwise: Cross Entropy

[0315] Pairwise: RankNet, LambdaRank

[0316] Listwise: ApproxNDCG, ListRank

[0317] 4 Experimental setup and comparative analysis of results

[0318] ① Experimental setup

[0319] The model experimental parameters are set as follows: sequence length 64; embedding layer Word2vec static word vector dimension 200; JWE embedding word granularity vector dimension 100; query, transaction name and related government knowledge SIF embedding sentence vector dimension 300; encoding layer BiLSTM Units 128; CNN convolution kernel size 2, 3, 4; two layers of fully connected layer are set, respectively containing 256, 128 neurons; dropout rate 0.3; optimizer using Adam, learning rate set to 1 x 10 -4 ; mini-batch gradient descent is used during training to improve training speed and effect; the number of iterations (epoch) is set to 15 rounds; batch-size is 64, and the optimal check point is selected according to the effect of the validation set.

[0320] The hyperparameters of the comparative model are obtained by the built-in dynamic parameter tuning function of MatchZoo.

[0321] ② Comparative results and analysis of model effects

[0322] Five experiments were conducted, and the final results were averaged. Tables 4 and 5 show the performance of MAP, MRR and NDCG from the aspects of model comparison and loss function comparison.

[0323] Table 4 Comparative experimental results table one

[0324]

[0325] Table 5 Comparative experimental results table two

[0326]

[0327]

[0328] Table 6 Comparative experimental results table three

[0329]

[0330] Figure 13(a) intuitively shows the retrieval effect comparison of the model under ServiceRank (TF-IDF and BM25 are trained under Pointwise). According to the experimental result analysis, due to the strong text feature extraction ability, the deep learning method generally has stronger retrieval ability, which is higher than the statistical algorithm in three indexes, and the KEMG proposed in the application increases by 18.4% and 17.6% compared with TF-IDF and BM25 in NDCG. Among the deep models, the dual tower model is inferior to the interactive model in extracting fine-grained semantic features because it does not have text interaction, and is lower than the interactive model, BERT model and KEMG in indexes. ESIM uses soft attention to interact with text at the word level, and the retrieval effect is the best in the interactive model. The model KEMG proposed in the application strengthens the feature extraction of query-transaction name pairs by introducing multiple modules in multiple ways on the basis of ESIM, and compared with ESIM, MAP, MRR and NDCG are increased by 1.4%, 1.5% and 1.8%, respectively, and the text feature extraction ability in the government field is improved.

[0331] The model fuses government knowledge through graph attention, increases the complexity, but reduces the overall parameter quantity through GRU, CNN and other modules, and reduces the training cost. The model parameter quantity, training time and test time results are shown in Table 7, and the time cost index comparison chart is shown in Figs. Figure 13 (c) and (d). Analysis of the data shows that the parameter quantity of KEMG is slightly less than that of ESIM, and is much less than that of the BERT model, and is increased by 1.1%, 1.2% and 0.7% compared with BERT-base in three indexes. S-BERT uses BERT as an encoder combined with a dual tower model architecture to achieve the best retrieval effect, but the parameter quantity of KEMG is only 1 / 1000 of that of S-BERT, the training time and test time are averagely fast by 76.4% and 93.6%, and the test time represents the online inference efficiency of the model to a certain extent, which verifies the feasibility and effectiveness of KEMG in industrial actual deployment and online algorithm.

[0332] Figure 13(b) Intuitively shows the retrieval performance comparison results of the deep model applied to different loss function schemes (Pointwise takes the TF-IDF training value, and other loss function schemes take the application to KEMG). According to the experimental result analysis, the Pointwise training mode only considers the relevance of a single query-item name pair, does not learn the relative size order of multiple associated items, and has the worst effect on the retrieval index considering the overall ranking effect of the list. RankNet and LambdaRank belong to the Pairwise algorithm, which learns by constructing a pair of partial order document pairs as samples, so that the model has the ability to judge the order of relevance, and the retrieval ability is greatly improved by 7.0%, 7.0%, 6.3% and 7.1%, 7.3%, 7.0% respectively.

[0333] ApproxNDCG and ListRank belong to the Listwise algorithm, and Listwise directly optimizes the overall returned list, which is usually better than Pointwise and Pairwise. In the matter retrieval task of the application, the two Listwise-based algorithms are generally lower than the Pairwise method in the index, and the Listwise RankList index is 1.1%, 0.8%, 0.5% higher than LambdaRank. There are a large number of random selection cases under the secondary label data set of list ranking, which cannot accurately reflect the real relevance ranking, which verifies that the Listwise algorithm cannot fully utilize the overall ranking advantage, affecting the retrieval ability.

[0334] The ServiceRank loss function proposed in the application retains the secondary label information in a single sample and introduces a hyperparameter m to judge the relevance threshold, maximally retains the supervision information, introduces the weight for balance, reduces the risk brought by random selection of positive and negative examples in the traditional Pairwise algorithm, and solves the problem of uneven sample distribution. Experimental results show that ServiceRank is better than Pairwise algorithm in three indexes, and is improved by 2.2%, 2.1%, 2.4% respectively compared with LambdaRank, which verifies its effectiveness.

[0335] ③ Model convergence process

[0336] Figure 14 The KEMG loss function value on the training set and the validation set and the change of the three indexes on the validation set are shown with the iteration of the self-updating of the KEMG. The horizontal coordinate in the figure represents the number of iterations (Epoch) of the model training, and the vertical coordinate represents the loss function value and the evaluation index value of the model.

[0337] ④ Ablation experiment results and analysis

[0338] The effectiveness of each module is verified by an ablation experiment. The experimental results are trained based on ServiceRank. The experimental results are shown in Table 7.

[0339] Table 7 Ablation experiment results

[0340]

[0341] Figure 15 The ablation experiment results are compared and displayed from the aspects of single modules and combined modules, so as to more intuitively reflect the contribution of each module to the improvement of the overall retrieval ability of the model.

[0342] From the comparison of the experimental results of single modules, it is found that the government knowledge enhancement module has the largest contribution to the improvement of the retrieval ability of the model. Among the single modules, the word granularity embedding and the CNN module have smaller improvement effects, and some indicators are slightly lower than ESIM. This may be because ESIM does not introduce related modules, and after the difference calculation in the fusion layer, noise is added to the features, which also indicates the necessity and effectiveness of the design of the feature fusion layer.

[0343] From the experimental results of the combined modules, it is found that the three combined modules containing knowledge have better performance. Combined with the experimental results of the single modules, it is analyzed that the government knowledge enhancement module has the largest contribution to the improvement of the overall retrieval ability of the model, which proves the effectiveness of the knowledge enhancement model service item retrieval ability proposed in the application.

[0344] ⑤ Hyperparameter experiment and analysis

[0345] In order to verify the influence of some hyperparameters on the retrieval ability of the model, the experimental effects of the values of SIF parameter a, m in ServiceRank, the number of neurons in the full connection layer and the dropout rate of the last layer are compared respectively (since the index trend is consistent, only the NDCG index is shown in the experimental results).

[0346] The adjustment range of a in the SIF sentence vector generation method is 10 0 to 10 -5 . The m in ServiceRank is fine-tuned between 0.1 and 0.6 with a step of 0.1. The experimental results are shown in Figure 16 (a) and Figure 16 (b). The influence trend of both on the model is first increased and then gradually decreased. When a = 10 -3 and m = 0.3, the NDCG value of the model reaches the highest.

[0347] The number of neurons is fine-tuned between 16 and 256 with a step of 16. The dropout rate is fine-tuned between 0.1 and 1 with a step of 0.1. The experimental results are shown inFigure 16 (c) and Figure 16 (d) are shown. The number of neurons has a first effect on the model, then a plateau in the range 128-192 and a maximum, then a decreasing trend. The dropout rate has a maximum at 0.3, then a general downward trend.

[0348] According to Figure 16 In the analysis, the three items that contribute the most to the overall when fusing enhanced matter name representation are "acceptance conditions", "setting basis", and "application materials". Combined with knowledge content analysis, there is a large amount of information in "acceptance conditions" and "setting basis" that is descriptive of the matter itself. For example, in the matter "handling of provident fund withdrawal", the keywords in the acceptance conditions "…repayment of provident fund loans (including combined loans, interest subsidy loans)…workers, spouses, and…" and the setting basis "workers have one of the following situations…(1) purchase, build, rebuild, or major repair self-owned housing; (2) retired, retired…" contain information about the associated handler and supplementary description information other than the name of the matter, which is easy for users to input into the query.

[0349] Similar to "application materials", "handling location", or "approval person" and other knowledge information, they contain information about the handling process. Some users who have some understanding of the matter to be handled may supplement the details of the handling in the input query, such as the query corresponding to the matter "student entrepreneurship subsidy application in Chunhua Avenue" is "Can I handle the student entrepreneurship subsidy application in Chunhua Avenue in Xiantao Street, Yubei District, Chongqing?", and the associated handling location "Chunhua Avenue in Xiantao Street, Yubei District, Chongqing…" contains the keyword information "Chunhua Avenue" that is related to the query.

[0350] The results of the service matter knowledge weight allocation not only show the contribution of different matter information to the enhancement of matter representation, but also verify the rationality of the proposed fusion of government knowledge to enhance the matter retrieval capability of the model.

[0351] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A knowledge-enhanced multi-granularity government affair service matter recommendation method, characterized in that: Comprise the following steps: S1: obtaining a plurality of user input query text and to be matched government affair name, constructing government affair knowledge graph; S2: constructing and training KEMG model, the KEMG model includes embedding layer of fusion word granularity information, coding layer based on CNN enhancement, interactive layer based on attention mechanism, fusion layer and multi-layer fully connected layer to obtain similarity prediction score; S2-1: the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carried out word vector, and the embedding corresponding of all user input query text and to be matched government affair name is carries ​ ​ ​ wherein, respectively represent the new encoding values of the user input query text and the to-be-matched government affair item name after BiGRU at the i-th time step and the j-th time step, d g represents the hidden layer dimension set in the BiGRU; ​ wherein, concatenation matrix of word vectors concatenation matrix of word vectors denotes a concatenation operation, and the convolution kernel It extracts new features in the form of a window of size w sliding over the text, as shown in equation (12): wherein C i represents the ith feature extracted by the convolution kernel during the convolution. input vectors that are out of range (i < 1 or i > 1) as zero vectors, representing word vectors q i q i+1 q i+2 q w q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Applying a convolution kernel of size w to a sentence Each of the possible hidden states is extracted from the feature map as shown in equation (13); q = [q1, q2,..., q l-w+1 ] T (13) wherein The resulting six feature maps are input into a max-pooling layer to obtain a fixed-length feature vector representation: Q i = P({q i1 ,...,q i2 ,...,q i(l-w+1)}) (14) wherein Q i represents the feature vector of the i-th feature map after the max-pooling operation, q i1 represents the 1st feature of the i-th feature map, q i(l-w+1) represents the (l-w+1)th feature of the i-th feature map; Finally, the feature vectors after the pooling are spliced together to form the final query keyword feature encoding through formula (15) The same method is used to obtain the government affair keyword feature code ​ ​ ​ ​ ​ 2. The knowledge-enhanced multi-granularity government affair service matter recommendation method according to claim 1, characterized in that: ​ The user input query text and the government affair matter text to be matched are embedded with word2vec to obtain word vectors, as shown in equations (1) and (2): wherein Q w and S w represent the vector matrix composed of word vectors after the user input query (Query) and the to-be-matched government service item name (ServiceName) are processed by Word2vec, respectively, represent the w-dimensional vector obtained after each word in the two texts is embedded, d w represents the dimension of the word vector, l a and l b represent the length of the user input query and the to-be-matched government service item name, respectively.

3. The knowledge-enhanced multi-granularity government affair service matter recommendation method of claim 2, wherein: The process of obtaining the word granularity vector by S2-1 is as follows: The word embedding of the text is realized by using the joint learning word embedding model JWE, as shown in equation (3): wherein k = 1, 2, 3 respectively refer to the context word, the context Chinese character, and the context sub-character, w i is the target word, L(w i ) represents the sum of the log likelihood of the three prediction conditional probabilities of the target word w i ​ Predicted probability defined by the Softmax function as shown in equation (4): wherein is the output vector of the target word w i represents the current target word, represents any one of the sum of all target words from 1 to N;​ Each word in the user input query and the to-be-matched government service matter name is represented by a word vector through JWE, and the word vector representations of the user input query and the to-be-matched government service matter name are respectively denoted as Q j and S j As shown in formulas (5) and (6): wherein q i , respectively represent the word vector representation of the i-th and j-th Chinese character in the user input query and the to-be-matched government service matter name, d c represents the dimension of the word vector, z q , z s respectively represent the number of words in the user input query and the to-be-matched government service matter name.

4. The knowledge-enhanced multi-granularity government affair service matter recommendation method of claim 3, wherein: The process of splicing and fusing the word granularity vector and the word granularity vector in S2-1 to form the final text pair embedding representation is as follows: a new combination vector Q of the i-th word in the query input by the user i The calculation formula is shown as (7): wherein represents the feature vector after the pooling operation of all the word granularity vectors constituting the i-th word in the text; The query final text sentence representation vector Q is calculated, and the calculation formula is shown in equation (8): Q = (Q1,..., Q l ) (8) wherein, The word vector embedding dimension d defining the fusion word granularity of the model is d = d w + d c ; The government final text sentence representation vector S can be obtained by using the same method as obtaining Q.

5. The knowledge-enhanced multi-granularity government affair service matter recommendation method according to claim 4, characterized in that: The process of obtaining the user query final representation vector and the matter name final representation vector in S2-3 is as follows: The SIF sentence vector generation method comprises: 1) Get the initial sentence vector: traverse all the sentences in the corpus, and calculate the initial sentence vector v of the current sentence s by formula (16) s ; where |s| represents the number of words in sentence s, v w represents the word vector of word w in sentence s, p(w) is the probability of word frequency of word w in the whole corpus, and a is an adjustable parameter; 2) Principal component analysis of all preliminary sentence vectors: the first principal component u of all preliminary sentence vectors is calculated, which is called a public discourse vector, and u is the first principal vector of the matrix composed of the word vectors of the sentence; 3) Obtain the target sentence vector: the preliminary sentence vector is processed twice by formula (17) to obtain the target sentence vector; wherein represents the sentence vector value after SIF embedding, w respectively represents the sentence and each word. The sentence vector value of the user input query text embedded by SIF is obtained by using formula (18) The sentence vector value of the to-be-matched government affair matter name embedded by SIF The improved graph attention calculation formula is shown in equation (19); wherein, represents the enhanced government affair matter name sentence vector representation of the fusion government affair knowledge graph, represents all government affair knowledge nodes directly associated with the government affair matter, including the government affair matter itself, represents the sentence vector value obtained after the relevant government affair knowledge is embedded by the SIF. The soft attention mechanism interacts with the query keyword feature encoding value and the government keyword feature encoding value: First, calculate the correlation score, and the score value is calculated as shown in equation (20): where e ij represents the query final text sentence representation vector and the query final text sentence representation vector the relevance score derived by the dot product The correlation score is normalized to convert the correlation score into a weight coefficient for subsequent weighted representation, as shown in equation (21): wherein, represents the user input query text new representation vector, is the weighted sum result of the i th word in the query final text sentence representation vector and the attention coefficient distribution of each word in the government final text sentence representation vector, e ik represents the word vector of any word in the government final text representation vector; For the jth word in the transaction name, The calculation process and meaning are the same as shown in equation (22): represents the new representation vector of the government affair matter to be matched, which is the weighted sum result of the attention coefficient distribution of the jth word in the final text sentence representation vector of the government affair and each word in the final text sentence representation vector of the query, e kj represents the word vector of any word in the query text representation vector input by the user; The pooling strategy is adopted to reduce the dimension of the feature map and The average pooling and the maximum pooling are combined, and the calculation process is shown in formulas (23) and (24): v q,ave , v q,max , v s,ave , v s,max respectively represent four vectors obtained by average pooling and maximum pooling on the user input query text and the to-be-retrieved matter name respectively; Then, the representation vectors of the user input query text and the government affair matter name obtained by the soft attention interaction operation are spliced: wherein, respectively represent the user query final representation vector corresponding to the user input query text and the matter name final representation vector corresponding to the to-be-matched government affair matter name after the soft attention interaction operation, d g represents the hidden layer dimension set in the GRU.

6. The knowledge-enhanced multi-granularity government affair service matter recommendation method according to claim 5, characterized in that: The process of obtaining the matching degree between the query and the matter name by S2-4 is as follows: The vector pair subtraction and vector pair multiplication are used to realize multi-feature cross fusion, and the user input query text and the government affair matter name are spliced with their respective three features to store in a sequence: The two sequences are respectively subtracted and multiplied, and then the results and the original sequence values are spliced: m out = [m Q ; m S ; m Q - m S ; m Q ⊙ m S ] (29) Through the fusion layer, the degree of matching between the query-transaction name is represented by a vector .

7. The knowledge-enhanced multi-granularity government affair service matter recommendation method according to claim 6, characterized in that: The process of calculating the matching score by S2-5 is as follows: as shown in equation (30): m score = F s (W·m out + b) (30) where W, b represent the parameters of the multi-layer fully connected layer, F s (·) is the Sigmod activation function.

8. The multi-granularity government affair service matter recommendation method based on knowledge enhancement according to claim 7, characterized in that: The Margin Ranking Loss introduces the minimum margin of relevance between two items when calculating the loss, for each sample pair (x i ,x j ), the loss function is as follows: l(x k ,x j ) = max(0, m - (s(x k ) - s(x j ))) (31) where w denotes a Margin parameter, s(x i ) and s(x j ) are the relevance scores of samples x k and x j , respectively; The ServiceRank loss function is proposed: wherein, denotes a dataset consisting of user input texts and to-be-retrieved matter name texts, denotes the number of samples in the dataset, s i denotes the number of government affair matter names corresponding to the i-th user input query text, denotes the set of all government affair matter names corresponding to the i-th user input query text, l(x j ,x k ) is the Margin Ranking Loss calculated by (31).

Images