A Case Similarity Prediction Method Integrating Knowledge Representation Models and Its Application

Abstract

This invention discloses a case similarity prediction method integrating a knowledge representation model and its application. The method includes: 1. acquiring and preprocessing government consultation cases; 2. constructing a case similarity prediction model; 3. iteratively training the model; and 4. using the trained model to complete the case similarity prediction. This invention comprehensively considers the characteristics of existing consultation cases, extracts keyword features using a knowledge representation model, and integrates the topic similarity and case feature similarity of cases, thereby more accurately finding cases similar to new cases among existing cases and responding to the public quickly and accurately.

Description

Technical Field

[0001] This invention belongs to the field of intelligent question answering for government affairs consultation, specifically a case similarity prediction method that integrates knowledge representation models and its application. Background Technology

[0002] The current field of government consultation suffers from two main problems. Firstly, consultation responses rely heavily on manual processes, resulting in a large workload, broad scope, and high repetition. Secondly, the existence of various public-facing government consultation platforms and channels fails to adequately consider the diverse sources of consultation cases. Furthermore, the topic of government consultation, as a crucial influencing factor in consultation cases, is not adequately addressed. Summary of the Invention

[0003] The present invention addresses the shortcomings of the existing technology by proposing a case similarity prediction method and its application that integrates a knowledge representation model. This method aims to quickly identify similar cases, improve the efficiency of intelligent question answering, and enable rapid and accurate responses to inquiries.

[0004] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0005] The present invention provides a case similarity prediction method that integrates a knowledge representation model. The method is applied to a government affairs consultation knowledge base and includes the following steps:

[0006] Step 1. Obtain and preprocess government consultation cases:

[0007] Retrieve any i-th government consultation case R from the government consultation knowledge base. i =(q i ,T i ,b i ); where q i T represents the i-th Chinese inquiry question; i Let q represent the i-th Chinese consultation question. i a collection of keywords; b i Let q represent the i-th Chinese consultation question. i Corresponding channels;

[0008] Let the i-th government consultation case R i Chinese consultation questions q i First translate it into English, then into Chinese, to obtain a similar question q' i ; From similar problems q' i and its corresponding channel b i and keyword set T' i Forming a sequence R′ of similar government consultation cases i ;

[0009] Step 2. Construct a case similarity prediction model, including: a case feature extraction module, a topic representation module, and a similarity calculation module;

[0010] Step 3. The case feature extraction module includes: a text semantic representation unit, a channel representation unit, and a case fusion representation unit; and the case sequence R... i and R′ i The case feature extraction module is input to obtain the corresponding case vector, Case. i and the corresponding similar case vector Case' i :

[0011] Step 3.1. The text semantic representation unit uses the BERT model to represent the i-th government consultation case R. i Chinese consultation questions q i Word embedding is performed to obtain all character vectors in the corresponding Chinese consultation question. Then, the average value of the character vectors is calculated to obtain the mean vector, which is used as the corresponding Chinese consultation question q. i sentence vector O i ;

[0012] Step 3.2. The channel representation unit includes a one-hot encoding embedding layer, which embeds the i-th government consultation case R. i Consultation channels b i One-hot vector encoding is performed, and then word embedding is performed to obtain the i-th government consultation case R. i Consultation channels b i The vector representation of B i ;

[0013] Step 3.3. The case fusion representation unit will represent the i-th government consultation case R. i Sentence vector O of Chinese consultation questions i Vector B of consultation channels i By concatenating the vectors, we obtain the i-th government consultation case R. i Vector Case i ;

[0014] Step 3.4. Sequence R′ of similar government consultation cases i The data is input into the feature extractor and processed according to steps 3.1-3.3 to obtain the sequence R′ of similar government consultation cases. i Similar vectors Case' i ;

[0015] Step 4. The topic representation module includes a knowledge representation layer and an attention layer;

[0016] The knowledge representation layer obtains the encoding of each topic term and its relationship from the government affairs consultation knowledge base and inputs it into the knowledge representation layer. The knowledge representation layer uses the trans-E model to obtain the representation vector corresponding to each topic term.

[0017] The attention layer first processes the i-th government consultation case R. i Chinese consultation questions q i Keyword set T i After aligning the length L of the representation vector corresponding to each topic term, we obtain the topic term set T. i L topic word vectors; in the sentence vector O i Under supervision, the L topic word vectors are processed using the Attention mechanism to obtain the attention weights of the L topic word vectors. Finally, the L topic word vectors and their corresponding attention weights are weighted to obtain the i-th government consultation case R. i Topic Representation Vector C i ;

[0018] The attention layer focuses on the sequence R′ of similar government consultation cases. i After performing the same processing on the keyword set in the dataset, a sequence of similar government consultation cases R′ is obtained. i The topic representation vector C' i ;

[0019] Construct a system based on the i-th government affairs consultation case R i And a sequence of similar government consultation cases R′ i The positive sample pairs formed;

[0020] Construct a system based on the i-th government affairs consultation case R i and the j-th government consultation case sequence R j The negative sample pairs formed;

[0021] Step 5. The similarity prediction module includes two fully connected layers, a fusion layer, and a Sigmod function module:

[0022] Step 5.1. Convert the vector Case i and Case' i After concatenation, the similarity representation vector D of the positive sample pairs is obtained. 1i ;

[0023] Vector Case i Vector Cases of Other J-th Government Affairs Consultation Cases j After concatenation, we obtain the dissimilarity representation vector D of the negative sample pairs. 2i ;

[0024] D 1i and D 2iAfter dimensionality reduction through two fully connected layers, the scalar x is obtained. 1i x 2i Then, for the scalar x 1i x 2i After normalization, the text similarity representation X of positive sample pairs is obtained. 1i Text similarity representation X between negative sample pairs 2i ;

[0025] Step 5.2. The fusion layer calculates the Euclidean distance d(C) between positive sample pairs. i ,C' i The Euclidean distance d(C) between the negative sample pairs and the negative sample pairs i C j Then calculate the topic similarity of positive sample pairs. Topic similarity with negative sample pairs Therefore, the fusion similarity representation Y of positive sample pairs is calculated. 1i =α·Z 1i +(1-α)X 1i The fusion similarity representation of the negative sample pair Y 2i =α·Z 2i +(1-α)X 2i Where α represents the weight, and α∈[0,1];

[0026] Step 5.2. Merge the similarity representation Y 1i and Y 2i The results are input into the Sigmod function module for processing to obtain the corresponding similarity prediction result σ(Y). 1i ) and σ(Y 2i );

[0027] Step 6. Training the case similarity prediction model:

[0028] Step 6.1 Construct the loss function L(W) using equation (1):

[0029] L(W)=L1(W)+L2(W) (1)

[0030] In equation (1), L1(W) represents the loss function for positive sample pairs, which is obtained from equation (2); L2(W) represents the loss function for negative sample pairs, which is obtained from equation (3).

[0031]

[0032]

[0033] In equations (2) and (3), σ(x) is the Sigmoid function. This is the true label value of the sample pair. If it is a positive sample pair, then... If it is 1, then if it is a negative sample pair, then =0;

[0034] Step 6.2 Train the case similarity prediction model using gradient descent and calculate the loss function L(W). Stop training when the loss function L(W) converges or reaches its maximum value, thereby obtaining the optimal case similarity prediction model with fused knowledge representation.

[0035] Step 6.3 When acquiring new government affairs consultation cases R * First, the case feature extraction module and topic representation module, after training, are processed to obtain the case R. * Vector Case * and its topic representation vector C * and Case i and C i The data are input together into the trained similarity prediction module for processing, and the similarity prediction result σ(Y) is output. * If σ(Y) * If ) is 1, then the new government consultation case R will be... * The fusion similarity representation of Y * The top s maximum values ​​are used as similar cases for push notifications.

[0036] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing a case similarity prediction method of the fused knowledge representation model, and the processor is configured to execute the program stored in the memory.

[0037] The present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the steps of the case similarity prediction method of the fused knowledge representation model.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] 1. This invention proposes a method for case similarity prediction that combines knowledge representation learning and text similarity calculation. This method can effectively predict case similarity and push similar cases of government consultation to users, reducing repetitive work. Furthermore, this invention believes that the topic of government consultation plays a key role in government consultation cases; therefore, case similarity prediction that integrates the topic similarity of government consultation cases can effectively improve the accuracy of prediction, making it more convenient and valuable in practical applications.

[0040] 2. This invention adopts the idea of ​​contrastive learning, uses existing cases for similarity preprocessing, and constructs similarity prediction positive sample pairs between the similarity preprocessed cases and existing cases, which saves a lot of manual annotation costs, can fully integrate existing government consultation cases, and also improves the accuracy of case feature extraction.

[0041] 3. This invention obtains semantically related representation vectors of keywords in government consultation cases through a knowledge representation learning model. Then, using an attention mechanism with sentence vectors as supervision, it obtains topic representation vectors that better fit the case topic, thus improving the representation effect of consultation case topics. Furthermore, by including channels in the case feature extraction scope and interactively fusing case topic similarity with case feature similarity, a more effective method for predicting government consultation cases can be obtained. Attached Figure Description

[0042] Figure 1 This is a diagram of the optimal case similarity prediction model that integrates knowledge representation in this invention. Detailed Implementation

[0043] In this embodiment, a case similarity prediction method integrating a knowledge representation model is applied to a government affairs consultation knowledge base. It utilizes a deep learning model to comprehensively consider the characteristics of existing consultation cases. The knowledge representation model obtains topic word representation vectors with semantic relationship features, and then an attention mechanism is used to extract case topics, improving the model's interpretability. Simultaneously, a deep neural network is used to fuse channel and question features to learn accurate case feature representations. Finally, the topic similarity and case feature similarity of cases are integrated to more accurately find cases similar to new cases among existing cases. These similar cases are then pushed to users to achieve intelligent question answering and quickly and accurately respond to the public. Specifically, for example... Figure 1 As shown, the method includes the following steps:

[0044] Step 1. Obtain and preprocess government consultation cases:

[0045] Retrieve any i-th government consultation case R from the government consultation knowledge base. i =(q i ,T i ,b i ); where q i T represents the i-th Chinese inquiry question; i Let q represent the i-th Chinese consultation question. i a collection of keywords; b i Let q represent the i-th Chinese consultation question. i Corresponding channels;

[0046] First, cases with incomplete key information are removed. Then, the text data is processed through word segmentation, stop word removal, standardization of public terms, and information anonymization. Finally, the i-th government consultation case R is... i Chinese consultation questions q i First translate it into English, then into Chinese, to obtain a similar question q' i ; From similar problems q' i and its corresponding channel b i and keyword set T' i Forming a sequence R′ of similar government consultation cases i ; For similar government consultation case sequences R′ i We will standardize the terminology used by the public again to improve the standardization of the cases.

[0047] Step 2. Construct a case similarity prediction model, including: a case feature extraction module, a topic representation module, and a similarity calculation module;

[0048] Step 3. Case feature extraction module, including: text semantic representation unit, channel representation unit, and case fusion representation unit; and the case sequence R i and R′ i Input the case feature extraction module to obtain the corresponding case vector. i and the corresponding similar case vector Case' i :

[0049] Step 3.1. The text semantic representation unit utilizes the BERT model and a pre-trained model to represent the i-th government consultation case R. i Chinese consultation questions q i Word embedding is performed to obtain all character vectors in the corresponding Chinese consultation question. Then, the average value of the character vectors is calculated to obtain the mean vector, which is used as the corresponding Chinese consultation question q. i sentence vector O i ;

[0050] Step 3.2. The channel representation unit includes a one-hot encoding embedding layer, which embeds the i-th government consultation case R. i Consultation channels b i One-hot vector encoding is performed, and then word embedding is performed to obtain the i-th government consultation case R. i Consultation channels b i The vector representation of B i ;

[0051] Step 3.3. The case fusion representation unit will represent the i-th government consultation case R. i Sentence vector O of Chinese consultation questions i Vector B of consultation channels iBy concatenating the vectors, we obtain the i-th government consultation case R. i Vector Case i Since government consultation cases from different channels have different levels of structuring and different themes, vector representation that takes into account channel factors can more comprehensively represent the characteristics of government consultation cases.

[0052] Step 3.4. Sequence R′ of similar government consultation cases i The input is processed in the feature extractor according to steps 3.1-3.3 to obtain the sequence R′ of similar government consultation cases. i Similar vectors Case' i ;

[0053] Step 4. The topic representation module includes a knowledge representation layer and an attention layer;

[0054] The subject terms and their relationship representation codes are obtained from the government affairs consultation knowledge base (subject terms and relationships can be constructed by referring to my country's "e-government subject terms list" and domain professional terminology standards, etc.) and input into the knowledge representation layer. The knowledge representation layer uses the trans-E model to obtain the representation vector corresponding to each subject term. The semantic relationship features between subject terms can be obtained by using the knowledge representation model, which is more suitable for representing the relationship between subject terms than using word embedding technology.

[0055] The attention layer first focuses on the i-th government consultation case R. i Chinese consultation questions q i Keyword set T i After aligning the length L of the representation vector corresponding to each topic term, we obtain the topic term set T. i L topic word vectors; in sentence vector O i Under supervision, the Attention mechanism is used to process L topic word vectors to obtain L attention weights for each topic word vector. Clearly, under the supervision of the consultation question, these weights are more closely aligned with the case topic. Finally, the L topic word vectors and their corresponding attention weights are weighted to obtain the i-th government consultation case R. i Topic Representation Vector C i ;

[0056] Attention layer on similar government consultation case sequences R′ i After performing the same processing on the keyword set in the dataset, a sequence of similar government consultation cases R′ is obtained. i The topic representation vector C' i ;

[0057] Construct a system based on the i-th government affairs consultation case R i And a sequence of similar government consultation cases R′ i The positive sample pairs formed;

[0058] Construct a system based on the i-th government affairs consultation case R i and the j-th government consultation case sequence R j To avoid the possibility of positive samples appearing in the construction of negative sample pairs, negative samples are formed by using cases where there are no identical subject words in the subject word set.

[0059] Step 5. The similarity prediction module consists of two fully connected layers, a fusion layer, and a Sigmoid function module:

[0060] Step 5.1. Convert the vector Case i and Case' i After concatenation, the similarity representation vector D of the positive sample pairs is obtained. 1i ;

[0061] Vector Case i Vector Cases of Other J-th Government Affairs Consultation Cases j After concatenation, we obtain the dissimilarity representation vector D of the negative sample pairs. 2i ;

[0062] D 1i and D 2i After dimensionality reduction through two fully connected layers, the scalar x is obtained. 1i x 2i Then, for the scalar x 1i x 2i After normalization, the text similarity representation X of positive sample pairs is obtained. 1i Text similarity representation X between negative sample pairs 2i ;

[0063] Step 5.2. The fusion layer calculates the Euclidean distance d(C) between positive sample pairs. i ,C' i The Euclidean distance d(C) between the negative sample pairs and the negative sample pairs i C j Then calculate the topic similarity of positive sample pairs. Topic similarity with negative sample pairs Therefore, the fusion similarity representation Y of positive sample pairs is calculated. 1i =α·Z 1i +(1-α)X 1i The fusion similarity representation of the negative sample pair Y 2i =α·Z 2i +(1-α)X 2i Where α represents the weight, and α∈[0,1];

[0064] Step 5.2. Merge the similarity representation Y1i and Y 2i The results are input into the Sigmod function module for processing to obtain the corresponding similarity prediction result σ(Y). 1i ) and σ(Y 2i );

[0065] Step 6. Training the case similarity prediction model:

[0066] Step 6.1 Construct the loss function L(W) using equation (1):

[0067] L(W)=L1(W)+L2(W) (1)

[0068] In equation (1), L1(W) represents the loss function for positive sample pairs, which is obtained from equation (2); L2(W) represents the loss function for negative sample pairs, which is obtained from equation (3).

[0069]

[0070]

[0071] In equations (2) and (3), σ(x) is the Sigmoid function. This is the true label value of the sample pair. If it is a positive sample pair, then... If it is 1, then if it is a negative sample pair, then =0;

[0072] Step 6.2 In this embodiment, the positive and negative sample pair dataset is divided into a training set, a validation set, and a test set in an 8:1:1 ratio. A maximum number of iterations is set to 100. The case similarity prediction model is trained using gradient descent. The loss function L(W) is calculated using gradient descent, and training stops when the loss function L(W) converges or reaches the maximum number of iterations, thereby obtaining the optimal case similarity prediction model with fused knowledge representation.

[0073] Step 6.3 When acquiring new government affairs consultation cases R * First, the case feature extraction module and topic representation module, after training, are processed to obtain the case R. * Vector Case * and its topic representation vector C * and Case i and C i The data are input together into the trained similarity prediction module for processing, and the similarity prediction result σ(Y) is output. * If σ(Y) * If ) is 1, then the new government consultation case R will be... * The fusion similarity representation of Y *The top s maximum values ​​are used as similar cases for push notifications.

[0074] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor in executing a case similarity prediction method of the fused knowledge representation model, and the processor is configured to execute the program stored in the memory.

[0075] In this embodiment, a computer-readable storage medium stores a computer program, which, when executed by a processor, performs the steps of the case similarity prediction method of the fused knowledge representation model.

Claims

1. A case similarity prediction method of a fusion knowledge representation model, characterized by, The application is applied to a government affair consultation knowledge base and comprises the following steps. Step 1. Obtain a government affair consultation case and pre-process it. obtaining any one government affair consultation case from a government affair consultation knowledge base i , ); wherein, denotes the i th Chinese consultation question; denotes the i th Chinese consultation question 's keyword set; denotes the i th Chinese consultation question 's corresponding channel;​ Translate the given patent text from Chinese to English, ensuring that no details or formatting are omitted. Use formal and professional terminology, and adhere to the style and norms of American patents. Translate the given patent text from Chinese to English, ensuring that no details or formatting are omitted. Use formal and professional terminology, and adhere to the style and norms of American patents. i Chinese consultation questions in the first government affairs consulting case Translate the given patent text from Chinese to English, ensuring that no details or formatting are omitted. Use formal and professional terminology, and adhere to the style and norms of American patents. from English to Chinese, then from Chinese to English, thereby obtaining similar questions ; Similar government affairs consulting case sequences are composed of similar questions and their corresponding channel and subject word sets ; Step 2. Construct a case similarity prediction model, comprising a case feature extraction module, a topic representation module and a similarity calculation module. Step 3. The case feature extraction module includes a text semantic representation unit, a channel representation unit, and a case fusion representation unit; and the case sequence and is input into the case feature extraction module to obtain a case vector and a corresponding similar case vector : Step 3.

1. The text semantic representation unit carries out word embedding processing on the Chinese consultation question of the first government affair consultation case by using a BERT model to obtain all word vectors in the corresponding Chinese consultation question, and then carries out mean value operation on the word vectors to obtain a mean vector as a sentence vector of the corresponding Chinese consultation question i ;​​​​ Step 3.

2. The channel representation unit includes a one-hot encoded embedding layer, which embeds the first... i A case of government consultation Consultation channels One-hot vector encoding is performed, and then word embedding is performed to obtain the first one-hot vector. i A case of government consultation Consultation channels vector representation ; Step 3.

3. The case fusion representation unit will... i A case of government consultation Sentence vectors of Chinese consultation questions Vector of consultation channels Perform vector concatenation to obtain the first... i A case of government consultation vector ; Step 3.

4. Sequence of similar government consultation cases The data is input into the case feature extraction module and processed according to steps 3.1-3.3 to obtain a sequence of similar government consultation cases. Similar vectors ; Step 4. The topic representation module comprises a knowledge representation layer and an attention layer. Obtain each topic word and its relationship representation code from the government affair consultation knowledge base and input them into the knowledge representation layer, and the knowledge representation layer obtains a representation vector corresponding to each topic word by using a trans-E model. The attention layer first... i A case of government consultation Chinese consultation questions Keyword Collection After aligning the length L of the representation vector corresponding to each topic term, we obtain the topic term set. L topic word vectors; in the sentence vector Under supervision, the L topic word vectors are processed using the Attention mechanism to obtain the attention weights of the L topic word vectors. Finally, the L topic word vectors and their corresponding attention weights are weighted and summed to obtain the first... i A case of government consultation Topic Representation Vector ; The attention layer processes the set of topic words in the similar government affair consultation case sequence to obtain a topic representation vector of the similar government affair consultation case sequence ;​ a positive sample pair consisting of a first i government affairs consulting case and a similar government affairs consulting case sequence ​ constructing a negative sample pair consisting of a first i government affair consultation case and a jth government affair consultation case sequence group Step 5. The similarity calculation module comprises two full connection layers, a fusion layer and a Sigmod function module. Step 5.

1. Concatenate the vectors and to obtain the similarity representation vector for the positive pair ; the vector and other jth political affairs consultation case vector After splicing, the non-similarity representation vector of the negative sample pair is obtained ; will be described in detail below. and After dimension reduction processing through two full connection layers respectively, the corresponding scalar , , is obtained. , After normalization processing of the scalar , the text similarity representation of the positive sample pair is obtained. and the text similarity representation of the negative sample pair is obtained. Step 5.

2. Compute the Euclidean distance of the positive pair of samples and the Euclidean distance of the negative pair of samples ; re-compute the topic similarity of the positive pair of samples and the topic similarity of the negative pair of samples ; thereby compute the fused similarity representation of the positive pair of samples and the fused similarity representation of the negative pair of samples ; wherein, denotes the weight, and ; Step 5.

3. Input the fusion similarity representation into the Sigmod function module to obtain the corresponding similarity prediction result and respectively and ; Step 6. Train the case similarity prediction model. Step 6.1 Constructing the loss function with formula (1) : (1) In formula (1), represents the loss function of the positive sample pair, and is obtained by formula (2); represents the loss function of the negative sample pair, and is obtained by formula (3); (2) (3) In formula (2) and formula (3), σ(x) is a Sigmoid function, is a true label value of a sample pair, if it is a positive sample pair, then is 1, if it is a negative sample pair, then is 0; Step 6.2 training the case similarity prediction model by using gradient descent method and calculating the loss function , until the loss function converges or reaches the maximum number of times, stopping the training, thereby obtaining the optimal case similarity prediction model of the fused knowledge representation; Step 6.3 When acquiring new government consultation cases At that time, the case feature extraction module and topic representation module, which have been trained, are first processed to obtain the case. vector and its topic representation vector and with and The data are input together into the trained similarity prediction module for processing, and the similarity prediction results are output. ,like If the value is 1, then the new government consultation case will be... Fusion similarity representation Center front s The highest value is used as a similar case to push information.

2. An electronic device comprising a memory and a processor, characterized in that The memory is used for storing a program supporting the processor to execute the case similarity prediction method of the fusion knowledge representation model in claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is run by the processor to execute the steps of the case similarity prediction method of the fusion knowledge representation model in claim 1.

Images