A method and system for processing government documents

Abstract

The present invention relates to the field of natural language processing, and specifically to a method and system for processing government texts, including: obtaining a user's government text and the user's historical residence, determining the user's current region and a first alignment model corresponding to the user's current region, and obtaining a second alignment model based on the correspondence between the historical residence and the user's current region; wherein the second alignment model includes the correspondence between the user's government text and a second feature vector, the first feature vector and the second feature vector respectively have a correspondence with a government business, obtaining a first feature vector based on the first alignment model and the government text, obtaining a second feature vector based on the second alignment model and the government text, and determining a target government business based on the first feature vector and the second feature vector. The alignment model eliminates misunderstandings of the same business due to regional differences and quickly determines the government business handled by the user.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for processing government texts. Background Art

[0002] At present, the smallest service object of government services is individuals, but individuals are highly mobile. When individuals move across regions, due to differences in government language systems and work norms in various places, there are information docking deviations for essentially the same affairs for the same object in cross-regional and cross-time scenarios, which affects the consistency and efficiency of services. For example, in some southern cities, the term "residence" focuses more on "long-term stay" rather than "settlement", while northern cities understand it more as "living here for a long time and having a residence foundation". This difference in understanding may cause residents to misunderstand when applying for residence permits, thinking that they only need to stay temporarily, rather than settling down for a long time according to local actual requirements. Therefore, when the government system provides services, it is easy to cause differences in the handling of the same business due to regional differences. Summary of the invention

[0003] In order to solve the above-mentioned problems in the prior art, the present invention provides a method and system for processing government documents, which solves the problem of misunderstanding of the same event or business due to regional differences in the prior art.

[0004] According to one aspect of the present application, a method for processing government documents is provided, comprising:

[0005] Obtain the user's government documents and the user's historical residence;

[0006] Determine the region where the user is currently located and a first alignment model corresponding to the region where the user is currently located; wherein the first alignment model includes a correspondence between the user's government affairs text and the first feature vector;

[0007] According to the correspondence between the historical residence and the region where the user is currently located, a second alignment model is obtained; wherein the second alignment model includes the correspondence between the user's government affairs text and the second feature vector, and the first feature vector and the second feature vector have a correspondence with the government affairs business respectively;

[0008] Obtaining a first feature vector according to the first alignment model and the government affairs text;

[0009] Obtaining a second feature vector according to the second alignment model and the government affairs text;

[0010] A target government affairs business is determined according to the first feature vector and the second feature vector.

[0011] In one embodiment, the method for constructing the second alignment model includes:

[0012] Based on the convolutional neural network, obtaining a first reference feature vector corresponding to a first reference government affairs text of the first region;

[0013] Based on the convolutional neural network, a second reference feature vector corresponding to a second reference government text of a second region is obtained; wherein the first reference government text and the second reference government text describe the same government business, and the first region and the second region are different regions;

[0014] Acquire a first loss value according to the first reference feature vector and the second reference feature vector;

[0015] Based on the first loss value, gradient backpropagation is performed on the parameters of the convolutional neural network and the convolutional neural network is updated; wherein the first updated convolutional neural network is the second alignment model.

[0016] In one embodiment, the method for constructing the first alignment model includes:

[0017] Obtaining a third reference government affairs text of the user for the same government affairs service;

[0018] Acquiring high-level semantic information according to the third reference government affairs text;

[0019] Acquire an analysis result according to the high-order semantic information and the second alignment model;

[0020] Obtaining a second loss value according to the analysis result and the high-order semantic information;

[0021] Based on the second loss value, gradient backpropagation is performed on the parameters of the convolutional neural network and the parameters of the convolutional neural network are updated; wherein the second updated convolutional neural network is the first alignment model.

[0022] In one embodiment, when there are multiple third reference government affairs texts, acquiring high-order semantic information according to the third reference government affairs texts includes:

[0023] Based on multiple third reference government affairs texts, construct a reference graph set; wherein the reference graph set includes multiple subgraphs;

[0024] Selecting a subgraph from the reference graph set as an aggregation center;

[0025] Selecting other subgraphs other than the subgraph from the reference graph set as potential neighbors; wherein the potential neighbor indicates that the probability of association between the subgraph and the other subgraphs is greater than a preset probability threshold;

[0026] Based on a multi-layer perceptron, high-order semantic information is obtained according to the potential neighbors and the aggregation center.

[0027] In one embodiment, constructing a reference graph set based on multiple third reference government documents includes:

[0028] Convert each third reference government affairs text into a sequence vector; wherein the sequence vector includes a plurality of word vectors;

[0029] Determining trigger words in the multiple word vectors;

[0030] Based on the conditional random field, obtain the relevance score between each word vector and the trigger word;

[0031] The word vector whose relevance score is greater than a first preset score threshold is used as a target trigger word;

[0032] The word vector whose relevance score is greater than a first preset score threshold and less than a second preset score threshold is used as a target non-trigger word; wherein the second preset score threshold is less than the first preset score threshold;

[0033] A reference graph is constructed according to the target trigger word and the target non-trigger word to obtain a graph set.

[0034] In one embodiment, obtaining the relevance score between each word vector and the trigger word based on the conditional random field includes:

[0035] Based on a bidirectional long short-term memory network, obtaining a recognition result according to the multiple word vectors;

[0036] Based on the conditional random field and according to the recognition result, a relevance score between each word vector and the trigger word is obtained.

[0037] In one embodiment, the acquiring of high-order semantic information based on the potential neighbors and the aggregation center based on a multi-layer perceptron includes:

[0038] Taking government affairs as labels, construct an adjacency matrix according to the multiple subgraphs; wherein the adjacency matrix includes multiple matrix elements, the row and column of each matrix element are respectively the serial numbers of the subgraphs in the multiple subgraphs, and the matrix value of the matrix element is the association weight between the subgraph in the row and the subgraph in the column at the corresponding position of the matrix element;

[0039] Get the weight corresponding to each potential neighbor;

[0040] Obtain the weighted word vector based on the word vector of each potential neighbor and its corresponding weight;

[0041] Vertically concatenate the weighted word vector with other weighted word vectors to obtain a concatenated first vector;

[0042] Based on a multi-layer perceptron, high-order semantic information is obtained according to the concatenated first vector and the word vector of the aggregation center.

[0043] In one embodiment, the acquiring high-order semantic information based on the concatenated first vector and the word vector of the aggregation center based on a multi-layer perceptron includes:

[0044] Based on a multi-layer perceptron, obtaining a third eigenvector according to the concatenated first vector;

[0045] horizontally concatenating the third feature vector and the word vector of the aggregation center to obtain a concatenated second vector;

[0046] Based on the convolutional neural network and the concatenated second vector, obtaining a fourth eigenvector;

[0047] Based on the fourth eigenvector, obtaining a classification result;

[0048] Calculate a third loss value between each label and the classification result;

[0049] Calculating the partial derivative of the third loss value with respect to the weight;

[0050] If the partial derivative of the weight is negative, the potential neighbor corresponding to the weight is removed;

[0051] Determine the matrix value corresponding to the potential neighbor after removal;

[0052] Setting the matrix values ​​of the potential neighbors and the matrix values ​​of the positions of the potential neighbors corresponding to the aggregation center to 0 to update the adjacency matrix and the aggregation center;

[0053] Based on the updated adjacency matrix and the updated aggregation center, high-order semantic information is obtained.

[0054] In one embodiment, acquiring high-order semantic information based on the updated adjacency matrix and the updated aggregation center includes:

[0055] Determine that the subgraph corresponding to the matrix value of 1 in the updated adjacency matrix is ​​the target subgraph;

[0056] Based on the convolutional neural network, high-order semantic information is obtained according to the target subgraph, the adjacent subgraphs to the target subgraph, and the updated aggregation center.

[0057] According to another aspect of the present application, a system for processing government documents is provided, comprising:

[0058] The acquisition module is used to obtain the user's government affairs text and the user's historical residence;

[0059] A determination module, used to determine the region where the user is currently located and a first alignment model corresponding to the region where the user is currently located; wherein the first alignment model includes a correspondence between the user's government affairs text and a first feature vector; a second alignment model is obtained according to the correspondence between the historical residence and the region where the user is currently located; wherein the second alignment model includes a correspondence between the user's government affairs text and a second feature vector, and the first feature vector and the second feature vector respectively have a correspondence with the government affairs business;

[0060] A feature module, configured to obtain a first feature vector according to the first alignment model and the government text; and obtain a second feature vector according to the second alignment model and the government text;

[0061] A business determination module is used to determine a target government business based on the first feature vector and the second feature vector.

[0062] The beneficial effect of the present invention is that when solving information differences, the present invention determines the relationship between the various elements of the government text in different time descriptions through a graph. Then, the high-level semantic information between multiple graphs is extracted through an adaptive graph convolution method. By extracting high-level semantic information, the true meaning of words and sentences can be accurately understood according to factors such as context, and ambiguity can be eliminated. By aligning government texts in different regions, the differences in understanding of the same government business in different regions are eliminated. Then, by aligning individuals with government business, the differences in understanding of all government business and individual events are eliminated. Finally, after the user uploads the information to the government system, the government system can quickly determine the government business that the user wants to handle based on the user's information. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of the method for processing government documents provided by the present invention.

[0064] Figure 2 A schematic diagram of the relationship between government documents provided by the present invention.

[0065] Figure 3 A schematic diagram of the flow of the graph convolutional neural network provided by the present invention.

[0066] Figure 4 This is a schematic diagram of the structure of the adjacency matrix provided by the present invention.

[0067] Figure 5 A schematic diagram of the flow of a graph convolutional neural network provided by another embodiment of the present invention.

[0068] Figure 6 This is a schematic diagram of the structure of the government document processing system provided by the present invention.

[0069] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0070] Figure numerals: 11, processor; 12, memory; 13, input device; 14, output device. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] Embodiment 1: Figure 1 The figure is a flow chart of the method for processing government documents provided by the present invention. Figure 1 As shown in the figure, the processing methods of government documents include:

[0073] S110: Obtain the user's government affairs text and the user's historical residence.

[0074] In an embodiment of the present invention, when a user handles a business, the user will submit the business handling materials to the government affairs system in the current area, and the materials submitted by the user will be processed by the government affairs system. After the user submits the materials to the government affairs system, these materials are identified on the government affairs system and the user's government affairs text is generated. Since the text descriptions of the government affairs texts for the same government affairs business in different regions are different, which leads to misunderstandings among users, it is also necessary to obtain the user's historical residence, align the historical residence with the government affairs texts for the same government affairs business in the current area, ensure that the descriptions of the same government affairs business in different regions are consistent in the core content, avoid misunderstandings caused by regional differences, and ensure that different terms or expressions in the text descriptions of the same government affairs business in different regions are used to describe the same concept, thereby eliminating the differences in the text descriptions of the government affairs texts for the same government affairs business in different regions.

[0075] S120: Determine the region where the user is currently located and a first alignment model corresponding to the region where the user is currently located, wherein the first alignment model includes a corresponding relationship between the user's government affairs text and the first feature vector.

[0076] S130: Obtain a second alignment model based on the correspondence between the historical residence and the user's current region, wherein the second alignment model includes the correspondence between the user's government affairs text and the second feature vector, and the first feature vector and the second feature vector respectively have a correspondence with the government affairs business.

[0077] In an embodiment of the present invention, the present invention constructs a second alignment model between different regions, and the government affairs texts of the same government affairs business in different regions can be aligned through the second alignment model. Specifically, the construction method of the second alignment model includes:

[0078] S210: Based on a convolutional neural network, obtain a first reference feature vector corresponding to a first reference government affairs text in a first region.

[0079] In an embodiment of the present invention, in order to construct an alignment model for the same government business between different regions, it is necessary to obtain the government texts for the same government business from the first region and the second region respectively, and then align the two government texts to eliminate semantic differences.

[0080] Among them, the first reference government text of the first region can be denoted as P.

[0081] Furthermore, the first reference government text of the first region is input into the RoBERTa-large pre-trained model for embedding processing to obtain the embedded first reference government text. Specifically, for each word in the first reference government text, RoBERTa-large generates a corresponding embedding vector. These word embeddings are context-dependent, that is, the same word may have different representations in different contexts. The embedded first reference government text is then input into the convolutional neural network to obtain the first reference feature vector α.

[0082] Therefore, the RoBERTa-large pre-trained model can generate context-dependent embeddings that capture semantic information and subtle language changes in the text, allowing convolutional neural networks to more accurately understand the complex expressions and contextual relationships in government texts. Convolutional neural networks are good at extracting local features, especially when processing data with spatial structures (such as images). In text processing, CNN can effectively extract local patterns and features in embedded representations, such as phrases, keywords, etc. Among them, RoBERTa-large is a pre-trained language model based on the Transformer architecture, and its main purpose is to generate context-dependent feature vectors (embeddings) through deep learning of text data.

[0083] S220: Based on a convolutional neural network, obtain a second reference feature vector corresponding to a second reference government text of the second region, wherein the first reference government text and the second reference government text describe the same government business, and the first region and the second region are different regions.

[0084] In an embodiment of the present invention, similar to S121, the second reference government text of the second region is input into the RoBERTa-large pre-trained model for embedding processing to obtain the embedded second reference government text. Specifically, for each word unit in the second reference government text, RoBERTa-large will generate a corresponding embedding vector. These word unit embeddings are context-dependent, that is, the same word unit may have different representations in different contexts. The embedded second reference government text is then input into the convolutional neural network to obtain the second reference feature vector β.

[0085] S230: Obtain a first loss value according to the first reference eigenvector and the second reference eigenvector.

[0086] In the embodiment of the present invention, the first reference feature vector is input into the text classification pre-training Bert to obtain a first training result, that is, by inputting the first reference feature vector into the text classification pre-training Bert, the first key feature vector is further extracted. The second reference feature vector is input into the text classification pre-training Bert to obtain the second training result. That is, by inputting the second reference feature vector into the text classification pre-training Bert, the second key feature vector is further extracted. This step does not require softmax output categories.

[0087] Further, the first key eigenvector is calculated and the second key eigenvector The first loss value. The calculation formula of the first loss value is:

[0088] ,in, is the first key eigenvector, is the second key eigenvector.

[0089] Among them, the text classification pre-trained BERT (Bidirectional Encoder Representations from Transformers) is a deep learning model based on the Transformer architecture, which is specifically used to handle natural language processing (NLP) tasks, including text classification.

[0090] Among them, Softmax is a mathematical function that is often used for classification tasks in machine learning and deep learning, especially in multi-class classification problems. Its main function is to convert the elements in a vector into a probability distribution so that all output values ​​are between 0 and 1, and the sum of all output values ​​is 1.

[0091] S240: Based on the first loss value, perform gradient backpropagation on the parameters of the convolutional neural network and update the convolutional neural network; wherein the first updated convolutional neural network is a second alignment model.

[0092] In the embodiment of the present invention, gradient back propagation is performed through the loss value to update the parameters of the convolutional neural network (CNN), and the model finally trained is called the second alignment model. This process involves using forward propagation to calculate the output and loss value, and then calculating the gradient through back propagation and updating the network parameters, thereby optimizing the performance of the model.

[0093] Therefore, the correspondence between the first region and the second region and the correspondence between the second alignment model, when two regions are determined, the second alignment model for the government affairs of the two regions can be obtained accordingly. For example, the historical residence is Beijing, and the user's current region is Shanghai. Beijing and Shanghai build a correspondence relationship, and through this correspondence, the second alignment model corresponding to the correspondence between Beijing and Shanghai is obtained.

[0094] In addition, the second alignment model in the present invention can only align the government affairs of two regions.

[0095] Among them, gradient backpropagation is the core algorithm used to calculate gradients when training neural networks. It is based on the chain rule and updates various parameters in the network (such as weights and biases) through the gradient of the loss function. In convolutional neural networks (CNNs), the purpose of gradient backpropagation is to adjust network parameters to improve the prediction performance of the model by backpropagating loss information.

[0096] S140: Obtain a first feature vector according to the first alignment model and the government text.

[0097] S150: Obtain a second feature vector according to the second alignment model and the government text.

[0098] S160: Determine a target government affairs business according to the first eigenvector and the second eigenvector.

[0099] In the embodiment of the present invention, in order to determine the government affairs service that the user wants to handle, the similarity between the first feature vector and the second feature vector can be calculated, and the government affairs service corresponding to the second feature vector or the first feature vector with a plurality of similarities greater than the preset similarity is taken as the target government affairs service, and the target government affairs service is the government affairs service that the user wants to handle. It can be understood that the present invention selects the government affairs service corresponding to the first feature vector or the second feature vector with the highest similarity match as the target government affairs service. Among them, the similarity can be calculated using cosine similarity.

[0100] When solving information differences, the present invention determines the relationship between the various elements of the government text in different time descriptions through a graph. Then, the high-level semantic information between multiple graphs is extracted through an adaptive graph convolution method. By extracting high-level semantic information, the true meaning of words and sentences can be accurately understood based on factors such as context, and ambiguity can be eliminated. By aligning government texts from different regions, differences in understanding of the same government business in different regions are eliminated. Then, by aligning individuals with government business, differences in understanding of all government business and individual events are eliminated. Finally, after the user uploads the information to the government system, the government system can quickly determine the government business the user wants to handle based on the user's information.

[0101] Example 2: In one embodiment, the method for constructing the first alignment model can be specifically implemented as follows: obtaining a third reference government text for the same government business of the user; obtaining high-order semantic information based on the third reference government text; obtaining analysis results based on the high-order semantic information and the second alignment model; obtaining a second loss value based on the analysis results and the high-order semantic information; based on the second loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the parameters of the convolutional neural network; wherein the second updated convolutional neural network is the first alignment model.

[0102] In the embodiment of the present invention, the alignment between the user and the government affairs of the government affairs system in the present invention needs to rely on the second alignment model. Because the second alignment model can extract high-order semantic information (i.e., analysis results) from the individual events corresponding to the user and the official events of the government affairs of the government affairs system. This means that the model can understand and capture the deep meaning, context, and related semantic relationships of the event, so that the individual event can be effectively matched with the official event. Then, based on the third reference government affairs text, the high-order semantic information is obtained. . Input into the second alignment model to obtain the second alignment model for high-order semantic information Analysis results of the government affairs text of the corresponding government affairs business .

[0103] Then, the second loss value between the analysis result and the high-order semantic information is calculated. The calculation formula of the second loss value is: ,in, is high-level semantic information, To analyze the results.

[0104] Finally, based on the second loss value, the parameters of the convolutional neural network are gradient back-propagated and updated, and the second updated convolutional neural network is the first alignment model.

[0105] Example 3: In one embodiment, when there are multiple third reference government texts, the method for constructing the first alignment model can be specifically implemented as follows: construct a reference graph set based on multiple third reference government texts; wherein the reference graph set includes multiple subgraphs; select a subgraph from the reference graph set as an aggregation center; select other subgraphs except the subgraph from the reference graph set as potential neighbors; based on a multi-layer perceptron, obtain high-order semantic information according to potential neighbors and aggregation centers.

[0106] In the embodiment of the present invention, a user may have multiple government documents for the same government service, and one government document can generate a graph G. Then, when there are multiple government documents corresponding to the government service, multiple graphs will be generated. That is to say, when there are multiple third reference government documents, there are multiple corresponding graphs, and these graphs are constructed into a graph set as follows: ,in, Represents the Mth subgraph. The relationship between the elements in these subgraphs, that is, the high-level semantic information of each element, is extracted through a graph convolutional neural network. However, since the subgraphs are not connected to each other, the traditional graph convolutional neural network is not applicable to this scenario (the traditional graph convolutional neural network requires a clear adjacency matrix). In order to solve this problem, the present invention adopts a random aggregation method.

[0107] Specifically, from Randomly select a subgraph from As the aggregation center, randomly select k subgraphs other than (other subgraphs)as potential neighbors, where potential neighbors represent There may be associations with k subgraphs, i.e., subgraphs The probability of association with k subgraphs is greater than a preset probability threshold, which may be 1%. Then, based on a multi-layer perceptron, high-order semantic information is obtained according to potential neighbors and aggregation centers.

[0108] Example 4: In one embodiment, the method for constructing the first alignment model can be specifically implemented as follows: converting each third reference government text into a sequence vector; wherein the sequence vector includes multiple word vectors; determining trigger words among the multiple word vectors; based on conditional random fields, obtaining the correlation score between each word vector and the trigger word; using the word vector whose correlation score is greater than a first preset score threshold as the target trigger word; using the word vector whose correlation score is greater than the first preset score threshold and less than the second preset score threshold as the target non-trigger word; wherein the second preset score threshold is less than the first preset score threshold; constructing a reference graph based on the target trigger word and the target non-trigger word to obtain a graph set.

[0109] In the embodiment of the present invention, when resolving information differences, it is necessary to first determine the relationship between the elements in different time descriptions, so it is necessary to construct the correlation between the elements in the official events corresponding to the government affairs. First, obtain the user's third reference government affairs text for the same government affairs. Input the third reference government affairs text into the RoBERTa-large pre-training model to obtain the sequence vector , Represents the nth word vector. Each word vector has an attribute corresponding to it, which indicates whether the word vector is a trigger word or a non-trigger word.

[0110] like Figure 2 As shown, for example, the government text is "Due to poor management, the company has been losing money every year. Mr. Li applied for bankruptcy procedures on Sunday, December 1, 2024." From this government text, we can get "apply" as a trigger word, "bankruptcy procedures" and "Mr. Li" as non-trigger words. Trigger words are verbs, and non-trigger words are nouns.

[0111] Furthermore, the word vector with attributes is input into the conditional random field (CRF) to obtain the correlation score between each word vector and the trigger word. Among them, the trigger word has the highest correlation with itself, so the score of the trigger word itself is the highest, and the scores of the other non-trigger words are positively correlated with the correlation. The present invention believes that the core of the event description is the active and passive relationship of Chen Qing, and the rest can be regarded as the modification part of the event. Therefore, when constructing the graph, only the top three word vectors are taken, and the first word vector is used as the target trigger word. , the second and third word vectors are used as non-trigger words , non-trigger words Event trigger words There is a pointing relationship between , Set to 1, then the graph G is constructed. , L is the trigger word set, V is the non-trigger word set, For trigger words, is a non-trigger word, and the elements in E are represented as , Indicates non-trigger words Event trigger words There is a pointing relationship between them. , .

[0112] Example 5: In one embodiment, the method for constructing the first alignment model can be specifically implemented as follows: based on a bidirectional long short-term memory network, according to multiple word vectors, obtaining recognition results; based on a conditional random field, according to the recognition results, obtaining the correlation score between each word vector and the trigger word.

[0113] In the embodiment of the present invention, in order to fully understand the semantic role of each word in the sentence, the sequence vector can be input into a bidirectional long short-term memory network (BiLSTM) to obtain a recognition result, wherein the recognition result is:

[0114] ,in, represents the output weight, , Indicates the state update variable, for The word vectors in , represents the output of the forget gate, represents the output of the input gate, represents the weight, is the bias, Indicates the previous input Input the output of BiLSTM.

[0115] Among them, BiLSTM (Bidirectional Long Short-Term Memory) plays several important roles in natural language processing (NLP) tasks.

[0116] Embodiment 6: Figure 3 A schematic diagram of the flow of the graph convolutional neural network provided by the present invention. Figure 4 This is a schematic diagram of the structure of the adjacency matrix provided by the present invention. Figure 3-4, the construction method of the first alignment model can be specifically implemented as follows: using government affairs as labels, constructing an adjacency matrix according to multiple subgraphs; wherein the adjacency matrix includes multiple matrix elements, the rows and columns of each matrix element are the serial numbers of the subgraphs in the multiple subgraphs, respectively, and the matrix value of the matrix element is the association weight between the subgraph in the row and the subgraph in the column at the corresponding position of the matrix element; obtaining the weight corresponding to each potential neighbor; obtaining the weighted word vector according to the word vector of each potential neighbor and its corresponding weight; vertically splicing the weighted word vector with other weighted word vectors to obtain the spliced ​​first vector; based on the multi-layer perceptron, obtaining high-order semantic information according to the spliced ​​first vector and the word vector of the aggregation center.

[0117] In the embodiment of the present invention, since there are many processes in the process of handling government affairs, only when the processes are combined together can the government affairs that the user wants to handle be determined, so it is necessary to extract high-level semantic information, which can accurately help the government affairs system determine the government affairs that the user wants to handle. In order to obtain high-level semantic information, a government affairs business is used as a label to construct a The matrix is ​​a matrix where the rows and columns of each matrix element are the serial numbers of the subgraphs in the multiple subgraphs, and the value of the corresponding position represents the association weight between the two subgraphs. Initially, the weight matrix value is 1 (the weight is calculated using Convolution is used to implement the government affairs business, where the government affairs business includes ID card processing, visa processing, etc. Figure 4 As shown in (1, 1), the row is 1 and the column is 1, so the row of the matrix element is the subgraph with the number 1 in the multiple subgraphs, and the column is the subgraph with the number 1 in the multiple subgraphs. (M, M) is the same as (1, 1), which means that the row of the matrix element is the subgraph with the number M in the multiple subgraphs, and the column is the subgraph with the number M in the multiple subgraphs. Figure 4 As shown, (1, 1) = 1, where the 1 on the right side of the equal sign represents the association weight between the two subgraphs, which changes with subsequent updates.

[0118] Then, according to the word vector of each potential neighbor and its corresponding weight, a weighted word vector is obtained, and the weighted word vector is vertically spliced ​​with other weighted word vectors to obtain the spliced ​​first vector. Based on the multi-layer perceptron, high-order semantic information is obtained according to the spliced ​​first vector and the word vector of the aggregation center.

[0119] Embodiment 7: In one embodiment, the method for constructing the first alignment model can be specifically implemented as follows: based on a multi-layer perceptron, a third eigenvector is obtained according to the spliced ​​first vector; the third eigenvector and the word vector of the aggregation center are horizontally spliced ​​to obtain the spliced ​​second vector; based on a convolutional neural network, a fourth eigenvector is obtained based on the spliced ​​second vector; based on the fourth eigenvector, a classification result is obtained; a third loss value of each label and the classification result is calculated; the partial derivative of the third loss value with respect to the weight is calculated; if the partial derivative of the weight is negative, the potential neighbor corresponding to the weight is removed; the matrix value corresponding to the removed potential neighbor is determined; the matrix value of the potential neighbor and the matrix value of the position corresponding to the potential neighbor and the aggregation center are set to 0 to update the adjacency matrix and the aggregation center; based on the updated adjacency matrix and the updated aggregation center, high-order semantic information is obtained.

[0120] like Figure 3 As shown, the word vector of potential neighbor A contains , , then multiply the word vector of potential neighbor A by weight A and the word vector of potential neighbor B by weight B for vertical splicing and input into the multi-layer perceptron, and output the third feature vector after passing through the multi-layer perceptron , the third eigenvector After horizontal concatenation with the word vector of the aggregation center, it is input into the convolutional neural network to obtain the fourth eigenvector . The fourth eigenvector Input into the pre-trained document classification model Bert to obtain the output vector, and input the output vector into softmax to output the classification result of the text. Calculate the partial derivative of the third loss value with respect to weight A and calculate the partial derivative of the third loss value with respect to weight B. Determine whether the partial derivative is positive or negative. If the partial derivative is negative, it is determined that the weight is related to the government business label corresponding to the aggregation center, and the potential neighbor corresponding to the weight is retained, where the label is government business. For example, if the partial derivative of weight A is negative, the potential neighbor A is retained, and if the partial derivative of weight B is positive, the potential neighbor B is removed. Among them, since government text is related to government business, the government text corresponds to a government business label, and the corresponding graph corresponds to a government business label.

[0121] Furthermore, the matrix values ​​of the potential neighbors and the matrix values ​​of the positions corresponding to the potential neighbors and the aggregation center are set to 0 until the elements of all positions in the adjacency matrix and the aggregation center are judged once, thereby updating the adjacency matrix and the aggregation center.

[0122] Embodiment 8: Figure 5 A flow chart of a graph convolutional neural network provided by another embodiment of the present invention. Figure 5, the construction method of the first alignment model can be specifically implemented as follows: determine that the subgraph corresponding to the matrix value of 1 in the updated adjacency matrix is ​​the target subgraph; based on the convolutional neural network, obtain high-order semantic information according to the target subgraph, the adjacent subgraphs to the target subgraph, and the updated aggregation center.

[0123] In an embodiment of the present invention, the subgraphs corresponding to the matrix value of 1 in the updated adjacency matrix are determined in sequence using the horizontal axis as an index, and then the neighbors corresponding to the target subgraph with a matrix value of 1 are obtained, that is, the adjacent subgraphs to the subgraph. The first weight of the subgraph corresponding to the matrix value of 1 and the second weight corresponding to the adjacent subgraph are determined, the first product between the first weight and the target subgraph is calculated, and the second product between the neighbor subgraph and the second weight is calculated, the third weight of the word vector of the aggregation center is determined, the third product of the updated word vector of the aggregation center and the third weight is calculated, the first product, the second product, and the third product are concatenated, and the concatenated product vector is input into a multi-layer perceptron to obtain the final feature vector, and the final feature vector is input into a convolutional neural network to obtain high-order semantic information. .

[0124] Figure 6 This is a schematic diagram of the structure of the government document processing system provided by the present invention. Figure 6 , an acquisition module is used to obtain the user's government text and the user's historical residence; a determination module is used to determine the user's current region and the first alignment model corresponding to the user's current region; wherein the first alignment model includes the user's government text and the first feature vector having a corresponding relationship; according to the corresponding relationship between the historical residence and the user's current region, a second alignment model is obtained; wherein the second alignment model includes the corresponding relationship between the user's government text and the second feature vector, and the first feature vector and the second feature vector have a corresponding relationship with the government business respectively; a feature module is used to obtain a first feature vector according to the first alignment model and the government text; according to the second alignment model and the government text, a second feature vector is obtained; a business determination module is used to determine the target government business according to the first feature vector and the second feature vector.

[0125] In one embodiment, the first construction module can be specifically configured as follows: based on a convolutional neural network, obtaining a first reference feature vector corresponding to a first reference government text in a first region; based on a convolutional neural network, obtaining a second reference feature vector corresponding to a second reference government text in a second region; wherein the first reference government text and the second reference government text describe the same government business, and the first region and the second region are different regions; obtaining a first loss value based on the first reference feature vector and the second reference feature vector; based on the first loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the convolutional neural network; wherein the first updated convolutional neural network is a second alignment model.

[0126] In one embodiment, the second construction module can be specifically configured as follows: obtaining a third reference government text of the user for the same government business; obtaining high-order semantic information based on the third reference government text; obtaining analysis results based on the high-order semantic information and the second alignment model; obtaining a second loss value based on the analysis results and the high-order semantic information; based on the second loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the parameters of the convolutional neural network; wherein the second updated convolutional neural network is the first alignment model.

[0127] In one embodiment, when there are multiple third reference government texts, the second construction module can be specifically configured as follows: based on the multiple third reference government texts, construct a reference graph set; wherein the reference graph set includes multiple subgraphs; select a subgraph from the reference graph set as an aggregation center; select other subgraphs except the subgraph from the reference graph set as potential neighbors; wherein the potential neighbor indicates that the probability of association between the subgraph and other subgraphs is greater than a preset probability threshold; based on a multi-layer perceptron, obtain high-order semantic information according to the potential neighbors and the aggregation center.

[0128] In one embodiment, the second construction module can be specifically configured as follows: converting each third reference government text into a sequence vector; wherein the sequence vector includes multiple word vectors; determining trigger words among the multiple word vectors; obtaining the correlation score between each word vector and the trigger word based on the conditional random field; using the word vector whose correlation score is greater than the first preset score threshold as the target trigger word; using the word vector whose correlation score is greater than the first preset score threshold and less than the second preset score threshold as the target non-trigger word; wherein the second preset score threshold is less than the first preset score threshold; constructing a reference graph based on the target trigger word and the target non-trigger word to obtain a graph set.

[0129] In one embodiment, the second construction module can be specifically configured as follows: based on a bidirectional long short-term memory network, obtaining recognition results according to multiple word vectors; based on a conditional random field, obtaining a correlation score between each word vector and a trigger word according to the recognition results.

[0130] In one embodiment, the second construction module can be specifically configured as follows: using government affairs as labels, constructing an adjacency matrix based on multiple subgraphs; wherein the adjacency matrix includes multiple matrix elements, the rows and columns of each matrix element are respectively the serial numbers of the subgraphs in the multiple subgraphs, and the matrix value of the matrix element is the association weight between the subgraph in the row and the subgraph in the column at the corresponding position of the matrix element; obtaining the weight corresponding to each potential neighbor; obtaining a weighted word vector based on the word vector of each potential neighbor and its corresponding weight; vertically splicing the weighted word vector with other weighted word vectors to obtain a spliced ​​first vector; based on a multi-layer perceptron, obtaining high-order semantic information based on the spliced ​​first vector and the word vector of the aggregation center.

[0131] In one embodiment, the second construction module can be specifically configured as follows: based on a multilayer perceptron, a third eigenvector is obtained according to the spliced ​​first vector; the third eigenvector and the word vector of the aggregation center are horizontally spliced ​​to obtain the spliced ​​second vector; based on a convolutional neural network, a fourth eigenvector is obtained based on the spliced ​​second vector; based on the fourth eigenvector, a classification result is obtained; a third loss value of each label and the classification result is calculated; the partial derivative of the third loss value with respect to the weight is calculated; if the partial derivative of the weight is negative, the potential neighbor corresponding to the weight is removed; the matrix value corresponding to the removed potential neighbor is determined; the matrix value of the potential neighbor and the matrix value of the position corresponding to the potential neighbor and the aggregation center are set to 0 to update the adjacency matrix and the aggregation center; based on the updated adjacency matrix and the updated aggregation center, high-order semantic information is obtained.

[0132] In one embodiment, the second construction module can be specifically configured as: determining that the subgraph corresponding to the matrix value of 1 in the updated adjacency matrix is ​​the target subgraph; based on the convolutional neural network, obtaining high-order semantic information according to the target subgraph, the adjacent subgraphs to the target subgraph, and the updated aggregation center.

[0133] Figure 7 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0134] like Figure 7 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0135] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0136] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the processing method of government affairs texts of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0137] In one example, the electronic device 10 may further include: an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0138] When the electronic device 10 is a stand-alone device, the input device 13 may be a communication network connector for receiving collected input signals from the first device and the second device.

[0139] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0140] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0141] Of course, to simplify, Figure 7 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0142] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0143] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0144] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for processing government documents, characterized in that: include: Obtain the user's government documents and the user's historical residence; Determine the region where the user is currently located and a first alignment model corresponding to the region where the user is currently located; wherein the first alignment model includes a correspondence between the user's government affairs text and the first feature vector; According to the correspondence between the historical residence and the region where the user is currently located, a second alignment model is obtained; wherein the second alignment model includes the correspondence between the user's government affairs text and the second feature vector, and the first feature vector and the second feature vector have a correspondence with the government affairs business respectively; According to the first alignment model and the government text, a first feature vector is obtained; wherein the method for constructing the first alignment model includes: Obtain the user's third reference government affairs text for the same government affairs service; Acquiring high-level semantic information according to the third reference government affairs text; Acquire an analysis result according to the high-order semantic information and the second alignment model; Obtaining a second loss value according to the analysis result and the high-order semantic information; Based on the second loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the parameters of the convolutional neural network; wherein the second updated convolutional neural network is the first alignment model; According to the second alignment model and the government text, a second feature vector is obtained; wherein the method for constructing the second alignment model includes: based on a convolutional neural network, obtaining a first reference feature vector corresponding to a first reference government text of a first region; Based on the convolutional neural network, a second reference feature vector corresponding to a second reference government text of a second region is obtained; wherein the first reference government text and the second reference government text describe the same government business, and the first region and the second region are different regions; Acquire a first loss value according to the first reference feature vector and the second reference feature vector; Based on the first loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the convolutional neural network; wherein the first updated convolutional neural network is a second alignment model; A target government affairs business is determined according to the first feature vector and the second feature vector.

2. The method for processing government documents according to claim 1, characterized in that: When there are multiple third reference government affairs texts, obtaining high-order semantic information according to the third reference government affairs texts includes: Based on multiple third reference government affairs texts, construct a reference graph set; wherein the reference graph set includes multiple subgraphs; Selecting a subgraph from the reference graph set as an aggregation center; Selecting other subgraphs other than the subgraph from the reference graph set as potential neighbors; wherein the potential neighbor indicates that the probability of association between the subgraph and the other subgraphs is greater than a preset probability threshold; Based on a multi-layer perceptron, high-order semantic information is obtained according to the potential neighbors and the aggregation center.

3. The method for processing government documents according to claim 2, characterized in that: The constructing of a reference graph set based on a plurality of third reference government affairs texts includes: Convert each third reference government affairs text into a sequence vector; wherein the sequence vector includes a plurality of word vectors; Determining trigger words in the multiple word vectors; Based on the conditional random field, obtain the relevance score between each word vector and the trigger word; The word vector whose relevance score is greater than a first preset score threshold is used as a target trigger word; The word vector whose relevance score is greater than a first preset score threshold and less than a second preset score threshold is used as a target non-trigger word; wherein the second preset score threshold is less than the first preset score threshold; A reference graph is constructed according to the target trigger word and the target non-trigger word to obtain a graph set.

4. The method for processing government documents according to claim 3, characterized in that: The step of obtaining the relevance score between each word vector and the trigger word based on the conditional random field includes: Based on a bidirectional long short-term memory network, obtaining a recognition result according to the multiple word vectors; Based on the conditional random field and according to the recognition result, a relevance score between each word vector and the trigger word is obtained.

5. The method for processing government documents according to claim 2, characterized in that: The obtaining of high-order semantic information based on the potential neighbors and the aggregation center based on the multi-layer perceptron includes: Taking government affairs as labels, construct an adjacency matrix according to the multiple subgraphs; wherein the adjacency matrix includes multiple matrix elements, the row and column of each matrix element are respectively the serial numbers of the subgraphs in the multiple subgraphs, and the matrix value of the matrix element is the association weight between the subgraph in the row and the subgraph in the column at the corresponding position of the matrix element; Get the weight corresponding to each potential neighbor; Obtain the weighted word vector based on the word vector of each potential neighbor and its corresponding weight; Vertically concatenate the weighted word vector with other weighted word vectors to obtain a concatenated first vector; Based on a multi-layer perceptron, high-order semantic information is obtained according to the concatenated first vector and the word vector of the aggregation center.

6. The method for processing government documents according to claim 5, characterized in that: The step of obtaining high-order semantic information based on the concatenated first vector and the word vector of the aggregation center based on the multi-layer perceptron includes: Based on a multi-layer perceptron, obtaining a third eigenvector according to the concatenated first vector; horizontally concatenating the third feature vector and the word vector of the aggregation center to obtain a concatenated second vector; Based on the convolutional neural network and the concatenated second vector, obtaining a fourth eigenvector; Based on the fourth eigenvector, obtaining a classification result; Calculate a third loss value between each label and the classification result; Calculating the partial derivative of the third loss value with respect to the weight; If the partial derivative of the weight is negative, the potential neighbor corresponding to the weight is removed; Setting the matrix values ​​of the potential neighbors and the matrix values ​​of the positions of the potential neighbors corresponding to the aggregation center to 0 to update the adjacency matrix and the aggregation center; Based on the updated adjacency matrix and the updated aggregation center, high-order semantic information is obtained.

7. The method for processing government documents according to claim 6, characterized in that: The obtaining of high-order semantic information based on the updated adjacency matrix and the updated aggregation center includes: Determine that the subgraph corresponding to the matrix value of 1 in the updated adjacency matrix is ​​the target subgraph; Based on the convolutional neural network, high-order semantic information is obtained according to the target subgraph, the adjacent subgraphs to the target subgraph, and the updated aggregation center.

8. A government document processing system, characterized in that: include: The acquisition module is used to obtain the user's government affairs text and the user's historical residence; A determination module, used to determine the region where the user is currently located and a first alignment model corresponding to the region where the user is currently located; wherein the first alignment model includes a correspondence between the user's government affairs text and a first feature vector; a second alignment model is obtained according to the correspondence between the historical residence and the region where the user is currently located; wherein the second alignment model includes a correspondence between the user's government affairs text and a second feature vector, and the first feature vector and the second feature vector respectively have a correspondence with the government affairs business; A feature module is used to obtain a first feature vector according to the first alignment model and the government text; and to obtain a second feature vector according to the second alignment model and the government text; wherein the method for constructing the first alignment model includes: Obtain the user's third reference government affairs text for the same government affairs service; Acquiring high-level semantic information according to the third reference government affairs text; Acquire an analysis result according to the high-order semantic information and the second alignment model; Obtaining a second loss value according to the analysis result and the high-order semantic information; Based on the second loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the parameters of the convolutional neural network; wherein the second updated convolutional neural network is the first alignment model; The method for constructing the second alignment model includes: obtaining a first reference feature vector corresponding to a first reference government affairs text of a first region based on a convolutional neural network; Based on the convolutional neural network, a second reference feature vector corresponding to a second reference government text of a second region is obtained; wherein the first reference government text and the second reference government text describe the same government business, and the first region and the second region are different regions; Acquire a first loss value according to the first reference feature vector and the second reference feature vector; Based on the first loss value, performing gradient backpropagation on the parameters of the convolutional neural network and updating the convolutional neural network; wherein the first updated convolutional neural network is a second alignment model; A business determination module is used to determine a target government business based on the first feature vector and the second feature vector.

Images