Government affair service guide data processing method and system based on domain feature construction

Abstract

The invention provides a government affair service guide data processing method and system based on domain feature construction, and relates to the technical field of data processing, and the method comprises the steps: obtaining multi-domain text data, and carrying out the preprocessing; extracting domain feature words in the preprocessed text data, obtaining interaction between the domain feature words and the text data, and calculating an average distance between each character in the text data and each character of the domain feature words; function mapping is carried out on the calculated average distance result, and text feature representation containing domain features is constructed; inputting the text feature representation into an attention mechanism, and enhancing the attention of the text feature representation containing the domain features to obtain final domain feature features; the method comprises the following steps: extracting text dependency features of multi-domain text data, fusing the text dependency features and domain feature features, and completing prediction output of text categories by using the fused features to obtain a multi-domain text classification result, thereby realizing rapid and automatic classification of government affair service guide data sets.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a government service guidance data processing method and system based on domain feature construction. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In recent years, the government service office has gradually changed from the traditional single service model to the integrated service model, forming an integrated service scenario of "one thing". With the iterative update of the new generation of information technology, the government service office has continuously promoted the improvement of the online platform and realized the integration of online and offline services. Through the service mode of "one-stop service", "one-stop service" and "one-stop service", users can complete part of the business online.

[0004] The government service guidance office makes full use of advanced technologies such as big data, artificial intelligence, and blockchain to promote the digital transformation and intelligent upgrading of government services. For example, through the implementation of application scenarios such as intelligent pre-filling, intelligent pre-examination, intelligent customer service, and intelligent guidance, the degree of automation of business handling and the convenience of user services have been improved. In typical service guidance service practices, the quality of the data set is a decisive factor in the training effect of the artificial intelligence model. Therefore, the construction of a highly available, highly reliable, and highly credible data set has become a key technical link in the field of artificial intelligence in the service guidance office.

[0005] The service guidance data set relies on the basic capabilities, domain knowledge, and generation capabilities of the big model, and is achieved through post-pre-training. The construction of a high-quality service guidance data set is a prerequisite for conducting big model training and improving service efficiency.

[0006] Among existing methods, although automation can be achieved in service areas such as intelligent customer service, there are still problems with low data set quality and insufficient diversity. This may cause the model to perform well in certain scenarios, but poorly in other scenarios, or even fail to correctly identify and handle problems; when traditional machine learning faces complex, multi-step or cross-domain knowledge-required problems, because the model lacks sufficient complex problem samples during the training process, it lacks the necessary reasoning and judgment capabilities when dealing with such problems. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a government service data processing method and system based on domain feature construction, designs a semantic representation method including domain features, identifies the influence of domain feature words in text sequences, and enables the attention mechanism to focus on domain feature words that express the main theme of the data set during operation, thereby learning more obvious category features to provide directional semantic information for subsequent processing of the features.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions: The government service data processing method based on domain feature construction includes: Obtain text data from multiple fields and perform preprocessing; Extracting domain characteristic words from preprocessed multi-domain text data, obtaining the interaction between domain characteristic words and text data, and calculating the average distance between each character in the text data and each character of the domain characteristic words; The calculated average distance result is mapped to a function to construct a text feature representation containing domain characteristics; Input the text feature representation containing domain characteristics into the attention mechanism to enhance the attention of the text feature representation containing domain characteristics and obtain the final domain characteristic features; Extract text-dependent features from text data in multiple fields, fuse the text-dependent features with field-specific features, and use the fused features to complete the prediction output of text categories to obtain multi-field text classification results.

[0009] According to some embodiments, the present disclosure adopts the following technical solutions: Data acquisition module, used to acquire text data in multiple fields and perform preprocessing; The feature extraction module is used to extract domain feature words from the preprocessed multi-domain text data, obtain the interaction between the domain feature words and the text data, and calculate the average distance between each character in the text data and each character of the domain feature words; perform function mapping on the calculated average distance result to construct a text feature representation containing domain features; input the text feature representation containing domain features into the attention mechanism to enhance the attention of the text feature representation containing domain features, and obtain the final domain feature features; Feature fusion module, used to extract text-dependent features of text data in multiple fields and fuse text-dependent features with field-specific features; The text classification module is used to use the fused features to complete the prediction output of text categories and obtain multi-domain text classification results.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions: A computer program product includes a computer program, which, when executed by a processor, implements the government service data processing method based on domain feature construction.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions: A non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the government service data processing method based on domain feature construction is implemented.

[0012] According to some embodiments, the present disclosure adopts the following technical solutions: An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the government service data processing method constructed based on domain features.

[0013] Compared with the prior art, the present invention has the following beneficial effects: This paper discloses a method for processing government service data based on domain feature construction, proposes a FastText algorithm model LAFT (Lstm Attention FastText, LAFT) based on LSTM and attention mechanism improvement, proposes a construction method containing domain-specific features and an improvement method for the model, enriches the text semantic information obtained by the model, and strengthens the model's attention to long-term dependencies and key words. Compared with algorithms based on bag-of-words models such as FastText, the LAFT model can more deeply understand the structure and meaning of the text and more accurately extract the main idea of ​​the text.

[0014] The present invention discloses a government service data processing method based on domain feature construction, innovatively designs a semantic representation method including domain features, identifies the influence of domain feature words in text sequences, and provides directional semantic information for subsequent processing of features.

[0015] The present invention discloses a method for processing government service data based on domain feature construction, which integrates domain attention into the attention mechanism, allowing the attention mechanism to focus on domain vocabulary that expresses the main theme of the data set during operation, thereby learning more obvious category features.

[0016] The disclosed method for processing government service guide data based on domain feature construction improves the LSTM model structure, uses LSTM to obtain the main idea of ​​the text, uses the attention mechanism to obtain the local key information of the text, and designs a key word calculation method for different key words in different fields to obtain the degree of influence of each field characteristic word in the text on the text. By improving the text representation form of the model input, enriching the semantic information obtained by the model, strengthening the model's attention to short-distance dependencies and key words, and completing the processing of the government service guide data set, a residual network and a series of learnable parameters are added, so that LSTM has the ability to recognize global main idea semantics while mining long-term features. And innovatively, the domain attention representation and the long-term representation of the improved LSTM are fused and input into the improved FastText model, so that the improved FastText model itself has the ability to mine local features of the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0018] Figure 1 This is a flow chart of a method for processing government service data based on domain feature construction according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0021] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0022] Example 1 The present invention discloses a method for processing government service guidance data based on domain feature construction, and the method steps include: Step 1: Obtain text data from multiple fields and perform preprocessing; Step 2: extracting domain feature words from the preprocessed multi-domain text data, obtaining the interaction between the domain feature words and the text data, and calculating the average distance between each character in the text data and each character of the domain feature words; Step 3: Perform function mapping on the calculated average distance result to construct a text feature representation containing domain characteristics; Step 4: Input the text feature representation containing domain characteristics into the attention mechanism to enhance the attention of the text feature representation containing domain characteristics and obtain the final domain characteristic features; Step 5: Extract text-dependent features from text data in multiple fields, fuse the text-dependent features with the field-specific features, and use the fused features to complete the prediction output of the text category to obtain the multi-field text classification results.

[0023] As an embodiment, the present invention discloses a method for processing government service data based on domain feature construction, by proposing a FastText algorithm model based on LSTM and attention mechanism improvement - LAFT, designing a text semantic representation method containing domain features, identifying the influence of domain feature words in text sequences, and providing directional semantic information for subsequent processing of features. The specific implementation process of the disclosed method is as follows: Step 1: Obtain text data from multiple fields and perform preprocessing; Specifically, we obtain text data in multiple fields such as education, medical treatment, and birth from relevant websites and construct a data set; Furthermore, the preprocessing process includes: The Python regular expression tool library (re) is used to preprocess text data in multiple fields. Except for Chinese and English characters, Arabic numerals, and regular punctuation marks, other characters are deleted, and special characters such as line breaks and manual line breaks are replaced with periods. The purpose of data preprocessing is to remove non-critical characters, reduce data noise, maintain data consistency and neatness, and make the processing results easier to understand and use.

[0024] Then, the preprocessed text data is labeled. In order to improve the training and prediction accuracy of the model, 10% of the data set is selected as the training set and manually labeled, and the other 90% of the data is the data to be labeled. For the training set, according to the key points of the text description, the training set is labeled according to category labels such as "household registration information", "residence permit", "medical insurance card", "outpatient clinic", "birth medical certificate", and "household registration". Manual labeling can ensure the accuracy and consistency of the labels and reduce the prediction bias caused by label errors or ambiguity. In addition, using a smaller training set can significantly reduce the training time of the model and reduce the demand for computing resources.

[0025] Step 2: extract domain feature words from the preprocessed multi-domain text data, obtain the interaction between the domain feature words and the text data, and calculate the average distance between each character in the text data and each character of the domain feature words; Specifically, domain feature words (name, former name, gender, date of birth, place of birth, ethnicity, political status, ID number, native place, religious belief, actual residential address, blood type, height, marital status, spouse's name, whether or not to serve in the military, etc.) are obtained from the preprocessed multi-domain text data. In order to make the domain feature words gain better attention in the model, a position feature mining method is designed to extract features from multi-domain text data, so as to obtain the interaction between domain feature words and text, that is, to calculate the average distance between each character in the text and each character of the domain feature word d :

[0026] in, Indicates the number of domain-specific words in the text data. Domain-specific words are sorted based on expert experience. Indicates Domain-specific word characters, Indicates the length of the text. Indicates the first characters.

[0027] Step 3: Perform function mapping on the calculated average distance result to construct a text feature representation containing domain characteristics; Specifically, the sigmoid function is used to map the distance to the [0,1] interval to construct a text representation that covers the characteristics of the field:

[0028] in, d It is the average distance between each character in the text data and each character of the domain-specific word.

[0029] Step 4: Input the text feature representation containing domain characteristics into the attention mechanism to enhance the attention of the text feature representation containing domain characteristics and obtain the final domain characteristic features; Specifically, domain-specific words are integrated into the attention mechanism to complete the key expression of domain-specific words, and the text feature representation containing domain characteristics is input into the attention mechanism. Among them, the traditional attention mechanism is:

[0030] The present invention integrates the text feature representation containing domain characteristics into the attention mechanism, and obtains the final expression formula of domain characteristic features as follows:

[0031] in, Indicates a query, Indicates the key, Indicates the value, Represents the scaling factor, which is used to adjust the dot product result. represents transpose, DS Represents text features that contain domain characteristics.

[0032] The present invention integrates the text feature representation containing domain characteristics into the attention mechanism, which effectively enhances the model's attention to domain feature words. Integrating domain feature words into the attention mechanism reflects the domain characteristics and contextual information of the text, making the text representation richer and more comprehensive.

[0033] Step 5: Extract text-dependent features from text data in multiple fields, fuse the text-dependent features with the field-specific features, and use the fused features to complete the prediction output of the text category to obtain the multi-field text classification results.

[0034] Specifically, use Sogou's open-source pre-trained Chinese word vector model sgns.sogou.char to construct word vectors for the training set characters and obtain the input of the LSTM layer in the LAFT model. Specifically, use the Python statement open(". / sgns.sogou.char","r",encoding='UTF-8') to load the pre-trained Chinese word vector model; use embeddings[idx]=np.asarray(emb, dtype='float32') to load the character vectors present in the collected data, where the variable idx is a single character in the collected data, and emb is the Sogou vector model representation of a single character. Using a pre-trained word vector model that has captured the semantic relationship and contextual information between words to obtain word vectors can enable the model to have a certain semantic understanding ability in the early stages of training and accelerate the convergence of the model.

[0035] By using word embedding to extract text dependency features, the present invention first improves the output gate of the LSTM layer to capture the text dependency features, that is, to extract the main information of the text.

[0036] The first step is to use the forget gate to remove redundant information that interferes with the main idea of ​​the text:

[0037] in, is the output of the forget gate, is the sigmoid function, is the weight matrix of the forget gate, is the bias term, is the output state of the previous time step, is the input for the current time step.

[0038] In the second step, the input gate is used to recall other valid semantic information required for the text: a sigmoid layer determines which i t The value needs to be updated.

[0039] A tanh layer looks for a new candidate value vector , may be added to the state, .

[0040] in, , is a weight matrix used to map the context to the corresponding space, is a bias vector used to The activation function provides additional input conditioning, is a bias vector corresponding to Wi, used to adjust The calculated linear transformation result.

[0041] The third step is to update the cell state of LSTM so that LSTM can continuously learn the main idea representation of the text.

[0042] in, is the new cell state, is the old cell state and * indicates element-wise multiplication.

[0043] In the fourth step, the original output gate formula is:

[0044] This paper discloses the latest theme of output text, which part of the cell state will be output through the sigmoid layer. The output layer is improved using the residual network, and the improved output gate formula is:

[0045] in, and It is a learnable parameter for deep learning, and the adaptive model changes with each model iteration. For the input branch. is a weight matrix that is used to transform the hidden state of the previous time step and the input of the current time step The concatenated vectors are linearly transformed. is a bias vector that provides a fixed offset.

[0046] Finally, the disclosure passes the cell state through the tanh function (normalizing the value to between -1 and 1), and multiplies it with the output of the improved output gate to obtain the final text-dependent feature.

[0047] The traditional LSTM model effectively captures the long-term dependencies in the text through its unique gating mechanism, and through the residual structure innovatively introduced by the present invention, the present invention can capture the main information and dependency features of the text, significantly improving the model's ability to understand and process the text content.

[0048] Furthermore, the obtained domain-specific features and text-dependent features are fused, specifically:

[0049] in, Indicates The vector representation of a text, Represents matrix multiplication. Represents the text dependency features of LSTM output.

[0050] The domain-specific features and text-dependent features of the present disclosure are two important aspects of the text processing of the present disclosure. The domain-specific features can reflect the uniqueness and professionalism of the field to which the text belongs, while the text-dependent features can capture the semantic relationship and contextual information within the text. The fusion of these two features can make full use of their respective advantages and improve the performance of the model in processing specific field text tasks.

[0051] Furthermore, based on STEP5 After completing the prediction of the text category, the final output of the LAFT model is the text classification result, which is:

[0052] in, is the input document, is the category label, is the fasttext weight, is the fasttext bias, is the total number of categories.

[0053] The disclosure maps text vectors to category space through linear transformation of FastText weights and biases, and can capture words that have not appeared in the training set through n-grams in model prediction, thereby accurately predicting the prediction probability of each category. At the same time, by adjusting model parameters and training data, FastText has the ability to handle cross-domain categories such as the disclosure.

[0054] The present disclosure uses the above-mentioned trained improved fasttext model to complete the prediction work of the test set consisting of 90% of the data set. First, the data preprocessing operation of the test set is completed according to the steps of data collection and data preprocessing. Then, the model training method is used to predict the category of each text in the test set, and then the construction of the government service guide data set is completed. Automatic classification of the government service guide data set can effectively reduce data development costs, improve data development efficiency, and improve data set quality, meeting the needs of high-quality government service guide data sets.

[0055] Example 2 In one embodiment of the present disclosure, a government service data processing system based on domain feature construction is provided, including: Data acquisition module, used to acquire text data in multiple fields and perform preprocessing; The feature extraction module is used to extract domain feature words from the preprocessed multi-domain text data, obtain the interaction between the domain feature words and the text data, and calculate the average distance between each character in the text data and each character of the domain feature words; perform function mapping on the calculated average distance result to construct a text feature representation containing domain features; input the text feature representation containing domain features into the attention mechanism to enhance the attention of the text feature representation containing domain features, and obtain the final domain feature features; Feature fusion module, used to extract text-dependent features of text data in multiple fields and fuse text-dependent features with field-specific features; The text classification module is used to use the fused features to complete the prediction output of text categories and obtain multi-domain text classification results.

[0056] Example 3 In one embodiment of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the government service data processing method constructed based on domain features.

[0057] Example 4 In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the government service guidance data processing method constructed based on domain features is implemented.

[0058] Example 5 In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the government service guidance data processing method constructed based on domain features.

[0059] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0061] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A government service data processing method based on domain feature construction, characterized in that: include: Obtain text data from multiple fields and perform preprocessing; Extracting domain characteristic words from preprocessed multi-domain text data, obtaining the interaction between domain characteristic words and text data, and calculating the average distance between each character in the text data and each character of the domain characteristic words; The calculated average distance result is mapped to a function to construct a text feature representation containing domain characteristics; Input the text feature representation containing domain characteristics into the attention mechanism to enhance the attention of the text feature representation containing domain characteristics and obtain the final domain characteristic features; Extract text-dependent features from text data in multiple fields, fuse the text-dependent features with field-specific features, and use the fused features to complete the prediction output of text categories to obtain multi-field text classification results.

2. The government service data processing method based on domain feature construction as claimed in claim 1 is characterized in that: Obtain text data from multiple fields and use the regular expression tool library to preprocess the text data from multiple fields. Delete all characters except Chinese and English characters, Arabic numerals, and regular punctuation marks, and replace line breaks, manual line breaks, and other special characters with periods.

3. The government service data processing method based on domain feature construction as claimed in claim 1 is characterized in that: The FastText algorithm model LAFT based on LSTM and attention mechanism is used to extract features from text data in multiple fields. After entering LAFT, the position features of domain feature words are mined to obtain the interaction between domain feature words and text data. The average distance between each character in the text data and each character of the domain feature word is calculated as follows: in, Indicates the number of domain feature words in the text. Domain feature words are sorted based on expert experience. Indicates Domain-specific word characters, Indicates the length of the text. Represents the first characters.

4. The government service guide data processing method based on domain feature construction as claimed in claim 1 is characterized in that: The calculated average distance result is subjected to function mapping to construct a text feature representation containing domain characteristics, including: using the sigmoid function to map the calculated average distance to the [0,1] interval, and constructing a text feature representation containing domain characteristics as follows: in, d It is the average distance between each character in the text data and each character of the domain-specific word.

5. The government service guide data processing method based on domain feature construction as claimed in claim 1 is characterized in that: The text feature representation containing domain characteristics is input into the attention mechanism to enhance the attention of the text feature representation containing domain characteristics and obtain the final domain characteristic features, which are: in, Indicates a query, Indicates the key, Indicates the value, Represents the scaling factor, which is used to adjust the dot product result. represents transpose, DS Represents text features that contain domain characteristics.

6. The government service guide data processing method based on domain feature construction as claimed in claim 1 is characterized in that: The LSTM layer in the LAFT model is used to extract text dependency features of text data in multiple fields, capture the semantic relationship and context information between words, and fuse the domain-specific features and text dependency features. Specifically: in, Indicates The fusion vector representation of the text data, Represents matrix multiplication; Represents the text dependency features of LSTM output.

7. The government service data processing system constructed based on domain characteristics is characterized by: include: Data acquisition module, used to acquire text data in multiple fields and perform preprocessing; The feature extraction module is used to extract domain feature words from the preprocessed multi-domain text data, obtain the interaction between the domain feature words and the text data, and calculate the average distance between each character in the text data and each character of the domain feature words; perform function mapping on the calculated average distance result to construct a text feature representation containing domain features; input the text feature representation containing domain features into the attention mechanism to enhance the attention of the text feature representation containing domain features, and obtain the final domain feature features; Feature fusion module, used to extract text-dependent features of text data in multiple fields and fuse text-dependent features with field-specific features; The text classification module is used to use the fused features to complete the prediction output of text categories and obtain multi-domain text classification results.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the government service data processing method based on domain feature construction as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the government service data processing method based on domain feature construction as described in any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the government service data processing method based on domain feature construction as described in any one of claims 1-6.

Images