Policy question and answer oriented intention classification method and system

Abstract

The invention provides an intention classification method and system oriented to policy questions and answers, and the method comprises the steps: firstly constructing a large-scale policy question and answer data set, and pre-training a policy domain language model, so as to learn the expression capability of a policy question and answer domain language; then, according to common scenes and modes of policy questions and answers, a template is constructed, question samples are generated, input with prompts is formed through a prompt learning method, the input with prompts is provided for model reference and prompts, and then intention classification is carried out. Compared with a traditional method based on rules or machine learning, the policy question and answer intention classification method has the advantages that the accuracy and generalization ability of policy question and answer intention classification can be remarkably improved, meanwhile, the classification effect is remarkably improved, and therefore the prediction accuracy and generalization performance of the model are improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and natural language processing, and specifically to an intent classification method and system for policy question answering. Background Art

[0002] A policy question and answer system refers to a system that provides users with policy-related question and answer services. Policy texts are usually cumbersome and complex, and difficult for the public to understand. The policy question and answer system can sort out and classify policy content, making it easier for the public to understand and grasp relevant policies. The policy question and answer system can provide the public with fair, objective and transparent policy information, improve the government's credibility and transparency, and enhance the mutual trust between the government and the people.

[0003] However, traditional rule-based methods require manual definition of rules or keyword libraries. Some data related to user intentions may be more than other data, making it difficult to cover various complex user intentions. This will cause the model to be more biased towards these data and ignore other data, and it is prone to loopholes. Machine learning methods require a large amount of labeled data and need to be retrained in different fields. The workload is large and the robustness is insufficient. The policy question-answering system involves a wide range of policy fields and complex policy issues. The classifier may be disturbed in new fields or issues, resulting in a decrease in classification effect. Summary of the invention

[0004] To solve the above problems, the present invention provides the following technical solution: an intention classification method for policy question answering, the intention classification method for policy question answering comprising the following steps:

[0005] Build a large-scale policy question-answering dataset and pre-train the policy domain language model;

[0006] Construct a template, and build training samples according to the constructed template;

[0007] performing intent classification on the training samples by using a prompt learning method;

[0008] Optimize the intent classification effect by adjusting template parameters and model parameters.

[0009] Furthermore, the construction of a large-scale policy question and answer dataset is used to pre-train the policy field language model, including: data preprocessing, collecting policy text data and policy question and answer corpus through the Internet, and pre-processing the policy question and answer data; pre-training model, using large-scale unlabeled policy field text, using autoencoder or masked language model-based pre-training model.

[0010] Furthermore, the pre-trained model includes pre-training on unlabeled data and fine-tuning on labeled data.

[0011] Furthermore, the pre-training model also includes feature selection.

[0012] Furthermore, constructing the template and constructing the training sample according to the constructed template includes:

[0013] Define intent classification labels based on actual business scenarios;

[0014] Design templates based on intent classification tags;

[0015] Generate corresponding question samples based on templates, keywords and question types.

[0016] Furthermore, the intention classification label includes at least one of: processing method, processing conditions, processing time, processing progress, and common problems.

[0017] Furthermore, the training samples are classified for intent through a prompt learning method, including: extracting key information of the training samples; and constructing prompt information to form prompt input.

[0018] Furthermore, the method of performing intent classification on the training samples by using a prompt learning method includes:

[0019] Extract key information from training samples;

[0020] Construct prompt information to form input with prompts.

[0021] Furthermore, it also includes designing templates based on text, keywords and question types.

[0022] An intention classification system for policy question and answer includes a computer device, wherein the computer device includes a processor, a memory, and a large-scale policy question and answer data set stored on the memory and executable by the processor, and is used to implement the steps of the above-mentioned intention classification method for policy question and answer.

[0023] Beneficial Effects

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. Compared with traditional rule-based or machine learning methods, the present invention can significantly improve the accuracy and generalization ability of policy question-answering intent classification by introducing prompt learning. The Transformer-based language model will adaptively process context information according to the input text. Through learning, when predicting the next word, the model can adaptively select appropriate context information, significantly improve the classification effect, and thus improve the prediction accuracy and generalization performance of the model.

[0026] 2. It can reduce the need for labeled data and improve model performance with less labeled data. Models pre-trained through unsupervised learning can fully learn a large amount of language knowledge and rules, thereby improving the model's prediction accuracy. This method is more efficient and flexible than traditional supervised learning and can adapt to different application scenarios and data sets. In the fine-tuning stage, the model can perform supervised learning based on specific tasks and data to further improve the model's prediction accuracy.

[0027] 3. The prompt learning method can effectively improve the adaptability and intelligence of the system, make the policy question-answering system more in line with actual needs, and better improve the prediction accuracy of the model: the model pre-trained by unsupervised learning can fully learn a large amount of language knowledge and rules, thereby improving the prediction accuracy of the model. This method is more efficient and flexible than traditional supervised learning and can adapt to different application scenarios and data sets. In the fine-tuning stage, the model can be supervised according to specific tasks and data to further improve the prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0029] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0030] Figure 2 This is a schematic diagram of the method flow of Example 3 of the present invention. DETAILED DESCRIPTION

[0031] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0033] An intention classification method for policy question and answer involved in an embodiment of the present invention is mainly applied to a computer device. The above-mentioned computer device includes a processor, a memory, and a large-scale policy question and answer data set stored on the memory and executable by the processor, which is used to implement the steps of an intention classification method for policy question and answer in an embodiment of the present invention.

[0034] Embodiment 1:

[0035] An intention classification method for policy question answering includes the following steps:

[0036] First, we construct a large-scale policy question-answering dataset and pre-train the policy domain language model to learn the language expression ability in the policy question-answering field. Then, based on the common scenarios and patterns of policy question-answering, we design and construct a series of templates to generate training samples. Through the prompt learning method, we form inputs with prompts, which are provided to the model for reference and prompts, and then perform intent classification.

[0037] When building a large-scale policy question-answering dataset, data preprocessing is required first. Policy question-answering data preprocessing is to convert unprocessed data into a format acceptable to the model.

[0038] Specifically, policy text data and policy question and answer corpus are collected through the Internet, and the policy question and answer data are preprocessed. Data preprocessing includes vocabulary segmentation, stop word filtering, punctuation removal, number normalization, synonym conversion, etc., to improve the robustness of the model, for example, converting text to lowercase letters, deleting stop words, converting numbers into special tags, etc.

[0039] It should be noted that in the data preparation stage, steps such as data collection, processing and cleaning are required. First, the target task needs to be determined and a suitable corpus needs to be selected based on the target task. Generally speaking, the larger the data and the more diverse the corpus, the better for the pre-trained language model. When processing and cleaning data, the following operations are usually required: removing HTML tags from the text, replacing URL links, emails and phone numbers with unified symbols, etc.

[0040] After the data preprocessing is completed, we start pre-training the model based on the constructed policy question-answering dataset. By using large-scale unlabeled policy field texts, we use autoencoders or pre-training models based on masked language models to perform unsupervised learning of semantic information and pre-train a Transformer-based language model to learn language expression capabilities.

[0041] The language training model includes pre-training with unlabeled data and fine-tuning with labeled data, where:

[0042] Pre-training of unlabeled data Pre-training is an unsupervised process. This step can use an autoencoder or a pre-training model based on a masked language model. The pre-training model can be trained with a large amount of unlabeled text, so that the model can learn the language rules implicit in a large amount of text data.

[0043] It should be noted that the autoencoder is a neural network model used for data denoising and feature extraction, while the masked language model is a model related to natural language processing, in which the masked words need to predict the correct context position.

[0044] Labeled data fine-tuning is based on the pre-trained model, using labeled policy question and answer data for fine-tuning. The parameters in the pre-trained model are fine-tuned in a supervised manner using labeled data, and only the parameters of the semantic information module are retained to obtain a classification model for policy question and answer.

[0045] It should be noted that fine-tuning with labeled data is the process of supervised learning for a specific task based on the pre-trained model. This step uses labeled policy question-answering data for supervised fine-tuning to update the parameters of the pre-trained model to make it more suitable for policy question-answering tasks.

[0046] Embodiment 2:

[0047] On the basis of Example 1, in order to better improve the functionality of the pre-trained model, the pre-trained model also includes feature selection. Feature selection is to select the most representative features from the data for training models or other data classification. In policy question and answer data, key features such as word vectors, syntax trees, syntactic structures, etc. can be selected to perform feature selection on labeled policy question and answer data to extract more representative features.

[0048] Templates constructed and designed based on pre-training can be used to solve downstream natural language processing tasks, such as text classification and named entity recognition. These templates can be fine-tuned to adapt to different downstream tasks, improve the performance and generalization ability of the model, and use the language knowledge and rules learned by the pre-trained language model to design appropriate templates to help the model better understand text features and information, so as to complete downstream tasks more accurately. When designing templates, it is necessary to consider the characteristics of the task and the characteristics of the data in order to better describe the text information and improve the prediction accuracy of the model.

[0049] For example, in a text classification task, a fully connected layer can be added on top of the pre-trained model to map the output of the pre-trained model to the target classification label. In a named entity recognition task, an annotation layer can be added after the output of the pre-trained model to annotate the entity information in the text. The pre-trained model provides us with a powerful framework. By constructing and designing appropriate templates, we can better utilize the language knowledge and rules learned by the pre-trained model and improve the generalization ability and performance of the model.

[0050] In this embodiment, a series of templates are designed and constructed according to common scenarios and patterns of policy questions and answers, and question samples are generated, including: according to common scenarios and patterns of policy questions and answers, a series of templates are designed and constructed to generate question samples; corresponding training samples are generated according to templates, keywords and question types.

[0051] Embodiment 3:

[0052] On the basis of Example 2, among the intent classification tags defined according to actual business scenarios, the intent classification tags involved in the present invention include:

[0053] (1) Handling method: used to answer how to handle a certain service or business, such as online handling, on-site handling, etc.;

[0054] (2) Application conditions: used to answer the conditions that need to be met to apply for a certain service or business, such as age, identity, work unit, etc.;

[0055] (3) Processing time: used to answer the time required to process a certain service or business, such as processing cycle, acceptance time, etc.

[0056] (4) Processing progress: used to answer inquiries about the processing progress of a service or business, such as processing status, review results, etc.

[0057] (5) Frequently Asked Questions: used to answer common questions and solutions about a certain service or business, such as re-application procedures, problem consultation, etc.

[0058] In the construction template, according to the common scenarios and patterns of policy questions and answers, design the template, including text, keywords, question types, etc. For example, for the question "How to apply for a social security card?", you can design the template "How to apply + service name?".

[0059] In generating corresponding training samples according to templates, keywords and question types, the following are examples of constructed templates and generated training samples:

[0060] (1) How to apply for a certain service? For example: How to apply for a social security card? How to apply for housing provident fund?

[0061] (2) What are the application requirements for a particular service? For example: What are the application requirements for a social security card? What are the application requirements for a housing provident fund?

[0062] (3) How long does it take to process a certain service? For example: How long does it take to process a social security card? How long does it take to process a housing provident fund?

[0063] (4) How can I check the processing progress of a certain service? For example: How can I check the processing progress of a social security card? How can I check the processing progress of a housing provident fund?

[0064] (5) What are the common questions about a certain service? For example: What are the common questions about social security cards? What are the common questions about housing provident funds?

[0065] Prompt learning is to generate training samples to form input data with prompt information, and input the input data into the pre-trained model to learn the ability of intent classification. During the input process, the key information in the template is distinguished from the input information by marking special symbols.

[0066] In this embodiment, prompt learning includes: extracting key information for each question sample, such as policy type, policy field, application conditions, etc. These key information can be extracted through text parsing, keyword extraction, entity recognition and other technologies. For example, for "How to apply for unemployment insurance?", the following key information can be extracted:

[0067] Policy type: Unemployment insurance policy;

[0068] Policy areas: Social security;

[0069] Application process: Apply for unemployment insurance benefits.

[0070] Constructing prompt information to form a prompted input, constructing the extracted key information into a form of prompt information, such as inserting special marks or symbols into the question text to form a prompted input.

[0071] Here are some examples:

[0072] For “How to apply for unemployment insurance benefits?”, the following input with prompts can be formed: [Unemployment insurance] [Application process] How to apply for unemployment insurance benefits? .

[0073] In addition to the above methods, in this embodiment, the intent classification method for policy question answering also includes model training and optimization. The model training process adopts a prompt learning method, using input data with prompts for supervised training. The model optimization fine-tunes the model parameters, such as learning rate, batch size, etc., to optimize the intent classification effect. The model parameters are updated through the back propagation algorithm to continuously adjust the model.

[0074] The significance of adjusting template parameters is to optimize the performance of the pre-trained model in specific tasks. By adjusting template parameters, the model can be better adapted to specific data and tasks, and the performance and generalization ability of the model can be improved. The significance of adjusting model parameters is to improve the accuracy and generalization ability of the model by optimizing parameters such as weights and biases. By selecting different optimization methods and adjusting hyperparameters such as learning rate to optimize the model training process, the performance and generalization ability of the model can be improved, the performance of the model can be optimized, and better results can be achieved.

[0075] In this embodiment, adjusting the template parameters and model parameters includes: inputting the test set into the model, evaluating the performance indicators of the model, such as accuracy, recall rate, F1 score, etc. If the model effect is not good, the model and template can be fine-tuned again, such as increasing the number of hidden layers or changing the activation function, etc., to enhance the representation ability of the model. At the same time, you can also consider adjusting the model parameters, such as adjusting the learning rate, using different optimizers or regularization methods, etc., to optimize the model training process and improve the overfitting problem of the model. Finally, test the performance of the model again to see if it meets the requirements, continuously iterate and optimize to achieve good model performance until the model meets the performance requirements.

[0076] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intent classification method for policy question answering, characterized by: The policy question-answering oriented intention classification method comprises the following steps: Build a large-scale policy question-answering dataset and pre-train the policy domain language model; Construct a template, and build training samples according to the constructed template; performing intent classification on the training samples by using a prompt learning method; Optimize the intent classification effect by adjusting template parameters and model parameters.

2. The intent classification method for policy question answering according to claim 1 is characterized by: The large-scale policy question-answering dataset is constructed to pre-train the policy domain language model, including: Data preprocessing: collect policy text data and policy question and answer corpus through the Internet, and preprocess the policy question and answer data; Pre-trained models use large-scale unlabeled policy domain texts and adopt autoencoders or pre-trained models based on masked language models.

3. The intention classification method for policy question answering according to claim 2 is characterized by: The pre-trained model includes pre-training with unlabeled data and fine-tuning with labeled data.

4. The intention classification method for policy question answering according to claim 2 is characterized by: The pre-trained model also includes feature selection.

5. The intention classification method for policy question answering according to claim 1 is characterized by: The constructing template, constructing a training sample according to the constructed template, includes: Define intent classification labels based on actual business scenarios; Design templates based on intent classification tags; Generate corresponding question samples based on templates, keywords and question types.

6. The intention classification method for policy question answering according to claim 1 is characterized by: The intention classification label includes at least one of the following: processing method, processing conditions, processing time, processing progress, and common questions.

7. The intention classification method for policy question answering according to claim 1 is characterized by: The training samples are classified for intent by using a prompt learning method, including: extracting key information of the training samples; and constructing prompt information to form prompt-containing input.

8. The intention classification method for policy question answering according to claim 1 is characterized by: The method of performing intent classification on the training samples by using a prompt learning method comprises: Extract key information from training samples; Construct prompt information to form input with prompts.

9. The intention classification method for policy question answering according to claim 1 is characterized by: Also included is the design of templates based on text, keywords, and question types.

10. An intention classification system for policy question answering, characterized by: It comprises a computer device, which includes a processor, a memory, and a large-scale policy question and answer data set stored on the memory and executable by the processor, and is used to implement the steps of an intent classification method for policy question and answer as described in any one of claims 1-9.

Images