A fair competition review auxiliary method based on text classification model

Abstract

The present invention discloses a fair competition review assistance method based on a text classification model, including building a government policy text information database, obtaining policy texts issued by the government, performing subclass division, collecting different policy texts, building a pre-trained language model, having experts annotate data of the collected policy texts, identifying the data annotated by experts in the policy texts, performing model training and analysis on the policy texts in the review sample library, preliminarily screening and reviewing the policy texts according to the training and analysis results, counting the fair review results of different policy texts, and iterating information on the constructed model, aiming to solve the problems of low accuracy and lack of semantic understanding of the original similarity model, building a model based on government policy documents to train data in real time, obtaining semantic features of the text through autonomous learning of the text, and using the model to classify and review the government policy documents to be reviewed, assisting experts in making judgments, and reducing the labor cost of the review.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a fair competition review auxiliary method based on a text classification model. Background Art

[0002] If the existing review scheme uses legal personnel to review the policy texts issued by the government one by one, it will be a very time-consuming and laborious task, and manual review is prone to omissions. The existing review system is based on the similarity model for judgment. However, at present, the judgment method using the similarity model has certain defects. The judgment method using the similarity model lacks an understanding of the semantics of the text and only compares the similarity of words. The accuracy of the similarity model is not high. It only compares with the text of the review standard. However, the text expression of many government policies is far from the review standard, making it difficult to make a reasonable judgment.

[0003] This application aims to build a fair competition review assistance technology method to solve the problems of low accuracy and lack of semantic understanding of the original similarity model. The model is built based on government policy documents to train data in real time. Through autonomous learning of the text, the semantic features of the text are obtained. The model is used to classify and review the government policy documents to be reviewed, assist experts in making judgments, and reduce the labor cost of the review. Summary of the Invention

[0004] The purpose of the present invention is to provide a fair competition review auxiliary method based on a text classification model to solve the problems in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A fair competition review auxiliary method based on text classification model:

[0007] S1: Build a government policy text information database, obtain policy texts issued by the government, tag different policy texts with keywords, classify them according to different keywords, and subclassify the classified policy texts according to release time, priority, newly added records and text validity;

[0008] S2: Collect different policy texts, screen them for expert-annotated data, classify them into those with and without expert-annotated data, extract policy texts without expert-annotated data, perform pre-training, and build a pre-trained language model. The pre-trained model structure adopts the BERT model structure.

[0009] S3: Have experts annotate the collected policy texts, identify the expert-annotated data within the policy texts, classify the extracted policy texts according to whether they violate the review standards or not, and enter the classified policy texts into the review sample library according to their categories;

[0010] Assume that there are N review standards within the review sample library, add the option of not violating all review standards, set the review sample library to include N+1 review categories, set a policy text x as the input of the pre-trained language model, set the text length to L, set the category y, convert the input parameters to one-hot vector representation, set the model output dimension to N+1, and set the dimension corresponding to category y to 1. The pre-trained BERT model maps each word of the policy text to a 512-dimensional high-dimensional vector space. The policy text is processed by the BERT model and converted into an L×512 hidden vector.

[0011] The vectors of different words in the policy text are averaged, and the policy text is represented as a 512-dimensional text vector. A fully connected layer + softmax is used to map the text vector to a probability vector prop. The fully connected layer + softmax is set to contain a weight matrix w, whose dimension is set to R 512×(N+1) , where the prop vector dimension is set to N+1 dimensions, and the value of each dimension of the prop vector represents the probability that the policy text belongs to the review category represented by this dimension;

[0012] The cross entropy loss function is used to calculate the loss value of the predicted result probability and the true result of the pre-trained language model. The loss value of the predicted result probability and the true result is set as loss, and back propagation is used to adjust the parameters of the pre-trained language model according to the formula:

[0013] x=(x1,x2...x L ),y=(0....1....0)

[0014] hidden=Bert(x),hidden∈R L×512

[0015] text=average(hidden),text∈R 1×512

[0016] prop=softmax(w·text),w∈R 512×(N+1) ,prop∈R 1×(N+1)

[0017]

[0018] The model is trained on the input policy text according to the above formula until the loss value stops decreasing, and a policy text review category classification model is built.

[0019] S4: Conduct preliminary screening and review of policy texts based on training analysis results, compile statistics on the fairness review results of different policy texts, and iterate information on the constructed model.

[0020] Further setting: In step S1, the classified policy text is divided into subcategories according to release time, priority, newly added records and text validity, and the following steps are also included:

[0021] S1-1: The release time of the classified policy documents is sub-classified by year, and the policy documents of each year are sub-classified according to the release priority;

[0022] S1-2: Screen duplicate policy text data from different years and determine whether there are new records for the duplicate policy text data. If there are new records for the duplicate policy text data, mark the policy text data with the earlier year as historical record data.

[0023] S1-3: Review the validity of policy text data, obtain the release time of different policy texts, focus on marking and reviewing policy texts whose release time is greater than the set threshold, count the browsing data of different policy texts in the government policy text information database, analyze the average value of browsing data of different policy texts, set the average value of browsing data of different policy texts as R(f), set the browsing data of a certain policy text as R(0), when Conduct a focused review of the policy text.

[0024] Further settings: In step S2, extract policy text without expert annotation data, perform pre-training, and build a pre-trained language model. The pre-trained model structure adopts the BERT model structure, and further includes the following steps:

[0025] S2-1: Collect any policy text, mask any characters within the collected policy text, input the policy text with masked characters into the pre-trained language model, and the pre-trained language model uses the policy text context to predict the masked characters;

[0026] S2-2: Calculate the prediction accuracy of the pre-trained language model and compare it with the set threshold. If the accuracy is lower than the set threshold, adjust the strategy parameters of the pre-trained language model and continue training. If the accuracy is higher than or equal to the set threshold, stop training.

[0027] S2-3: Repeat the training cycle for policy texts where the pre-trained language model cannot predict masked characters until accurate prediction is achieved.

[0028] Further setting: In step S3, until the loss value stops decreasing, building the policy text review category classification model also includes the following steps:

[0029] The policy texts to be reviewed are classified and screened according to the policy text review category classification model. The policy text review category classification model outputs the probability of the review standard category to which each government text belongs, and the probabilities of different review standard categories to which the government text belongs are counted. The probability of each review standard category is sorted in descending order according to the size of the probability, and the review standard category with the largest probability is extracted as the review category to which the government text belongs. At the same time, the government text is screened to see if it violates the review standards, and the government texts that violate the review standards are highlighted.

[0030] Further configuration: In step S4, the policy texts are preliminarily screened and reviewed based on the training analysis results, the fairness review results of different policy texts are counted, and the information of the constructed model is iterated, which also includes the following steps:

[0031] S4-1: Obtain the review categories of different government documents, eliminate government documents that violate the review standards, and form a preliminary screening statistical list. This list is sent to the manual review channel, and experts review the preliminary screening results within the preliminary screening statistical list;

[0032] S4-2: Count the review results of different government documents to form a final screening statistical list, and upload the final screening statistical list as the government document fairness review result to the government policy text information database;

[0033] S4-3: The final screening statistical list is simultaneously entered into the review sample library, and the government text input pre-trained language and policy text review category classification model annotated within the review sample library is regularly and arbitrarily called for re-pre-training and review training, and model information iterative training is performed.

[0034] Compared with the existing technology, the beneficial effects of the present invention are: the present invention aims to construct a fair competition review auxiliary technical method to solve the problems of low accuracy and lack of semantic understanding of the original similarity model, build a model based on government policy documents to train data in real time, and obtain the semantic features of the text through autonomous learning of the text. The model is used to classify and review the government policy documents to be reviewed, assist experts in making judgments, and reduce the labor cost of the review. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments in conjunction with the accompanying drawings.

[0036] Figure 1 This is a schematic diagram of the steps of a fair competition review auxiliary method based on a text classification model of the present invention;

[0037] Figure 2 This is a schematic diagram of the specific steps of S1 in a fair competition review auxiliary method based on a text classification model of the present invention;

[0038] Figure 3 This is a schematic diagram of the specific steps of S2 of a fair competition review auxiliary method based on a text classification model of the present invention;

[0039] Figure 4 This is a schematic diagram of the specific steps of S3 of a fair competition review auxiliary method based on a text classification model of the present invention;

[0040] Figure 5 This is a schematic diagram of the specific steps of S4 of a fair competition review auxiliary method based on a text classification model of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] See also Figures 1 to 5 In an embodiment of the present invention, a fair competition review auxiliary method based on a text classification model is provided:

[0043] S1: Build a government policy text information database, obtain policy texts issued by the government, tag different policy texts with keywords, classify them according to different keywords, and subclassify the classified policy texts according to release time, priority, newly added records and text validity;

[0044] Specific reference Figure 2 ,In the above steps, the classified policy texts are divided into subcategories according to ,release time, priority, newly added records and text validity, ,and also include the following steps:

[0045] S1-1: The release time of the classified policy documents is sub-classified by year, and the policy documents of each year are sub-classified according to the release priority;

[0046] S1-2: Screen duplicate policy text data from different years and determine whether there are new records for the duplicate policy text data. If there are new records for the duplicate policy text data, mark the policy text data with the earlier year as historical record data.

[0047] S1-3: Review the validity of policy text data, obtain the release time of different policy texts, focus on marking and reviewing policy texts whose release time is greater than the set threshold, count the browsing data of different policy texts in the government policy text information database, analyze the average value of browsing data of different policy texts, set the average value of browsing data of different policy texts as R(f), set the browsing data of a certain policy text as R(0), when Conduct a focused review of the policy text.

[0048] S2: Collect different policy texts, screen them for expert-annotated data, classify them into those with and without expert-annotated data, extract policy texts without expert-annotated data, perform pre-training, and build a pre-trained language model. The pre-trained model structure adopts the BERT model structure.

[0049] Specific reference Figure 3 In the above steps, policy texts without expert annotation data are extracted, pre-trained, and a pre-trained language model is constructed. The pre-trained model structure adopts the BERT model structure, and the following steps are also included:

[0050] S2-1: Collect any policy text, mask any characters within the collected policy text, input the policy text with masked characters into the pre-trained language model, and the pre-trained language model uses the policy text context to predict the masked characters;

[0051] S2-2: Calculate the prediction accuracy of the pre-trained language model and compare it with the set threshold. If the accuracy is lower than the set threshold, adjust the strategy parameters of the pre-trained language model and continue training. If the accuracy is higher than or equal to the set threshold, stop training.

[0052] S2-3: Repeat the training cycle for policy texts where the pre-trained language model cannot predict masked characters until accurate prediction is achieved.

[0053] S3: Have experts annotate the collected policy texts, identify the expert-annotated data within the policy texts, classify the extracted policy texts according to whether they violate the review standards or not, and enter the classified policy texts into the review sample library according to their categories;

[0054] S3-1: Set a policy text x as the input to the pre-trained language model, the length of the text is L, and the category is y. Convert the input parameters to a one-hot vector representation, set the model output dimension to N+1, and set the dimension value corresponding to category y to 1. The pre-trained BERT model maps each word of the policy text to a 512-dimensional high-dimensional vector space. The policy text is then converted into an L×512 hidden vector through the BERT model.

[0055] S3-2: Average the vectors of different words in the policy text, represent the policy text as a 512-dimensional text vector, and use a fully connected layer + softmax to map the text vector to a probability vector prop. Set the fully connected layer + softmax to contain a weight matrix w, whose dimension is set to R 512×(N+1) , where the prop vector dimension is set to N+1 dimensions, and the value of each dimension of the prop vector represents the probability that the policy text belongs to the review category represented by this dimension;

[0056] S3-3: Use the cross entropy loss function to calculate the loss value of the predicted result probability and the true result of the pre-trained language model, set the loss value of the predicted result probability and the true result as loss, and use backpropagation to adjust the parameters of the pre-trained language model according to the formula:

[0057] x=(x1,x2...x L ),y=(0....1....0)

[0058] hidden=Bert(x),hidden∈R L×512

[0059] text=average(hidden),text∈R 1×512

[0060] prop=softmax(w·text),w∈R 512×(N+1) ,prop∈R 1×(N+1)

[0061]

[0062] The model is trained on the input policy text according to the above formula until the loss value stops decreasing, and a policy text review category classification model is built.

[0063] It should be specifically explained that in step S3-3, the policy texts to be reviewed are classified and screened according to the policy text review category classification model. The policy text review category classification model outputs the probability of the review standard category to which each government text belongs, and the probabilities of the different review standard categories to which the government text belongs are counted. The probabilities are sorted in descending order according to the size of the probability of each review standard category, and the review standard category with the largest probability is extracted as the review category to which the government text belongs. At the same time, the government text is screened to see if it violates the review standards, and the government texts that violate the review standards are highlighted.

[0064] S4: Conduct preliminary screening and review of policy texts based on training analysis results, compile statistics on the fairness review results of different policy texts, and iterate information on the constructed model.

[0065] Specific reference Figure 5 , the above steps further include the following steps:

[0066] S4-1: Obtain the review categories of different government documents, eliminate government documents that violate the review standards, and form a preliminary screening statistical list. This list is sent to the manual review channel, and experts review the preliminary screening results within the preliminary screening statistical list;

[0067] S4-2: Count the review results of different government documents to form a final screening statistical list, and upload the final screening statistical list as the government document fairness review result to the government policy text information database;

[0068] S4-3: The final screening statistical list is simultaneously entered into the review sample library, and the government text input pre-trained language and policy text review category classification model annotated within the review sample library is regularly and arbitrarily called for re-pre-training and review training, and model information iterative training is performed.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A fair competition review assistance method based on a text classification model, characterized by: S1: Build a government policy text information database, obtain policy texts issued by the government, tag different policy texts with keywords, classify them according to different keywords, and subclassify the classified policy texts according to release time, priority, newly added records and text validity; S2: Collect different policy texts, screen them for expert-annotated data, classify them into those with and without expert-annotated data, extract policy texts without expert-annotated data, perform pre-training, and build a pre-trained language model. The pre-trained model structure adopts the BERT model structure. S3: Have experts annotate the collected policy texts, identify the expert-annotated data within the policy texts, classify the extracted policy texts according to whether they violate the review standards or not, and enter the classified policy texts into the review sample library according to their categories; Assume that there are N review standards within the review sample library, add the option of not violating all review standards, set the review sample library to include N+1 review categories, set a policy text x as the input of the pre-trained language model, set the text length to L, set the category y, convert the input parameters to one-hot vector representation, set the model output dimension to N+1, and set the dimension corresponding to category y to 1. The pre-trained BERT model maps each word of the policy text to a 512-dimensional high-dimensional vector space. The policy text is processed by the BERT model and converted into an L×512 hidden vector. The vectors of different words in the policy text are averaged, and the policy text is represented as a 512-dimensional text vector. A fully connected layer + softmax is used to map the text vector to a probability vector prop. The fully connected layer + softmax is set to contain a weight matrix w, whose dimension is set to R 512×(N+1) , where the prop vector dimension is set to N+1 dimensions, and the value of each dimension of the prop vector represents the probability that the policy text belongs to the review category represented by this dimension; The cross entropy loss function is used to calculate the loss value of the predicted result probability and the true result of the pre-trained language model. The loss value of the predicted result probability and the true result is set as loss, and back propagation is used to adjust the parameters of the pre-trained language model according to the formula: x=(x1,x2...x L ),y=(0....1....0) hidden=Bert(x),hidden∈R L×512 text=average(hidden),text∈R 1×512 prop=softmax(w·text),w∈R 512×(N+1) ,prop∈R 1×(N+1) According to the above formula, the model is trained on the input policy text until the loss value stops decreasing, and a policy text review category classification model is built; S4: Conduct preliminary screening and review of policy texts based on training analysis results, compile statistics on the fairness review results of different policy texts, and iterate information on the constructed model.

2. The fair competition review auxiliary method based on a text classification model according to claim 1 is characterized in that In step S1, the classified policy texts are divided into subcategories according to release time, priority, newly added records and text validity, and the following steps are also included: S1-1: The release time of the classified policy documents is sub-classified by year, and the policy documents of each year are sub-classified according to the release priority; S1-2: Screen duplicate policy text data from different years and determine whether there are new records for the duplicate policy text data. If there are new records for the duplicate policy text data, mark the policy text data with the earlier year as historical record data. S1-3: Review the validity of policy text data, obtain the release time of different policy texts, focus on marking and reviewing policy texts whose release time is greater than the set threshold, count the browsing data of different policy texts in the government policy text information database, analyze the average value of browsing data of different policy texts, set the average value of browsing data of different policy texts as R(f), set the browsing data of a certain policy text as R(0), when Conduct a focused review of the policy text.

3. The fair competition review auxiliary method based on text classification model according to claim 1 is characterized in that In step S2, policy text without expert annotation data is extracted, pre-trained, and a pre-trained language model is constructed. The pre-trained model structure adopts the BERT model structure, and the following steps are also included: S2-1: Collect any policy text, mask any characters within the collected policy text, input the policy text with masked characters into the pre-trained language model, and the pre-trained language model uses the policy text context to predict the masked characters; S2-2: Calculate the prediction accuracy of the pre-trained language model and compare it with the set threshold. If the accuracy is lower than the set threshold, adjust the strategy parameters of the pre-trained language model and continue training. If the accuracy is higher than or equal to the set threshold, stop training. S2-3: Repeat the training cycle for policy texts where the pre-trained language model cannot predict masked characters until accurate prediction is achieved.

4. The fair competition review auxiliary method based on a text classification model according to claim 1 is characterized in that In step S3, until the loss value stops decreasing, building the policy text review category classification model further includes the following steps: The policy texts to be reviewed are classified and screened according to the policy text review category classification model. The policy text review category classification model outputs the probability of the review standard category to which each government text belongs, and the probabilities of different review standard categories to which the government text belongs are counted. The probability of each review standard category is sorted in descending order according to the size of the probability, and the review standard category with the largest probability is extracted as the review category to which the government text belongs. At the same time, the government text is screened to see if it violates the review standards, and the government texts that violate the review standards are highlighted.

5. The fair competition review auxiliary method based on text classification model according to claim 1 is characterized in that In step S4, the policy texts are preliminarily screened and reviewed based on the training analysis results, the fairness review results of different policy texts are counted, and the constructed model is iterated. The following steps are also included: S4-1: Obtain the review categories of different government documents, eliminate government documents that violate the review standards, and form a preliminary screening statistical list. This list is sent to the manual review channel, and experts review the preliminary screening results within the preliminary screening statistical list; S4-2: Count the review results of different government documents to form a final screening statistical list, and upload the final screening statistical list as the government document fairness review result to the government policy text information database; S4-3: The final screening statistical list is simultaneously entered into the review sample library, and the government text input pre-trained language and policy text review category classification model annotated within the review sample library is regularly and arbitrarily called for re-pre-training and review training, and model information iterative training is performed.

Images